Sensor data as point cloud for environment detection
© Adobe Stock - AndSus
Sensor data as point cloud for environment detection.

Environment detection

Application-specific environmental sensor technology for increased automation on the water, in the air and on the road.

Environment detection

In order to make road traffic safer, create new intermobility solutions, and to conquer the urban areas with flying drones, new advances in reliable environment recognition research are needed.

On the one hand, powerful sensors are required that have high measuring ranges, often together with large fields of view, precise spatial resolution and fast acquisition times. On the other hand, sensor data must be processed and fused with other sensor data. However, sensor solutions must always be affordable and easy to integrate for the respective application and have low power consumption. Fast and reproducible test and calibration procedures for environment detection are becoming increasingly important in order to check the sensor behavior as realistically as possible and to improve it subsequently if necessary.

In the four categories of powerful sensor systems, sensor fusion, system integration and validation the Fraunhofer IMS develops application-specific solutions.

Compact LiDAR detector CSPAD αlpha with integrated readout electronics.
© Fraunhofer IMS
Representation of a sensor chip lying on a finger.

Powerful sensor technology

A new era of scalable and light-sensitive LiDAR detectors, the CSPADs, was launched in early 2021. CMOS Single-Photon Avalanche Diodes from Fraunhofer IMS are highly integrated, smart photodiode arrays with low power consumption and simultaneously high detection sensitivity of objects, even in difficult weather conditions.

With a pixel resolution of 32x24 and a readout speed of 26 fps, the CSPAD αlpha is a particularly compact solution on a 10x10 mm² chip area. The sensor, suitable for Flash LiDAR solutions (see video), allows the creation of small 3D cameras for environment detection with low weight, which is a decisive criterion especially for applications such as flying drones.

Based on the 3D integration technology demonstrated here, an available process developed in the in-house cleanroom infrastructure, further customized solutions with increased pixel resolution, higher sensitivity to near-infrared light (NIR) and more flexible pixel readout are now being developed.

Fraunhofer IMS is also continuously breaking new ground in the field of thermal radiation detection (far infrared wavelength, or FIR). Manufactured microbolometer arrays (IRFPAs) can detect heat with a resolution down to 50 mK. For more cost-effective FIR cameras for mobility applications, as environment detection, our new prototypes save 50% in chip area for the same number of pixels.

 

Schematic of 3D-integration process
© Fraunhofer IMS
Fraunhofer IMS’ developed BSI (Backside Illuminated) process to fabricate scalable LiDAR SPAD detectors on 8-inch wafer substrates.
Microscopic image of microbolometer pixels
© Fraunhofer IMS
Top view of microbolometer array for detection of thermal radiation.

Sensor data fusion

Reliable redundancy in the data acquisition of the environment requires different or even technically independent sensors that monitor each other or can serve as a substitute if necessary. The high volume of data generated and its processing in real time is currently a major challenge.

The Fraunhofer IMS deploys so-called "early sensor fusion" to solve this problem directly during the raw data generation process. Data bundles are reduced to their smallest possible size before they are used for object recognition, without losing relevant information of the scene. In addition to classical algorithms, the use of machine learning methods also plays a significant role.

Also relevant is the generation and use of smart measurement procedures, in which only relevant scene areas are adaptively recorded. The design of the LiDAR detectors therefore already integrates these functions for smart control of data acquisition.

In the current project ALBACOPTER, the topic of early sensor fusion is realized in the form of LiDAR target tracking through a special sensor fusion. Fraunhofer IMS supports the development of novel and safe flight controls within the ALBACOPTER project, an interdisciplinary project with five other Fraunhofer institutes. The ALBACOPTER is an experimental platform with the goal of developing innovative technology approaches that combine the advantages of agile multicopters with the efficiency of gliders.

Illustration of a sensor fusion of radar and LiDAR sensor data for object detection.
© Fraunhofer
Multicopter ALBACOPTER with active LiDAR tracking, a fusion of radar and LiDAR sensor data
Hatched illustration of a compact sensor housing presented on an open hand
© Fraunhofer IMS
Smallest designs of 3D sensors by developing compact integration solutions.

Systemintegration

The multitude of sensor systems necessary for the application compete in the search for a suitable integration position in the complete system. The Fraunhofer IMS is therefore initially focusing on area-reduced detectors, compact system solutions and new forms of integration with other sensors.

In the current project SMART HEADLIGHT, an interdisciplinary project with four other Fraunhofer institutes, our LiDAR detectors for environment detection are combined with radar and lighting technology in a headlamp. Design limitations, heat generation and also aspects of performance and scalability of the individual sensors play a major role in this development for integration in the smallest possible space.

Validation

The testing of optical sensors for environment detection and integrated solutions is an important part of new sensor developments. The validation of prototypes, the comparison with other or competing approaches (benchmarking), up to the end-of-line test in production are only a few examples where reproducible and fast testing is necessary. The Fraunhofer IMS supports its customers from the set-up of first test benches, the generation of specific simulation environments, up to the research of new verification and calibration methods.

For optical sensors, we offer opto-electrical testing at wafer level (wafer size 8-inch) or in device form, with the special support that parameters such as the dark count rate can also be tested automatically.

Next Level Photonics

Novel methods for generating and detecting light together with our partner Kyoto University

CSPAD αlpha

Application video of the CSPAD - alpha

Flash LiDAR Camera Owl

Fast and reliable distance measurements with the Owl LIDAR camera system

Lighthouse project ALBACOPTER

Lighthouse project ALBACOPTER

Our technologies - Innovations for your products

Trusted Electronics for Urban Air Mobility

We are researching and developing new technologies in the field of trusted electronics to enhance safety in the Urban Air Mobility sector.

Our fields of application - Our expertise for you

Emission-free mobility

Emission-free locomotion and efficient transport for a sustainable mobility.  With its solutions, the Fraunhofer IMS enables long-lasting, optimized and safe operation of new powertrain systems.

 

Mobility (Home)

Click here to return to the overview page of the Mobility business unit.