Feature extraction for Smart Sensors

Sensors are one of the most important components in today‘s digital world. Just like the human senses, sensors are the means by which technical systems acquire information about the external processes and situations. However, raw sensor data are not noise-and disturbance-free. Raw sensor data contain a large amount of noise and distortion, posing a challenge to obtain useful information. Processing the raw data in the sensor itself turns it into a smart sensor.

An example of a complex smart sensor is the SPAD-based flash LiDAR system. As shown in figure 1 the distance information is represented by the point cloud data. However, the time-correlated histograms, which are used to infer the point cloud data, contain 90 % of noise. Classical analysis methods such as digital filters are inefficient and significantly degraded by the dominated noise. Feature extraction, introduced as the first processing stage by the machine learning algorithm, is one way to overcome the processing inefficiency and reduce the influence of irrelevant information. Typically, the feature extraction belongs to the black-box model and will be automatically learned during the training procedure. However, in many machine learning algorithms, e.g. convolutional neural network, the scale and the complexity of the feature extraction can be greater than the classification algorithm itself.

Fraunhofer IMS is developing lightweight feature extraction methods for different applications. Taking LiDAR histogram data as an example the feature extraction is governed by statistics. By providing the corresponding parameters, e.g. laser pulse width, desired operation range of the background light intensity and number of measurements per histogram, we can optimize the feature extraction for your system, so that even a small machine learning algorithm can be utilized on the extracted features and a robust distance information can be obtained. The aforementioned feature extraction is applied on the measurement dataset from the Fraunhofer IMS LiDAR system called “OWL” and the simulation result is qualitatively shown in figure 2. One can see from the greater similarity of the AI-based plot (right) to the ideal plot (left) that the AI-based method is significantly better than the conventional method (center) and produces fewer outliers.

Fraunhofer IMS provides the following possibilities to improve your LiDAR measurements:

  1. Customized feature extraction method to improve the measurement robustness under high ambient light
  2. Classification algorithms selection and evaluation (SVM, neural network, Bayesian theory, or decision tree)
  3. Real-time measurement quality analysis
  4. Time-correlated single-pixel correction and macro-pixel-based spatial correction
SPAD-based LiDAR System and Data Processing Workflow
© Fraunhofer IMS
SPAD-based LiDAR System and Data Processing Workflow
SPAD-Lidar point cloud
© Fraunhofer IMS
Comparison between simulation and standard and AI-based simulation and evaluation of SPAD LiDAR point cloud.

Our technologies - Innovations for your products

Distributed Learning

Distributed learning enables training of complex tasks on multiple small embedded systems.

Hybrid Learning and PGNNs

In case of insufficient training data simulations based on physical models help to improve the data base.

Embedded AI for LiDAR

Embedded AI can be used to accelerate and improve the quality of distance measurement by LiDAR sensors.

Industry 4.0 Research

Against the background of "Industry 4.0", Fraunhofer IMS is researching predictive maintenance solutions for the manufacturing industry.

Our technology areas - Our technologies for your development

Communication and Networking

Communication interfaces allow data exchange with other devices and connection to networks

User Interfaces

User interfaces as interface between device and user allow the configuration and operation of a product

Machine Learning for Embedded Systems

Artificial intelligence on resource-limited systems can be used to extract higher quality information from raw sensor data

Computer Vision

Computer Vision methods extract the maximum amount of information from image data


Embedded Software and Artificial Intelligence (Home)

Here you can get back to the overview page of the core competence Embedded Software and Artificial Intelligence (ESA).