Hybrid Learning – Data- and physics-based models for smart sensors

Overview of a physics-guided neural network to generate vibration data for defective ball bearings.
© Fraunhofer IMS
Overview of a physics-guided neural network to generate vibration data for defective ball bearings.

Recent advances in artificial intelligence (AI) are primarily characterized by deep learning techniques in computer vision. However, application fields such as intelligent condition monitoring in the context of Industry 4.0 (also called Smart Production) or smart wearable computing also show the potential for processing signals from other, non-imaging and multimodal sensors.

Especially in the area of technical systems challenges arise when applying modern machine learning (ML) methods. For example, large amounts of data are needed for training, prior knowledge about the system is often not included, and the "black-box" character of purely data-based models leads to a low degree of explainability.

Fraunhofer IMS addresses these challenges by means of hybrid learning methods in order to design intelligent sensor systems that are efficient, robust and safe. A special focus is placed on the integration of physics-based expert knowledge and machine learning methods. In hybrid learning expert knowledge is applied at various points in the machine learning pipeline, i.e. from training data generation to the final model.

In the area of condition monitoring, for example, comprehensive data acquisition is often too resource-intensive and uneconomical. Therefore, there is a lack of meaningful data sets for diagnostic and prognostic tasks that represent different scenarios and events such as fault cases in sufficient quality and quantity. This is where simulation-based hybrid learning methods can provide a remedy. In this context, Fraunhofer IMS is researching methods for combining physics- and AI-based models in order to generate training data that is as true to the original as possible.

In addition, a wide range of methods for the efficient analysis of sensor signals already exists in the field of classical digital signal processing. Fraunhofer IMS is working on using this kind of expert knowledge in signal preprocessing to ensure faster training of neural networks while reducing the amount of data required, e.g. for pattern recognition or signal classification.

Another possibility is the use of expert knowledge to control the training process of machine learning methods. In this context, Fraunhofer IMS is specifically working on methods that check predictions of an AI model with respect to their physical plausibility in order to exclude unwanted or physically inconsistent behavior of the system from the outset. This hybrid learning can thus significantly increase the explainability and trustworthiness of the system.

Our technologies - Innovations for your products

Distributed Learning

Distributed learning enables training of complex tasks on multiple small embedded systems.

Feature Extraction for Smart Sensors

By means of adapted feature extraction the size of a neural network can be reduced.

Embedded AI for LiDAR

Embedded AI can be used to accelerate and improve the quality of distance measurement by LiDAR sensors.

Industry 4.0 Research

Against the background of "Industry 4.0", Fraunhofer IMS is researching predictive maintenance solutions for the manufacturing industry.

Our technology areas - Our technologies for your development

Communication and Networking

Communication interfaces allow data exchange with other devices and connection to networks

User Interfaces

User interfaces as interface between device and user allow the configuration and operation of a product

Machine Learning for Embedded Systems

Artificial intelligence on resource-limited systems can be used to extract higher quality information from raw sensor data

Computer Vision

Computer Vision methods extract the maximum amount of information from image data


Embedded Software and Artificial Intelligence (Home)

Here you can get back to the overview page of the core competence Embedded Software and Artificial Intelligence (ESA).