Human recognition with embedded AI

Human recognition
© Fraunhofer IMS
Human recognition


In the project "noKat" (development of a neural optical camera tracker for the detection of approaching persons), which is funded by the German Federal Ministry for Economic Affairs and Energy as part of the "Central Innovation Program for SMEs" (ZIM), Fraunhofer IMS is developing an optical proximity sensor together with the partner company van Rickelen GmbH & Co. KG. The optical proximity sensor that is able to detect approaching persons by means of artificial intelligence (AI) from RGB images of a low-cost camera. The system currently provides for stationary mounting of the camera, similar to the mounting of conventional motion detectors. The person detection algorithm runs fully integrated on an embedded system. A connection to a cloud is not planned in order to meet all data protection requirements. Thus, no image leaves the system, only control data such as "person detected". These can be used for further control in building management. The hardware is located in the low-cost segment and is particularly energy-saving. Through the use of novel feature extraction in combination with a very compact feedforward neural network, which is supported by the software framework "Artificial Intelligence for Embedded Systems (AIfES)“ has been realized, a system has been created that requires significantly fewer parameters than a comparable Convolutional Neural Network (ConvNet). Compared to the "EfficientDet-D7", "noKat" requires more than 99% fewer parameters and is thus significantly more performant on resource-limited systems such as microcontrollers. "noKat" recognizes and classifies people in a moving image, where several different objects can also be present in a moving image. First measurements on the target system (microcontroller with 160 MHz clock) showed that the algorithm needs only about 120 ms for an object classification at a resolution of 320x240. A presentation of the demonstrator is planned for the end of the project in November 2021.


Advantages of the system:

  • People are recognized
  • Fully integrated in an embedded system
  • No internet or cloud connection needed
  • No data or images leave the system
  • Low-cost and low-power hardware
  • Software also portable to other systems


van Rickelen GmbH & Co. KG

A publication of van Rickelen and GED:




Our applications - Examples of what we can do for you

Decentralized AI systems and AIfES platform

AI Framework, open roberta, Arduino

Federated learning for resource-constrained systems


Personalizable AI

Individually trainable gesture recognition

Our fields of application - Our expertise for you

Sustainable Production

  • Optimization of raw material and energy use
  • Use of alternative energy sources and energy-autonomous sensors
  • Green ICT

Mobile autonomous Manufacturing

  • Sensors / Control for Robots / Cobots
  • Industrial transport systems (AGV)
  • Human-Machine Interaction

Trustworthy Electronics

  • Protection against product piracy / Counterfeit-proof labeling
  • Tamper-proof and fail-safe electronics
  • Trustworthy supply chains

Industry (Home)

Click here to return to the overview page of the Industry business unit.