Handwriting recognition by ANN – enabled by AIfES

Nowadays, there are a wide variety of interfaces and methods that can be used to capture and recognize handwriting. With the help of the AIfES framework (Artificial Intelligence for Embedded Systems) developed by Fraunhofer IMS it is now possible to have a microcontroller recognize handwriting entered on a touchpad. Normally, handwriting recognition is a computationally intensive task that must be performed by a powerful PC. By using the extremely resource-saving AIfES library it is now possible to have such operations performed even on small microcontrollers or on the touchpad itself.

The character entered manually on the touchpad is captured in the form of the raw data generated. From this raw data, features are extracted by a software using an innovative patent-pending process which are then evaluated by a trained neural network. The training of the neural network is also done by AIfES on the microcontroller. The touchpad only sends the results, in this case the recognized character, to the computer. This effectively reduces data volumes, minimizes runtimes and, on the other hand, increases data security.

This is facilitated by the platform independence of AIfES which allows the framework to run on almost any microcontroller. This results in many degrees of freedom for the selection of suitable hardware. Instead of the usual keypads a simple and fast way of handwriting recognition is possible which can be used especially in the areas of smart home or smart city. Examples are the input fields of ATMs, security systems, telephones or even calculators. But also, an extended touchpad function of laptops can be implemented.

A first demonstrator by Fraunhofer IMS proves the functionality of handwriting recognition by the capturing of handwritten digits. An extension to the additional recognition of letters is possible as well as the recognition of more complex gestures for operating devices which are detected by suitable sensors. Fraunhofer IMS is happy to make its expertise and technology available for the development of new applications and products.

https://www.youtube.com/watch?v=vaHxPavwGKY

For the experimental setup shown here a standard PS/2 touchpad was used in combination with an Arduino Uno. The results of the handwriting recognition are shown on an LC display. Under these conditions a calculation time of about 20 to 25 ms could be achieved depending on the number of recorded measuring points and the length of the character string: Since, for example, the character string of an "8" is longer than that of a "1" and thus more measuring points are recorded, the evaluation of an "8" is more complex. To train the network only 360 digits were used in the form of 36 data sets of the numbers “0” to “9” entered by different persons. The recognition rate of the training data was already over 95% with this manageable data set and can be further optimized with a larger amount of training data.