As gesture control represents a seamless interface between humans and machines, more and more machines, robots and devices are able to respond to gestural cues. Researchers at Fraunhofer IOF in Jena are raising human-machine interaction to a new level: the high-speed 3D measurement and sensor technology developed in the 3D-LivingLab research project (see “3D-LivingLab” box) enables them to capture and interprete even complex movements – and does so in real time. At Hannover Messe 2019, the research team is demonstrating its gesture-based human-machine interaction technology using the example of a wall made up of 150 spheres, which copies in 3D every head, arm and hand movement of a person standing in front of it. The wall of spheres effectively imitates the body movements with contact-free 3D reactions in real time, free from irritating time lags. The wall of spheres was created as part of the “3D-LivingLab“ project.
Workflows are greatly simplified The system is made up of several modules: 3D sensor, 3D data processing and image fusion as well as the actuator system itself comprising 150 individual actuators. “The wall of spheres is not only a great toy, it also represents cutting-edge technology. Real-time 3D capture and interpretation of multiple gestures without tracking sensors can radically simplify workflows – from production scenarios to health and safety,” says Dr. Peter Kühmstedt, scientist and group leader at Fraunhofer IOF. The demonstrator system responds to the behavior of people, captures complex movements such as gestures and physical actions, and gives real-time feedback through a technical actuator system that converts electrical signals into movement on the wall of spheres. It is the person’s posture that controls the actuators. Specially developed algorithms enable human 3D movements to trigger control of the actuators, thus causing the spheres to move. “We are demonstrating very rapid measurement technology – the data is captured by a new generation of 3D sensors –, very rapid low-latency processing – the data is interpreted and converted immediately – and very rapid reactions in real time. According to the calculation results, the wall of spheres immediately mirrors the movement of the person in front of it,” says the researcher. In production environments, for example, the technology could be used to monitor a worker who is interacting with a robot and handing it parts. It could also be transferred to other application fields, such as health and safety, where it can make processes safer and more efficient. Other conceivable applications for the 3D sensor technology and the interaction components are in assembly assistance and quality control systems. Moreover, they also qualify for the monitoring of biometric access points.
