• Version
• Download 10
• File Size 185 KB
• File Count 1
• Create Date October 5, 2021
• Last Updated October 5, 2021

D2.5 Formalization of human gestures in an interaction representation - ABSTRACT

This document reports the advancements developed in the context of WP2, describing the work conducted and the achievements accomplished during TUDA’s work on task T2.5 “Formalization of human gestures in an interaction representation”. Jointly with T2.4, this report closes the integration for Module #4: Primitives Learning.

The aim of WP2 is to deliver a knowledge base and develop robotic cognition. In particular, this knowledge base should contain accessible information regarding human and robot tasks, should contain tools for analysing the human-operator’s workspace to detect task-specific
objects, monitor human activity and predict human tasks’ evolution, while a primitive’s learning algorithm will provide the robot with the ability to learn from demonstrations.

During task T2.5, the problems associated with understanding and classifying human gestures have been addressed. Using the motion primitive models presented in T2.4, during T2.5, we have studied gesture recognition and human motion generation for three different scenarios. In particular, we have studied applying generative models for human motion classification and motion generation. First, using the Normalizing Flows models presented in T2.4, we have developed a modular human gesture classifier. We have focused in developing a modular, sample efficient and fast to learn gesture classifier to apply it as human-robot interaction framework.

Second, we have extended the ideas from static gestures classification to dynamic gestures classification. With static gestures, we refer to gestures that are represented only by poses, while dynamic gestures, we refer to gestures that are represented as pose trajectories in time. Learning density models for dynamic gestures is a hard problem given the high dimensionality of the data and the low amount of it. We have applied the motion primitives introduced in T2.4 to model the probabilistic gesture models and used Inverse Reinforcement Learning algorithms to enhance the extrapolation properties of our models. Given the inductive biases introduced through the model and the learning algorithm, we can learn sample efficient representation of the dynamic gestures.

Finally, we explored how to extend the gesture models to higher dimensions. When applying the gesture recognition modules, we can learn the gesture density in low-dimensional spaces such as the task space of the human operator. Instead, if we are interested in generating motion, for example, for predicting how the human will move, we require to represent the learned density models in the human’s joint space. We have studied how to extend motion primitives modelled in the task space to a higher dimensional joint space, to define the movement of the human’s body. For the real-time execution of the respective modules, we have integrated them to a particular ROS package with the respective nodes. Those can be easily integrated to the core system of Sharework.

All above contributions are detailed in dedicated sections of this document. Further technical information is available in relevant disseminated project results, attached as papers in the Appendix.