The Sharework project endowed an industrial work environment of the necessary “intelligence” and methods for the effective adoption of Human-Robot Collaboration (HRC) with no fences. The project has developed a software and hardware modular system capable of understanding the environment and human actions through knowledge and sensors, future state predictions, smart data processing, augmented reality and gesture and speech recognition technology to make robots overcome human barriers and ensure a more effective cooperation. Sharework has been applied to four types of real industrial scenarios in the automotive, railway, metal, and capital goods manufacturing industries.
ALSTOM components addressed in Sharework.
In this use-case, the completely manual assembly process has been replaced by a HRC process powered by the Sharework developed modules. To achieve this transformation, Eurecat has led, as the technical coach, all the required implementations and module integrations.
This use-case departs from the initial manual procedure. In order to assemble the parts, the operator must perform an ordered series of tasks manually. The process starts by placing the two lateral parts in the supports. For each corner, the operator cleans the surface, applies silicone, and spreads the applied silicone. After that, the remaining metal parts are placed on top of the already loaded ones. The top metal parts are fixed (in addition to the glued silicone fixation) by introducing and riveting the rivets in each corner. Finally, the operator performs quality control to verify the assembly.
The following critical aspects of the process tasks have been identified:
- Big range of working areas.
- Quality application of structural rivets.
- High repeatability.
- Operator injuries caused by the riveting tool (5kg tool height, impact on riveting, high noise when rivets break).
- Special process, the operator must have a certification.
Hence, the process transformation has focused on addressing these critical aspects by means of HRC. A safe collaboration between operator and robot has been achieved. In Alstom use-case scenario, the robot performs most of the critical tasks, leaving to the operator the most valuable tasks and feeling safe around the robot.
With this collaboration, the tasks are distributed in a way that no longer requires a completely linear execution of the process, but now allows a certain freedom in the order some of the tasks are carried out.
Task distribution between the human and the robot in ALSTOM demonstrator.
To implement the demonstrator, a hardware and a software architecture has been designed, including cameras, and a robot equipped with a two-head end effector, a riveting gun, and a silicone gun.
Hardware used in Alstom demonstrator placed in Eurecat facilities.
On the software side, seven Sharework modules have been integrated with three task-specific modules.
TASK SPECIFIC SHAREWORK MODULES
|Knowledge base||Represents the information flow of the entire system. It contains updated information about the use-case, the task-diagram flow, the location of each task in the use-case, and whether the task must be executed by the worker or the robot.|
|Human tracking||Tracks in real time the position of the operator, to identify the corner where the operator is working, and the body posture of the worker, to avoid collisions.|
|Human task identification||Recognizes the task the human is performing. This allows to trigger the “Human-Aware dynamic planning and scheduling for HRC tasks” module and plan the next action that the robot should perform according to the human actions.|
|Human-aware dynamic planning and scheduling for HRC tasks||Triggers and monitors the execution of a set of tasks, given the updated information within the knowledge base.|
|Offline and real-time human-aware and safe robot motion planning||Plans synchronized trajectories for the robotic arm and the XY linear axes so that the robot can apply silicone in the desired corner and with the required application path, while considering the presence of the human in the workspace. It also plans the trajectories for the robotic system to be able to rivet all the rivets at each corner.|
|Direct and natural human-system and system-human communication||Customized tablet HMI facilitates communication between the operator and the system through some intuitive buttons to control some operations and display some useful information. Allows the worker to notify the system that the process has started and that some points in the task execution have been reached, to stop and resume the robot actions and receive error messages.|
|Continuous evaluation of human ergonomics and posture correction||Monitors the posture of the human operators, evaluating the ergonomics and providing feedback and recommendations.|
|ROS Controller for the riveting gun and the silicone gun||ROS service devoted to activating and deactivating the flow of silicone at the silicone gun and, on the other hand, to trigger the riveting at the riveting gun.|
|ROS Controller for the linear axes||ROS interface to allow the independent control of the two linear axes of the workstation trough ROS action services and integrate them into the MoveIt framework.|
|Task-in-zone recognition||Module that combines two sources of knowledge: the position of the operator (provided by the Human Tracking module) and the task that is being performed by the human (provided by the Task Identification module). This allows to know where has been performed a task that should be applied in more than one corner.|
The demonstrator was tested at the Alstom production plant by different workers. First, a training to the Alstom operators was performed. In this training the different tasks the operator was supposed to perform and the related actions the operator would have to expect from the robot were explained.
During the tests, the new process performance metrics, together with the human factors’ evaluations were assessed. For the human factors’ evaluation, a questionnaire was conducted, and the physical condition of the worker (such as the cardiac frequency) was monitored. The test was conducted in laboratory conditions to ensure repeatability of the experiment. No silicone was used during the scenarios and the rivets were made of plastic so that the sealing and the riveting operations could be easily undone to allow repetitive experiments.
Operator being recognized by the ”Task-in-zone recognition” module.
On the human factors side, good feedback has been obtained from the operators. They commented that it improves ergonomics and reduces physical effort. As far as coexistence with the robot is concerned, operators always understood the robot’s behavior, and they knew what the next step was. This gave them a sense of security while the application was running.
On the process performance side, the measured KPIs have shown a great improvement of the process. For example, the cycle time has been reduced from 20 minutes to 12 minutes and the time the worker must perform repetitive and/or high-load tasks have been cut by half. Furthermore, thanks to the intuitive interfaces and the comprehensive behavior of the robotic system, the human operators only need 16 minutes to be trained to use the system.
The improvement of the process has been notorious considering the assessment of KPIs thanks to the Sharework project. On the one hand, the human worker conditions have been significantly improved, especially by reducing the poor ergonomic tasks such as the silicone application and riveting. On the other hand, the process performance has improved, as can be seen from the obtained metrics.
However, the tests performed have given some useful insights to keep improving the new process. These improvements would help the application to evolve even further and make the operator’s work even safer, such as picking up the remains of the riveting to avoid possible stumbles and warning by acoustic signal when the robot starts to move.
Dr. Néstor García received the B.S. degree (with honors) in Industrial Engineering and the Ph.D. degree (also with honors) in Automatic Control, Robotics and Computer Vision, both from the “Universitat Politècnica de Catalunya” (UPC), Barcelona, Spain, in 2015 and 2019 respectively. He worked as a Research Assistant at the Institute of Industrial and Control Engineering (IOC) of the UPC, being involved in different R&D projects. He joined Eurecat in 2018, where he is the Principal Investigator of the Collaborative Manipulation research group. His current research interests are focused on task and motion planning, human-robot interaction, multirobot cooperation and learning in robotic systems. He is acting Eurecat technical lead for the European projects Sharework, Bots2Rec and national project Simbiots.
Magí Dalmau Moreno received the Industrial Engineering degree from the “Universitat Politècnica de Catalunya” (UPC), in 2019; and MSc in Intelligent Interactive Systems from “Universitat Pompeu Fabra” (UPF), in 2022.
He worked as a Research Assistant at the Institute of Industrial and Control Engineering (IOC) of the UPC, being involved in different R&D projects. He joined Eurecat in 2019, where he is Robotics Researcher within the Collaborative Manipulation research group and has actively participated in the research and developments of public (European and national) and private projects. His current research interests are focused on AI task and motion planning, RL-based robot behaviors generation, and human-robot interaction.
Eric Domingo Roca has studied a degree in Computer Engineering at “Universitat Politècnica de Catalunya” (UPC) and Universitat Oberta de Catalunya (UOC). He joined Eurecat in 2019 in the Industrial Robotics group where he carried out a series of projects focused on the integration of automation systems. He currently works as a Robotics Researcher in the Collaborative Manipulation group and has actively participated in the research and developments of public (European and national) and private projects. His current research interests are focused on the development of human-robot interfaces.
Eurecat is the leading Technology Centre of Catalonia. It provides the industrial and business sector with differential technology and advanced expertise, offers solutions to their innovation needs and boosts their competitiveness in a fast-paced environment. With a vast expertise in industrial robotics, Eurecat is Sharework project’s coordinator and is responsible for the development of the human tracking modules and the preliminary system integration.