Key Factor Computer Vision: The Role of Smart Cognition Systems in Industry 4.0

From Manual Work to Automation

Imagine a line worker picking and placing parts. This image is not unusual in certain industry fields in Europe. Those jobs are subject to high automation potential. The European Commission projects up to 50% of all jobs to be possibly automated in the future. Now, imagine the line worker to be blind and numb. This is the state of a robotic system, if no sensors are applied. Given a perfectly repeatable process, this isn’t even a problem. The robot could perform the same picking task over and over. In reality, the fewest processes are optimal to this degree.

Taking Cognition to the Next Level

While a picking task is pretty simple and constrained, managing an extensive process requires a completely different level of effort. Speaking of Industry 4.0, permanent surveillance of all process parameters is key to control the digitalized process. Most features can be assessed using local sensor systems that measure certain parameters directly. However, what if the process requires tasks outside the typical line structure?

Exemplary Chinese logistics expert Alibaba deploys a grid of tags that serves as localization solution for hundreds of mobile robots. However, an important feature of Industry 4.0 is the flexibility in the plant design. A fixed pathing solution means sacrificing future changes to the current process. Thus, flexible systems become more and more important in plant design and modern manufacturing.

As demonstrated commonly by household vacuum robot systems, modern mobile robots can already detect their environment autonomously using Computer Vision. While these robots come as an all-inclusive system, the demands in industry are different. In order to adapt to all possible scenarios, the cognition system is not only required to detect the environment, but also all the humans, robots and process elements within it.

SHAREWORK Smart Cognition SYSTEM

SHAREWORK is concerned with implementing a highly adaptive framework for task recognition and planning. The process information required for these tasks comes through Smart Cognition, the so called Semantic Mapping. Semantic Mapping is an abstract representation of the relation of objects with their respective environment.

In the SHAREWORK cognition system – to which RWTH Aachen University is contributing – a network of cameras is distributed in the respective working environment. On the video images generated by this network, objects are detected using Artificial Intelligence. One famous approach for this is YOLO (“You only look once”), which performs mass classification in real time. Using the type of object and its position in space, the underlying task description can be used to decide which task is currently performed and how far it has progressed.

An example: In the Goizper use case a ring element has to be screwed onto an assembly. In the initial state of this process step the ring element is located on the assembly without any screws. Through the cognition system, assembly and ring are detected with their respective center of mass (the mean of their estimated volume). From the relation of the centers we can come to the conclusion that the ring is located on the assembly. In addition, we could also check, if a screw driver is located close to a human worker’s hand. Both aspects are indications for the task to be performed, respectively the start of the screwing process.

Right before tightening the screws, a similar approach can be used to assess, whether all screws are in their respective spot. However, there is one important aspect of Computer Vision: The smaller the objects, the higher the uncertainties and the higher the uncertainties, the higher the possibility of erroneous estimates. Therefore, in SHAREWORK we aim to combine cognition with checks, performed by the human worker. 

This is where human-robot-collaboration becomes a major driver: Overcoming human limits and maintaining process control.

---