Robots weblog | Robotic automated pick-and-place processes with AI-based picture processing

Pick-and-Place applications are a central area of application for robotics. They are often used in industry to accelerate assembly processes and reduce manual activities – an exciting topic for computer science masters of the Institute for Data -optimized production of the Kempten University. They developed a robot that optimized processes through the use of artificial intelligence and computer vision. Based on an assembly drawing, the system is able to grab individual components and to put them off at a given point – comparable to a puzzle. Then the parts can be glued manually there by an employee.

Pick-and-Place robots placed components similar to a puzzle. (© Kempten University)

Two IDS industrial cameras provide the necessary image information

With the help of two Ueye XC cameras and a AI-based image processing, the system analyzes the environment and calculates precisely recording and storage coordinates. One of the cameras was placed above the work surface, the other above the withdrawal point. Specifically, a AI pipeline processes the pictures of the two cameras in several steps to determine the exact location and alignment of the objects. With the help of the computer vision algorithms and neuronal networks, the system recognizes relevant features, calculates the optimal gripping points and generates precise coordinates for the absorption and storage of the objects. In addition, the system clearly identifies the parts by segmenting their surface and comparing the contours with a database. In addition, it uses the results to enable an approximation to already stored parts. The automation solution thus reduces the dependence on expert knowledge, shortens process times and counteracts the shortage of skilled workers.

Robot grabs individual components and positions them at a given place. (© Kempten University)

Camera requirements

Interface, sensor, size and price were the criteria that were decisive for the choice of the camera model. The Ueye XC combines the user -friendliness of a webcam with the performance of an industrial camera. It only requires a cable connection for operation. Equipped with a 13-MP-On-Semi sensor (AR1335), the autofocus camera delivers high-resolution images and videos. An interchangeable macro attachment lens enables a shortened object distance, which means that the camera is also suitable for close-range applications. Their integration was also very simple, such as Raphael Seliger, research assistant at Kempten University of Applied Sciences, explains: “We tie the cameras to our Python Backend via the IDS Peak interface.”

Two Ueye XC industrial cameras provide the necessary image information. (© Kempten University)

outlook

In the future, the system is to be further developed by re -forcement learning – a method of machine learning based on learning through experiments and errors. “We would like to expand the AI functions to make the pick-and-place processes more intelligently. Under certain circumstances, we need an additional camera directly on the robot arm,” explains Seliger. An automatic accuracy check of the stored parts is also planned. In the long term, the robot should be able to carry out all necessary steps independently based solely on the assembly drawing.

Information on Ueye XC cameras