Fraunhofer Institute, Lamarr Institute, igus and University of Bonn are developing language models for cost-effective automation solutions in order picking
Cologne, October 1, 2024 – A hackathon took place at igus in Cologne in mid-September. Under the motto “Artificial Intelligence meets Robotics”, 17 participants worked in three interdisciplinary teams to develop advanced automation solutions for logistics. The goal of the hackathon was to create an automated method for packaging pizzas using a large AI-based language model (Large Language Models, or LLM). The event took place in collaboration with the Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS, the University of Bonn, the Lamarr Institute and the Fraunhofer Institute for Material Flow and Logistics IML.
20 times Tonno, 10 times salami and 25 times Margherita: This is what the supermarket around the corner orders from a large manufacturer of frozen pizzas. This sounds like a simple task, but it is still manual work that is currently done manually. A simple and undemanding job for which there is often no staff. 17 participants from the Fraunhofer Institutes IAIS and IML, the Lamarr Institute for Machine Learning and Artificial Intelligence and employees from the University of Bonn developed a solution for this during a 5-day hackathon at igus in Cologne. And with the help of kosteneffizienter „Low Cost Robotic“ made in Cologne. The task of the hackathon was: Develop an automation solution that automatically packs the pizzas using artificial intelligence (AI). The challenge was to control a robot so that it recognizes, grabs and packs the right products into mixed packages using voice instructions directly from employees. By combining generative AI (GenAI) and robotics, a solution should be developed that makes this repetitive work more efficient and cost-effective.
Automating industry with AI and low-cost robotics
During the hackathon, three teams worked on a real use case in which a ReBeL robot arm was placed in front of a conveyor belt with various pizza products. Using a webcam and an AI-based Segment Anything Model (SAM) system, the systems detected the different products on the conveyor belt and identified their position. The language model assigned these objects to the desired products based on the natural language instructions. The robot then placed the products in the boxes by following the voice model’s instructions.
Alexander Zorn from Fraunhofer IAIS was enthusiastic about the results of the hackathon: “We are very pleased to be able to use igus robotics know-how to develop real proof-of-concepts for customers in the industry. The combination of artificial intelligence and robotics offers enormous potential for automating work processes and making them more efficient.” Alexander Mühlens, authorized representative and head of the Low Cost Automation department at igus GmbH, also emphasized the importance of the hackathon: “The collaboration with the institutes gives us the opportunity to show our customers what is possible with the help of AI and low-cost robotics. Our dream is to be able to easily control robots using a voice command, in any application.” The hackathon showed that many other automation options are conceivable through the use of LLMs and robotics, such as packing several boxes at the same time, sorting Products following special instructions or checking ingredients for allergens using camera systems. The successful collaboration between igus, Fraunhofer IAIS, the Lamarr Institute at the University of Bonn and Fraunhofer IML underlines the potential of these technologies for industrial automation.
**About the partners**
The Fraunhofer IAIS is one of the leading research institutes in the fields of artificial intelligence (AI), machine learning and big data, while the University of Bonn is known as a center for artificial intelligence and robotics. Fraunhofer IML in Dortmund contributed its expertise in logistics automation and material flow. While that Lamarr Institute a leader in international AI research and machine learning.
GIPHY App Key not set. Please check settings