Automatic construction of Legolized 3D models utilising a synthetically trained machine-learning vision system
Authors
Jensen, Rasmus Busk ; Jomhur, Yad Sarwar ; Endelt, Christopher Ørtoft
Term
4. term
Education
Publication year
2024
Submitted on
2024-05-30
Pages
79
Abstract
This thesis presents the development of an automated platform for handling LEGO bricks. First, 3D structures are turned into LEGO-friendly designs through voxelization, which represents a model as tiny cubes. The design is then optimized to use fewer bricks while preserving overall shape and color, sliced into layers, and converted into step-by-step instructions a robot can follow. The physical setup consists of a single robot and a looped hopper feed system that brings large numbers of randomly mixed bricks into the camera's region of interest. The machine-vision pipeline uses two machine-learning models: a color model to identify brick colors and a type model for detecting and segmenting brick types. Trained on the full dataset, these models successfully detected bricks for pickup. A pick-and-place process was implemented, and a force/torque sensor was integrated to enable automatic handling. Key performance indicators (KPIs) were collected during testing and analyzed. A LEGO handling platform was achieved; however, the force/torque sensor broke near the end of the project, so fully automated operation of the platform could not be verified.
Dette speciale beskriver udviklingen af en automatiseret platform til håndtering af LEGO-klodser. Først omsættes 3D-strukturer til LEGO-egnede designs ved hjælp af voxelisering, hvor modellen repræsenteres som små kuber (voxler). Derefter optimeres designet for at bruge færre klodser, samtidig med at form og farver bevares; det skæres i lag og omdannes til trinvise instruktioner, som en robot kan følge. Den fysiske opstilling består af én robot og et cirkulerende hopper-fødesystem, der bringer store mængder tilfældigt blandede klodser ind i kameraets interesseområde (ROI). Billedbehandlingen anvender to maskinlæringsmodeller: en farvemodel til farvegenkendelse og en typemodel til detektion og segmentering af klodsetyper. Modellerne, der blev trænet på hele datasættet, detekterede klodser klar til opsamling med succes. En pick-and-place-proces blev implementeret, og en kraft-/momentsensor blev integreret for at muliggøre automatisk håndtering. Under test blev der indsamlet og analyseret nøgletal (KPI'er). En LEGO-håndteringsplatform blev realiseret; dog gik kraft-/momentsensoren i stykker mod slutningen, og derfor kunne fuldautomatisk drift ikke verificeres.
[This apstract has been rewritten with the help of AI based on the project's original abstract]
Keywords