A master thesis from Aalborg University
Human-to-Robot Handovers Based on Visual Data forOptimisation of Industrial Tasks: Handovers and Grasp Generation
[Human-to-Robot Handovers Based on Visual Data for Optimisation of Industrial Tasks: Handovers and Grasp Generation]
Author(s)
Term
4. semester
Education
Publication year
2021
Submitted on
2021-06-02
Pages
48 pages
Abstract
Dette projekt omhandler objekt-overdragelse fra menneske til robot, med fokus på robust realtids grasping. Dette system er udviklet på Little Helper 7-platformen, som er en dobbeltarmet robot. Projektet undersøger forskellige moderne grasp generationsmetoder for at udnytte dem til realtids grasping i et scenario omhandlende objekt-overdragelse. En standard metode til at konvertere imellem forskellige grasp repræsentationer er præsenteret, og brugt til at visualisere de genererede grasps, ved brug af ROS. Forskellige kamera-synspunkter er undersøgt, ved hjælp af et brugerdefineret grasp datasæt, som indeholder 50 scener fra tre forskellige synspunkter. To pixel-baserede realtids grasp generationsmetoder (GG-CNN og GR-ConvNet) er undersøgt og justeret, med henblik på batch-størrelse, optimeringsalgoritme and datasæt. Hovedsageligt det nye Graspnet 1-billion datasæt er undersøgt, som en metode til, at opnå bedre ydeevne på de valgte modeller, end de allerede har, da dette datasæt er vidt forskelligt sammenlignet med tidligere brugte datasæt. Den foreslåede grasp generationsmetode viser lovende ydeevne under evaluering. Især brugen af Graspnet datasættet viser sig at forbedre invarians til forskellige synspunkter hvilket er væsentligt ved objekt-overdragelse. For at opnå objekt-overdragelse fra menneske til robot, vil det kræve integration med andre systemer. Den største mangel er detektion af hænder og integration med de forudgående systemer i Little Helper 7-platformen.
This project revolves around human-to-robot handovers, with a focus on robust real-time grasping. The system is developed with the Little Helper 7 dual-arm UR5 platform in mind. This project explores multiple state of the art grasp generation methods to utilise them for real-time grasping in handover scenarios. A standardised conversion between grasp representation is presented for visualisation of grasp predictions using ROS. Multiple viewpoints are investigated using a custom grasp rectangle data-set, with 50 scenes from three different views. Two pixel-wise real-time grasp generation methods (GG-CNN and GR-ConvNet) are explored and tuned, including tweaking batch-size, optimiser and data-set. Mainly, the novel Graspnet 1-billion data-set is investigated to improve the existing performance of the models, as the data-set is widely different from the previously available ones. The grasp generation methods proposed shows promise during evaluation. Especially the use of Graspnet seems to improve invariance to viewpoints which is essential during a handover scenario. However, to achieve a human to robot handovers, integration with other systems is needed. Mainly hand detection is required and integration with previous systems of the Little Helper 7.
Keywords
Handover ; grasp ; grasping ; robot ; robotic ; antipodal ; gg-cnn ; gr-convnet ; pixel-wise ; depth ; image ; rgb ; rgb-d ; Little Helper ; deep learning ; cnn ; machine learning ; hri ; human robot interaction ; collaboration ; industrial ; visual data ; camera ; UR5 ; ur robot ; dual-arm ; ROS ; visualizer ; evaluation ; rviz ; thesis ; speciale
Documents
Colophon: This page is part of the AAU Student Projects portal, which is run by Aalborg University. Here, you can find and download publicly available bachelor's theses and master's projects from across the university dating from 2008 onwards. Student projects from before 2008 are available in printed form at Aalborg University Library.
If you have any questions about AAU Student Projects or the research registration, dissemination and analysis at Aalborg University, please feel free to contact the VBN team. You can also find more information in the AAU Student Projects FAQs.