Human-to-Robot Handovers Based on Visual Data forOptimisation of Industrial Tasks: Handovers and Grasp Generation
Translated title
Human-to-Robot Handovers Based on Visual Data for Optimisation of Industrial Tasks: Handovers and Grasp Generation
Authors
Jørgensen, Jan Kjær ; Grønhøj, Rune
Term
4. semester
Education
Publication year
2021
Submitted on
2021-06-02
Pages
48
Abstract
Passing an object from a person to a robot requires the robot to grasp reliably and quickly, even as the camera view and the object’s orientation change. This project studies real-time grasp generation for human-to-robot handovers, targeting Little Helper 7, a platform with two UR5 robot arms. We compare state-of-the-art, image-based methods that predict grasp poses in real time and provide a standardized conversion between common grasp representations to visualize predictions in ROS (Robot Operating System). To assess robustness to camera viewpoint, we use a small custom dataset of grasp rectangles covering 50 scenes captured from three different views. We investigate two pixel-wise grasping networks, GG-CNN and GR-ConvNet, and tune training settings such as batch size, optimizer, and dataset choice. A main focus is the new, very large GraspNet-1B dataset, which differs substantially from earlier datasets; we evaluate whether it improves model performance. The results are promising: training with GraspNet increases robustness to changes in viewpoint, a key requirement for handover scenarios. However, achieving full human-to-robot handovers will require integrating grasping with other capabilities, in particular hand detection and the existing systems on Little Helper 7.
At række en genstand fra et menneske til en robot kræver, at robotten kan gribe sikkert og hurtigt, selv når kameravinkel og objektets orientering varierer. Dette projekt undersøger realtids generering af gribeposer til menneske-til-robot overleveringer med fokus på Little Helper 7, en platform med to UR5-robotarme. Vi sammenligner moderne, billedbaserede metoder, der forudsiger gribepositioner i realtid, og vi indfører en standardiseret konvertering mellem udbredte repræsentationer af greb for at visualisere forudsigelser i ROS (Robot Operating System). For at vurdere robusthed over for synsvinkel anvender vi et lille, speciallavet datasæt med rektangler for gribepositioner fra 50 scener set fra tre forskellige kameravinkler. Vi undersøger to pixelvise gribenetværk, GG-CNN og GR-ConvNet, og tuner træningsindstillinger som batch-størrelse, optimeringsalgoritme og valg af datasæt. Et hovedfokus er det nye, meget store GraspNet-1B datasæt, som adskiller sig markant fra tidligere datasæt; vi vurderer, om det forbedrer modellernes ydeevne. Resultaterne er lovende: træning med GraspNet øger robustheden over for ændringer i synsvinkel, hvilket er afgørende i overleveringssituationer. For at opnå fulde menneske-til-robot overleveringer skal gribning dog integreres med andre systemer, især hånddetektion og de eksisterende komponenter på Little Helper 7.
[This apstract has been rewritten with the help of AI based on the project's original abstract]
Keywords
Handover ; grasp ; grasping ; robot ; robotic ; antipodal ; gg-cnn ; gr-convnet ; pixel-wise ; depth ; image ; rgb ; rgb-d ; Little Helper ; deep learning ; cnn ; machine learning ; hri ; human robot interaction ; collaboration ; industrial ; visual data ; camera ; UR5 ; ur robot ; dual-arm ; ROS ; visualizer ; evaluation ; rviz ; thesis ; speciale