A master thesis from Aalborg University
3D Volumetric and Semantic Reconstruction of a Robotic Workspace Using Deep Learning: A Proposal for a Symbiotic Human Robot Collaboration System
[3D Volumetric and Semantic Reconstruction of a Robotic Workspace Using Deep Learning]
Author(s)
Term
4. semester
Education
Publication year
2021
Submitted on
2021-06-02
Pages
48 pages
Abstract
Human-Robot Interaction (HRI) has evolved at a rapid rate in the last decade. This thesis tackles the aspect of contextualising the environment surrounding the robot in the form of semantics. A scene reconstruction system is proposed, composed of an offline stage (static objects) and an online one (dynamic objects). The system works as proof of concept. Its current application is to search for small objects of interest during the online stage, utilising larger objects found during the offline stage as landmarks. The semantics of the environment are computed by utilising a two-stage object segmentation pipeline, using YOLOv4 as an object detector and DeepLabV3+ for object segmentation. Object ontologies are used to create relations between static and dynamic objects. After offline and online reconstructions have been performed, the system projects the object masks to 3D coordinates. Many qualitative and quantitative results have been reported and demonstrate the robustness of the proposed system. The proof of concept is deemed successful since the system can utilise static objects as regions of interest to detect dynamic objects based on their ontological relations and infer where specific tasks will occur.
Keywords
Deep learning ; AI ; YOLO ; DeepLab ; 3D Reconstruction ; Object ontologies ; GUI ; Human Robot Collaboration ; HRC ; Human Robot Interaction ; HRI ; Symbiotic Human Robot Collaboration ; ROS ; Python ; JSON ; Pytorch ; Darknet
Documents
Colophon: This page is part of the AAU Student Projects portal, which is run by Aalborg University. Here, you can find and download publicly available bachelor's theses and master's projects from across the university dating from 2008 onwards. Student projects from before 2008 are available in printed form at Aalborg University Library.
If you have any questions about AAU Student Projects or the research registration, dissemination and analysis at Aalborg University, please feel free to contact the VBN team. You can also find more information in the AAU Student Projects FAQs.