Where can I go? Deep multi-modal scene understanding for outdoor navigation

Student thesis: Master thesis (including HD thesis)

  • Galadrielle Eve Giséle Elisabeth Humblot-Renaux
4. semester, Robotics, M.Sc. (Master Programme)
This project delves into deep learning-based computer vision for scene understanding in the context of autonomous outdoor navigation. Rather than relying on specific scene-dependent semantic categories, we take an affordance-based approach, proposing to parse egocentric images in terms of how a vehicle or robot can drive in them. We use a SegNet-based image segmentation network as our building block for classifying pixels into 3 driveability levels, and explore soft labelling, pixel-wise loss weighting, and deep adaptive fusion schemes to penalize severe mistakes during learning, improve segmentation in regions of interest, and incorporate infrared and depth data into the prediction. The proposed training schemes and multi-modal architecture are evaluated on 9 public datasets, showing promising results across unstructured forested environments, urban driving scenes, and multi-view hand-held captures.
Publication date3 Jun 2021
Number of pages150


Predictions on diverse hand-held images by the cross-dataset, LW model from Section 9.2.3
ID: 413627756