Interfaces for Presenting Summaries of Detections from Search and Rescue Drone Swarms

Abstract

Drones are currently being used in Search and Rescue (SAR) missions enabling ground operators to scan a large area for people in distress by utilising the drone’s video feed. Research has begun looking towards drone swarms which would enable the search of an area faster compared to using a single drone. Many aspects of the SAR mission will be automated including path planning, and detecting objects of interest such as people in distress. Using AI to identify possible targets will be essential as it is not possible to have an operator observe live video feeds from multiple drones over a prolonged period of time with the necessary attentiveness. Therefore, the video feeds must be summarized with the possible objects of interest being highlighted through augmentation or annotations to help the operator quickly understand the video feeds recorded by the drone swarm. In this paper we examine how summarization of video feeds from a drone swarm can be used to aid SAR operators during missions. Furthermore, the focus is on designing a user interface that among other features includes filtering functionality that intends to combat the situational awareness (SA) demons, information overload and attentional tunnelling. To do this, we look into theories on how humans interpret and understand images, and how to apply these theories in the context of drone swarms used for SAR missions. These theories revolve around making it easy to identify the key elements in an image and do so quickly. In the project several user interface designs for highlighting the findings of each individual drone and summarizing the findings of the drone swarm as a whole are explored. Some of these designs were presented to the Danish Emergency Management Agency’s (DEMA) Head of Drone operations in the Danish region of Jutland, and a developer from Robotto, who work with integrating drones and AI in the context of SAR. Based on their feedback, two prototypes were developed that present a summary of a drone swarm’s detections as keyframes in a story board while enabling filtering of the summary based on time and the category of the detected objects. The prototypes were used to conduct an online study with 8 drone operators, drone system developers, and university students as participants. During the study participants were instructed to complete one task using each prototype with an on-going SAR mission given as the imagined scenario. The results showed that presenting a summary as keyframes in a storyboard allowed for participants to correctly identify detections of interest with a significant degree of success. On average the participants correctly marked 4.75 detections out of a possible 6 when using one of the prototypes. The participants also stated that filtering was useful for avoiding information overload.

Colophon: This page is part of the AAU Student Projects portal, which is run by Aalborg University. Here, you can find and download publicly available bachelor's theses and master's projects from across the university dating from 2008 onwards. Student projects from before 2008 are available in printed form at Aalborg University Library.

If you have any questions about AAU Student Projects or the research registration, dissemination and analysis at Aalborg University, please feel free to contact the VBN team. You can also find more information in the AAU Student Projects FAQs.

A master thesis from Aalborg University

Interfaces for Presenting Summaries of Detections from Search and Rescue Drone Swarms