Interfaces for Presenting Summaries of Detections from Search and Rescue Drone Swarms
Student thesis: Master Thesis and HD Thesis
- Shpend Gjela
- Andreas Daugbjerg Christensen
4. term, Software, Master (Master Programme)
Drones are currently being used in Search and Rescue (SAR) missions enabling ground operators to scan a large area for people in
distress by utilising the drone’s video feed. Research has begun looking towards drone swarms which would enable the search of
an area faster compared to using a single drone. Many aspects of the SAR mission will be automated including path planning, and
detecting objects of interest such as people in distress. Using AI to identify possible targets will be essential as it is not possible to have
an operator observe live video feeds from multiple drones over a prolonged period of time with the necessary attentiveness. Therefore,
the video feeds must be summarized with the possible objects of interest being highlighted through augmentation or annotations to
help the operator quickly understand the video feeds recorded by the drone swarm. In this paper we examine how summarization of
video feeds from a drone swarm can be used to aid SAR operators during missions. Furthermore, the focus is on designing a user
interface that among other features includes filtering functionality that intends to combat the situational awareness (SA) demons,
information overload and attentional tunnelling. To do this, we look into theories on how humans interpret and understand images,
and how to apply these theories in the context of drone swarms used for SAR missions. These theories revolve around making it easy
to identify the key elements in an image and do so quickly. In the project several user interface designs for highlighting the findings
of each individual drone and summarizing the findings of the drone swarm as a whole are explored. Some of these designs were
presented to the Danish Emergency Management Agency’s (DEMA) Head of Drone operations in the Danish region of Jutland, and a
developer from Robotto, who work with integrating drones and AI in the context of SAR. Based on their feedback, two prototypes
were developed that present a summary of a drone swarm’s detections as keyframes in a story board while enabling filtering of the
summary based on time and the category of the detected objects. The prototypes were used to conduct an online study with 8 drone
operators, drone system developers, and university students as participants. During the study participants were instructed to complete
one task using each prototype with an on-going SAR mission given as the imagined scenario. The results showed that presenting a
summary as keyframes in a storyboard allowed for participants to correctly identify detections of interest with a significant degree
of success. On average the participants correctly marked 4.75 detections out of a possible 6 when using one of the prototypes. The
participants also stated that filtering was useful for avoiding information overload.
distress by utilising the drone’s video feed. Research has begun looking towards drone swarms which would enable the search of
an area faster compared to using a single drone. Many aspects of the SAR mission will be automated including path planning, and
detecting objects of interest such as people in distress. Using AI to identify possible targets will be essential as it is not possible to have
an operator observe live video feeds from multiple drones over a prolonged period of time with the necessary attentiveness. Therefore,
the video feeds must be summarized with the possible objects of interest being highlighted through augmentation or annotations to
help the operator quickly understand the video feeds recorded by the drone swarm. In this paper we examine how summarization of
video feeds from a drone swarm can be used to aid SAR operators during missions. Furthermore, the focus is on designing a user
interface that among other features includes filtering functionality that intends to combat the situational awareness (SA) demons,
information overload and attentional tunnelling. To do this, we look into theories on how humans interpret and understand images,
and how to apply these theories in the context of drone swarms used for SAR missions. These theories revolve around making it easy
to identify the key elements in an image and do so quickly. In the project several user interface designs for highlighting the findings
of each individual drone and summarizing the findings of the drone swarm as a whole are explored. Some of these designs were
presented to the Danish Emergency Management Agency’s (DEMA) Head of Drone operations in the Danish region of Jutland, and a
developer from Robotto, who work with integrating drones and AI in the context of SAR. Based on their feedback, two prototypes
were developed that present a summary of a drone swarm’s detections as keyframes in a story board while enabling filtering of the
summary based on time and the category of the detected objects. The prototypes were used to conduct an online study with 8 drone
operators, drone system developers, and university students as participants. During the study participants were instructed to complete
one task using each prototype with an on-going SAR mission given as the imagined scenario. The results showed that presenting a
summary as keyframes in a storyboard allowed for participants to correctly identify detections of interest with a significant degree
of success. On average the participants correctly marked 4.75 detections out of a possible 6 when using one of the prototypes. The
participants also stated that filtering was useful for avoiding information overload.
Language | English |
---|---|
Publication date | 1 Jun 2023 |
Number of pages | 25 |