Reinforcement Learning-Driven Drone Navigation in Simulated Environments Using Stochastic Model-Predictive Control and UPPAAL STRATEGO

Greve, Philip Christian ; Pedersen, Kasper Leander Helmer

4. term

Software, Master

2024

2024-06-07

38 pages

Drones are increasingly utilized in industrial applications, including field service management for service and repair in complex environments. In collaboration with Grundfos, a Danish company specializing in pump systems, this project aims to investigate the possible application of reinforcement learning (RL) to enhance drone navigation and task execution within technical rooms at customer locations. The relevant tasks for the drone are to localize pumps in the unknown environment, while also building a map. Our approach involves the use of UPPAAL STRATEGO for strategy synthesis, integrated with Robot Operating System (ROS) to manage drone actuation within a simulated environment. Utilizing co-simulation, we integrate real-time data from a virtual environment to continually synthesize strategies. Building on previous applications of the Stochastic Model-Predictive Control (STOMPC) framework in other domains, this study demonstrates the potential of RL for this use case through a Proof of Concept implementation. Our experiments show that our RL-based approach outperforms a baseline method, highlighting the effectiveness of our proposed solution. Using one ideal configuration, these experiments show that our solution completes the task of locating a single pump in an average of about 4 minutes compared to 14 minutes of the baseline. We further experiment with two different reward engineering approaches to assess which yields the fastest task completion times.

Download
View record in AAU Student Projects