A master thesis from Aalborg University
Solving Euclidean Markov Decision Processes with Neural Networks
Author(s)
Term
4. term
Education
Publication year
2021
Submitted on
2021-06-10
Pages
26 pages
Abstract
When dealing with machine learning on cyber-physical systems, one problem is to train the models without extensive cost or harm to the system or its surroundings as the method learns. One method is to use Priced Timed Markov Decision Processes over a Euclidean state space to define a formal model for these systems and train on. We attempt to use Neural Networks to find optimal strategies for such models. We do this by implementing Deep Q-Network in Uppaal Stratego, make a sweep over possible hyperparamters for DQN, select three candidates and test these against the current state of the art optimization algorithm in Uppaal Stratego. Our results show that DQN can with the right hyperparameters find the optimal strategy for simple models in fewer runs than the current method, and find better strategies on some of the more complex models. However, we could not find improved strategies for all models within the tested set hyperparameter configuration.
Documents
Colophon: This page is part of the AAU Student Projects portal, which is run by Aalborg University. Here, you can find and download publicly available bachelor's theses and master's projects from across the university dating from 2008 onwards. Student projects from before 2008 are available in printed form at Aalborg University Library.
If you have any questions about AAU Student Projects or the research registration, dissemination and analysis at Aalborg University, please feel free to contact the VBN team. You can also find more information in the AAU Student Projects FAQs.