ReLight: Capturing spatial-temporal context in Road Traffic Signal Control using recurrency in POMDPs

Kjær, Peter ; Andersen, Samuel Alexander Vall

4. term

Software, Master

2022

2022-06-10

13 pages

Traffic congestion in urban areas is a problem for the environment and the economy. One solution to minimize congestion is optimizing traffic lights. Traffic signal control is a challenging problem due to the complex traffic flow patterns. Conventional traffic control use pre-coded cycle pattern plans, which suffer from adapting to the complex flow dynamics. Reinforcement Learning allows for dynamic control but is unable to properly catch temporal-feature due to the Markov property. To solve this, recent papers propose incorporating prediction modules into Reinforcement Learning control, however, this suffers from additional loss and generalization. To circumvent these issues, we propose Recurrent Light (ReLight), which treats the environment as a Partially Observable Markov Decision Process which depends on the history of previous belief states. We utilize this dependency to capture spatial-temporal features and utilize an LSTM in the DQN network to capture important long-short term features through hidden states. To properly capture cycle phases, we propose two sampling and two training strategies. In our experiments, we demonstrate that ReLight outperforms state-of-the-art models on one, multi and city-wide datasets.

reinforcement learning

Download
View record in AAU Student Projects