Deep-Q Learning for Adaptive Beamforming in mmWaves Communication using a Centralised System

Nissen, Victor Mølbach ; Rasmussen, Nicolai Almskou

4. term

Signal Processing and Computing, Master

2022

2022-06-01

153 pages

Wireless communication in the millimeterwave (mmWave) band is attractive due to the available bandwidth, but large free-space propagation loss is a challenge. This challenge can be overcome by increasing the signal power using adaptive beamforming techniques in combination with steerable antenna arrays. In this report, we research whether it is possible to construct an adaptive beamforming algorithm for mmWave communication using Reinforcement Learning (RL). To be able to determine if this is possible data sets has been made using a realistic simulation of an urban environment closely following the 3GPP/5G standard modelling assumptions. Four base stations and one moving user has been simulated in two scenarios a Car and a Pedestrian for both Line-Of-Sight (LOS) and non-LOS (NLOS). Extensive research has been done on RL theory and methods to find a suitable algorithm for the urban environment. It was found that a Deep Q-Network (DQN) should be used on a centralised system. The DQN was tuned for the Car LOS scenario. The tuned DQN was then tested on both the Car and Pedestrian scenario in both LOS and NLOS conditions, and while performance is degraded w.r.t the scenario it was tuned for, we still see acceptable performance in Pedestrian or NLOS scenarios. In all scenarios the tuned DQN outperformed a simple, heuristic non-machine learning algorithm run on the same data sets.

Machine Learning ; Reinforcement Learning ; Beamforming ; Wireless Communication ; mmWave

Download
View record in AAU Student Projects