Adaptive beam alignment and tracking in mmWave systems using Reinforcement Learning

Fisker, Peter Kjær ; Sand, Dennis Kjærsgaard

4. term

Signal Processing and Computing, Master

2022

2022-06-02

173 pages

This project investigates if Reinforcement Learning is a suitable solution for the problem of beam alignment and tracking in mmWave communication systems with a single base station and a single piece of user equipment. First the mmWave communication systems is modelled, using QuaDRiGa for channel modelling, and a realistic mobility model is implemented for user equipment movement. This is followed by an analysis of the RL framework, first from a mathematical view and then in the context of fitting it to the mmWave communication problem. Two implementations are then proposed using tabular methods, a centralized version with a single agent and a distributed version with two agents cooperating. The parameters of these implementations are then tuned in LOS conditions with broad sweeps over their parameter spaces. After being tuned, the implementations are evaluated in different scenarios to determine the usefulness of RL as a solution. The tested scenarios include LOS/NLOS conditions and environments with different amounts of noise. As a point of reference a simple heuristic algorithm is implemented as well and tested in the same scenarios. The evaluation showed that a simple multi agent implementation performs best, with promising results in both LOS, NLOS and noisy conditions, that outperforms the heuristic reference algorithm.

5G ; mmWave ; Reinforcement Learning ; Beam tracking ; Beam alignment

Download
View record in AAU Student Projects