Deep-Q Learning for Adaptive Beamforming in mmWaves Communication using a Centralised System
Authors
Nissen, Victor Mølbach ; Rasmussen, Nicolai Almskou
Term
4. term
Education
Publication year
2022
Pages
153
Abstract
Trådløs kommunikation i millimeterbølgeområdet (mmWave) giver høj kapacitet, fordi der er meget båndbredde til rådighed, men signalet dæmpes kraftigt over afstand og blokeres let i bymiljøer. Dette kan afhjælpes ved at fokusere energien med adaptiv beamforming og retningsstyrbare antenne-arrays. I dette arbejde undersøger vi, om Reinforcement Learning (RL) kan bruges til at udvikle en adaptiv beamforming-algoritme til mmWave. For at teste det skabte vi datasæt fra en realistisk bysimulering, der følger 3GPP/5G-antagelser. Vi simulerede fire basestationer og én bevægende bruger i to scenarier (Bil og Fodgænger) og under to forhold: Line-Of-Sight (LOS, fri sigtelinje) og Non-Line-Of-Sight (NLOS, med forhindringer). Efter at have gennemgået RL-metoder valgte vi et Deep Q-Network (DQN) i et centraliseret setup. DQN'et blev tunet til Bil-LOS-scenariet og derefter testet i både Bil- og Fodgænger-scenarier under både LOS og NLOS. Selvom ydeevnen faldt uden for det scenarie, det var tunet til, så vi stadig acceptabel ydeevne i fodgænger- eller NLOS-scenarier. I alle scenarier overgik det tunede DQN en simpel, heuristisk algoritme uden maskinlæring på de samme datasæt.
Wireless communication at millimeter-wave (mmWave) frequencies offers high capacity because there is plenty of available spectrum, but signals weaken quickly over distance and are easily blocked in cities. Engineers address this by focusing energy with adaptive beamforming and steerable antenna arrays. This study examines whether Reinforcement Learning (RL) can be used to build an adaptive beamforming algorithm for mmWave. To test this, we created datasets from a realistic urban simulation that follows 3GPP/5G modeling assumptions. We simulated four base stations and one moving user in two scenarios (Car and Pedestrian) and under two conditions: Line-of-Sight (LOS, a clear path) and Non-Line-of-Sight (NLOS, with obstacles). After reviewing RL methods, we selected a Deep Q-Network (DQN) in a centralized setup. The DQN was tuned for the Car-LOS scenario and then tested in both Car and Pedestrian scenarios under both LOS and NLOS. Although performance decreased outside the scenario it was tuned for, we still observed acceptable performance in pedestrian or NLOS scenarios. In all cases, the tuned DQN outperformed a simple heuristic, non-machine-learning algorithm on the same datasets.
[This summary has been rewritten with the help of AI based on the project's original abstract]
Keywords
Documents