Federated Interference Management for Industrial 6G Subnetworks

Madsen, Bjarke Bak

4. term

Signal Processing and Computing, Master

2023

2023-06-02

100 pages

6G in-X subnetværk er kortdistances laveffektsceller, der er designet til at imødekomme ekstreme krav til kommunikation i form af datahastighed, forsinkelse og pålidelighed. Dog udgør interferens en betydelig begrænsende faktor for ekstrem kommunikation i tætte udrulninger af in-X subnetværk. Nylige studier har foreslået løsninger til styring af interferens baseret på ”multi-agent reinforcement learning”, hvor problemet med radioressourceoptimering modelleres som en ”multi-Markov decision process”. Studierne har været baseret på enten centraliseret eller distribueret træning. Mens centraliseret træning drager fordel af erfaringerne fra alle delnetværkene under træningen, kan det medføre kompromittering af privatliv og sikkerhed, da det kræver deling af målinger mellem delnetværkene og en centraliseret agent. Derimod er agenter i distribueret træning udelukkende afhængige af lokale målinger af miljøet for at træffe beslutninger, hvilket ofte fører til konvergensproblemer. For at overvinde disse udfordringer foreslås en klient-til-server ”horizontal federated reinforcement learning” model, hvor viden deles implicit gennem lokalt trænede modelvægte. Simulationer i et industrielt miljø ved hjælp af 3GPP-udbredelsesmodeller har vist lovende resultater med hensyn til hurtig konvergens, marginale præstationsforbedringer og robusthed over for ikke-stationære miljøer.

6G in-X subnetworks are short-range low-power cells envisioned to support extreme communication requirements for data rate, latency, and reliability. However, interference represents a major limiting factor to extreme communication in dense deployments of in-X subnetworks. Recent studies have proposed interference management solutions based on multi-agent reinforcement learning, where the radio resource optimization problem is modeled as a multi-Markov decision process. The studies have been based on centralized or distributed training. While centralized training benefits from the experiences of all subnetworks during the training, it may lead to compromised privacy and security issues since it requires sharing of measurements between the subnetworks and a centralized agent. In contrast, agents in distributed training rely solely on only local measurements of the environment for decision which often leads to convergence problems. To overcome these challenges, a client-to-server horizontal federated reinforcement learning framework is proposed, where knowledge is shared implicitly through locally trained model weights. Simulations in an industrial environment using 3GPP propagation models have shown promising results for quick convergence, marginal performance improvement, and robustness to non-stationary environments.

Reinforcement learning ; Federated learning ; 6G in-X subnetwork ; radio resource optimization ; interference management ; multi-agent reinforcement learning ; double deep q-network ; proximal policy optimization ; channel allocation ; data secutiry ; industrial ; 3GPP ; robots ; uncoordinated network

Download
View record in AAU Student Projects