6G in-X subnetworks are short-range low-power cells envisioned to support extreme communication requirements for data rate, latency, and reliability. However, interference represents a major limiting factor to extreme communication in dense deployments of in-X subnetworks. Recent studies have proposed interference management solutions based on multi-agent reinforcement learning, where the radio resource optimization problem is modeled as a multi-Markov decision process. The studies have been based on centralized or distributed training. While centralized training benefits from the experiences of all subnetworks during the training, it may lead to compromised privacy and security issues since it requires sharing of measurements between the subnetworks and a centralized agent. In contrast, agents in distributed training rely solely on only local measurements of the environment for decision which often leads to convergence problems.
To overcome these challenges, a client-to-server horizontal federated reinforcement learning framework is proposed, where knowledge is shared implicitly through locally trained model weights.
Simulations in an industrial environment using 3GPP propagation models have shown promising results for quick convergence, marginal performance improvement, and robustness to non-stationary environments.