AI in Congestion Control
Author
Abukar, Ubeyd Ali Muhamud
Term
4. semester
Education
Publication year
2025
Submitted on
2025-06-04
Pages
30
Abstract
Congestion control prevents 'traffic jams' in networks so that data arrives reliably. Over many years, protocols have been developed to provide end-to-end reliability: lost data is retransmitted, and out-of-order packets are put back in order. In recent years, artificial intelligence methods, especially reinforcement learning, have been explored for congestion control. This project examines how Q-learning, a reinforcement learning method, can be integrated into TCP and how the choice of reward function (the rule that tells the agent what to aim for) influences the agent's behavior. Several reward designs are created and tested across different network setups. The agents are also trained and evaluated against rule-based TCP protocols such as TCP Vegas and TCP New Reno. Finally, hyper-parameters are tuned, and the results show that this tuning is decisive for whether agents reach their goals.
Kongestionskontrol handler om at undgå 'trafikpropper' i netværk, så data kan komme sikkert frem. Gennem mange år er der udviklet protokoller, som sikrer pålidelig ende-til-ende kommunikation: mistede data bliver sendt igen, og pakker i forkert rækkefølge bliver sat rigtigt sammen. I de senere år er metoder fra kunstig intelligens, især forstærkningslæring, blevet afprøvet til kongestionskontrol. Dette projekt undersøger, hvordan Q-learning, en metode i forstærkningslæring, kan bygges ind i TCP, og hvordan valget af belønningsfunktion (den regel, der fortæller agenten, hvad den skal stræbe efter) påvirker agentens adfærd. Flere varianter af belønningsfunktioner bliver designet og testet i forskellige netværksopsætninger. Agenterne bliver også trænet og evalueret mod regelbaserede TCP-protokoller som TCP Vegas og TCP New Reno. Endelig finjusteres hyperparametre, og resultaterne viser, at denne finjustering er afgørende for, om agenterne når deres mål.
[This apstract has been rewritten with the help of AI based on the project's original abstract]