Transfer of Knowledge in a Reinforcement Learning Setting for a Complex Environment - Progressive Networks in StarCraft II

Jensen, Malthe Dahl ; Madsen, Kaare Bak Toxværd ; Aaes, Andi Rosengreen Kjærsig

4. term

Software, Master

2018

2018-06-08

84

Dette kandidatprojekt på 10. semester på Software-uddannelsen ved Aalborg Universitet undersøger, hvordan kunstig intelligens kan lære og genbruge viden i det komplekse spilmiljø StarCraft II. Vi arbejder med forstærkningslæring (hvor en agent lærer gennem forsøg og fejl ud fra belønninger) og overførselslæring (at bruge erfaring fra én opgave til at hjælpe på en ny opgave). Vi afprøver flere forskellige agentarkitekturer for at finde en løsning, der egner sig bedst til overførselslæring. For at teste om overførsel er mulig i StarCraft II, anvender vi progressive netværk, en netværksarkitektur foreslået af Google DeepMind i 2016, som gør det muligt at bygge videre på viden fra flere opgaver, når der trænes på nye. Samtidig undgår progressive netværk katastrofal glemsel (når et netværk glemmer tidligere opgaver, når det lærer nye), hvilket gør det muligt at vurdere, hvor meget overførsel der sker, og hvor i netværket den foregår.

This master's thesis project by a 10th-semester group in the Software programme at Aalborg University explores how artificial intelligence can learn and reuse knowledge in the complex game environment of StarCraft II. We use reinforcement learning (where an agent learns by trial and error based on rewards) and transfer learning (reusing what was learned on one task to help with a new task). We evaluate several agent architectures to identify a design well suited for transfer learning. To test whether transfer is possible in StarCraft II, we apply progressive networks, a network architecture proposed by Google DeepMind in 2016 that lets a new model build on knowledge from multiple tasks when training on new ones. Progressive networks also avoid catastrophic forgetting (when learning new tasks causes a model to forget old ones), which allows us to estimate how much transfer occurs and where in the network it happens.

[This abstract was generated with the help of AI]

Reinforcement learning ; transfer learning ; A3C ; progressive networks ; starcraft 2 ; starcraft II ; starcraft ; cartpole ; sonic ; sonic the hedgehog ; transfer analysis ; Average Fisher Sensitivity ; actor-critic ; neural networks ; machine learning

Download
View record in AAU Student Projects