Applying Reinforcement Learning to RTS Games
Authors
Christiansen, Jeppe Ravn ; Christensen, Allan Mørk ; Vinther, Lars ; Midtgaard, Martin
Term
4. term
Education
Publication year
2010
Submitted on
2010-06-03
Pages
96
Abstract
Dette speciale undersøger, hvordan forstærkningslæring (RL), en AI-metode der lærer gennem forsøg og fejl for at opnå belønninger, kan bruges på flere delopgaver i et kommercielt kvalitets-realtidsstrategispil (RTS). Vi adresserer tre centrale udfordringer: hvor hurtigt læringen stabiliserer sig på god adfærd (konvergenshastighed), hvordan man håndterer flere agenter, der handler samtidig (samtidige agenter), og hvordan man begrænser antallet af mulige situationer, som AI'en skal overveje (tilstandsrum). Vi undersøger to ideer til at løse disse problemer: at behandle tid som en vigtig variabel under læring, og at opdele tilstandsrum i mindre dele ved at identificere uafhængige objekter i spilverdenen. For de samtidige agenter analyserer vi, hvor meget information hver agent behøver om de andre for at handle optimalt. Givet visse begrænsninger i scenarierne kan vores fremgangsmåde øge konvergenshastigheden og reducere tilstandsrum for bestemte opgaver. Vi beskriver også, hvilken type informationsdeling mellem agenter der er passende i forskellige situationer. Til sidst diskuterer vi, hvordan disse løsninger kan kombineres for at tackle endnu mere komplekse problemer.
This thesis explores how reinforcement learning (RL)—an AI method that learns by trial and error to maximize rewards—can be applied to several subtasks in a commercial-quality real-time strategy (RTS) game. We focus on three core challenges: how quickly learning stabilizes on effective behavior (convergence rate), how to coordinate multiple agents acting at the same time (concurrent agents), and how to keep the number of situations the AI must consider manageable (state space). We test two ideas to address these issues: treating time as a key variable during learning, and breaking the state space into smaller parts by identifying independent objects in the game world. For multiple agents, we study how much information each agent needs about the others to act optimally. Under certain scenario constraints, our approach speeds up convergence and reduces the state space for specific tasks. We also outline what kinds of inter-agent information are suitable in different situations. Finally, we discuss how these strategies can be combined to tackle more complex problems.
[This abstract was generated with the help of AI]
Keywords
Documents