Reinforcement Learning in RTS Games

Toftgaard Andersen, Kresten ; Buch, Anders ; Dahl Christensen, Dennis ; Tran, Dung

4. term

Software Development, Master

2008

2008-06-10

126 pages

In this master thesis we want to apply Reinforcement Learning, which is a well known academic Machine Learning method, to a Real Time Strategy (RTS) game. Our goal is to investigate the feasibility of monitoring human opponents' strategy in an RTS game to counter it by predesigned rewards and policies, where policies are updated using Reinforcement Learning. We propose three extensions of the Reinforcement Learning structure to improve the use of RL in RTS games. We propose a simple and effective multi layered structure, which together with our player modelling technique called profiler, makes it easy to swap the current set of rewards and policy used by the Reinforcement Learning AI. We also propose a modification to the update function of RL algorithms which we call backtracking, which updates several steps back, instead of one step as TD(0) algorithms do. These proposed extensions constitute a framework, which can be applied to any RTS game. To verify that our proposals work in a practical setting, we have applied our framework to the RTS game Tank General, which mainly was developed in the preparation semester. While the backtracking does not give a performance boost, the multi layered structure and the profiler performs very well in an RTS game. Together they reduce the state space drastically, while reducing the time it takes for the RL algorithms to converge. Therefore the conclusion of this thesis is that it definitely is feasible to use our proposed framework in an RTS game.

Reinforcement Learning ; RTS ; Real Time Strategy ; RL ; Machine Learning

Download
View record in AAU Student Projects