Curiosity-driven Planning with Reinforcement Learning

Nguyen, Kim

4. term

Computer Science (IT), Master

2023

2023-06-09

12 pages

Reinforcement Learning (RL) approaches are most often clueless of what to do in environments of only sparse extrinsic rewards. Nonetheless, humans and animals are able to learn under similar conditions due to curiosity, as it gives us an intrinsic drive to explore what is novel. In this work, we investigate the potential of how curiosity can enhance a model-based RL agent’s learning and encourage exploration in a visual environment of only sparse extrinsic rewards. We introduce a novel model-based curiosity-driven RL agent, that learns and uses a compact latent representation of the visual environment as input, employs the concept of Random Network Distillation (RND) to generate episodic intrinsic rewards and encourage curiosity-driven planning in a Monte Carlo Tree Search (MCTS). We demonstrate that curiosity enhances the learning of a model-based agent, as our proposed agent is able to solve a visual environment of sparse rewards, that is otherwise unsolvable by a model-free and model-based agent without curiosity. Finally, we examine the prospect of building a world model that can be used as an MCTS simulation environment.

Model-based Reinforcement Learning ; Monte Carlo Tree Search ; Exploration ; Curiosity Learning ; Random Network Distillation ; World Models

Download
View record in AAU Student Projects