Reinforcement Learning is the task of teaching an agent optimal behavior in its environment by reinforcing actions with rewards and penalties. As domains grow larger, the representation of a domain and its solution becomes an increasingly important issue. Many real-world domains are often impossible to represent directly in a conventional table-based manner. In this work, we discuss two approaches towards a solution to this problem. The first is relational reinforcement learning, which combines reinforcement learning with inductive logic programming to obtain state abstractions and better generalization properties. The second is hierarchical reinforcement learning using the MAXQ value function decomposition. This approach creates the opportunity for state abstractions through procedural decomposition of the primary task of a domain. Following, the possibility of combining these two methods is investigated. The result is a general approach that benefits both from the use of inductive logic programming and hierarchical decomposition.