In this project methods for building a tabular q-learner over a continuous state space is proposed and investigated. Various problem areas are found and methods to counteract these are also tested. It is shown that it is possible to create a tabular learner over a continuous state space with the use of “q-points”, but optimal methods for managing and updating these are still to be investigated