Autonomous robots face many complexities in the real world, in particular: uncertainty about the effects of their actions, large numbers of potential state features and coexistence with multiple cooperative and potentially adversarial robots. In these complex tasks, it is impossible to sufficiently model and identify all the relevant world features necessary for effective goal achievement beforehand. Instead, autonomous robots must discover this knowledge themselves as they interact with their environment.