Daniel Willemsen is currently a master student at the Delft University of Technology. The research presented here was performed at Centrum Wiskunde & Informatica, where he worked on the interaction between planning and reinforcement learning, such as done by the AlphaZero line of algorithms.