Interactive fiction (IF) games are a genre of games where the player interacts with the fictional world via text-based commands, solving puzzles primarily by exploring the world and using items they collect along the way. Although there has been much work on playing IF using AI, there is relatively less work on the creation of such games using AI. While large language models (LLMs) have made the generation of text far easier in the past several years, they still struggle to be able to generate such highly-structured and consistent story worlds that one might see in IF. We present a three-part system called BERALL, which generates unique text adventure games by 1) maintaining the current state story world, 2) using retrieval-augmented generation (RAG) to create relevant location descriptions, and 3) combining these components to create a coherent experience for the player. Our results show that our approach is effective at generating room and story descriptions from setting and knowledge graphs, demonstrating the benefits of LLMs in IF. We find challenges remain in maintaining current game state due in part to LLMs not understanding the impact of actions on knowledge graphs.