Reinforcement learning (RL)—using past choices and outcomes to learn best policies—can model the effects of anxiety in human learning. RL mechanisms are commonly studied using game-like computer tasks; but real-life learning often happens in naturalistic settings, with artificial stimuli less involved than day-to-day situations and objects. We used an online “choose your own adventure” text game to test how modality of interaction—narrative versus gamified—impacts the effect of anxiety on RL. 211 participants completed five “chapters” (blocks) of seven choices, followed by informative feedback about underlying story structure. Higher anxiety scores linked to lower model-free accuracy and higher reaction times. An RL model with memory-decay and attention components showed comparable learning rates, but more decision noise and lower attention in anxious participants. This data shows that anxiety impacts RL in naturalistic processing contexts and holds potential insights into the use of narrative, RL-based gamification in educational settings.