United States

Automatic storytelling is a difficult NLP task that poses a challenge even for state-of-the-art large language models. This paper proposes a pipeline for interactive fairy tale generation in a mixed-initiative setting. Our approach introduces a story goal as a stopping condition, imposes minimal structure on the narrative in the form of a simple emotional arc, and controls the transition between the stages of the story via system prompt engineering. The resulting framework reconciles creating a structured and complete short-form narrative with retaining player agency and allowing users to influence the storyline through their input. We evaluate our approach with several proprietary and open-source language models and examine its transferability to different languages, specifically English and Russian.

EMNLP 2024

How to tame your plotline: A framework for goal-driven interactive fairy tale generation

mixed-initiative gameplay

fairy tale generation

emotional arcs

automatic storytelling

narrative structure

large language models

## Welcome to EMNLP 2024! 
We are excited to welcome you to one of the most prominent conferences in the field of Natural Language Processing. This year, EMNLP 2024 is being held in a hybrid format,
offering both virtual and in-person participation in beautiful Miami. Due to a record-breaking number of submissions, we've expanded the total number of accepted papers to accommodate more cutting-edge research from around the globe.
### [Conference Handbook](https://drive.google.com/file/d/1WPROgxjLAC96AJL7Ugy0tEnYm7dkrbHt/view?usp=sharing)

You are required to register for this event. **Please register [here](https://2024.emnlp.org/registration/).** The EMNLP 2024 event page on Underline will be open to public one week prior to the event.

Please register!

EMNLP 2024 will take place in Miami, Florida from Nov 12th to Nov 16th, 2024, at the Hyatt Regency Miami Hote and on Underline for remote participants.

This study explores the reception and understanding of the transmedia ensemble surrounding Bill Willingham's Fables (2002-2015), a comic series reimagining fairytale characters in a modern setting. Fables expands its narrative across multiple media, including spin-off comics, a novel, and the video game The Wolf Among Us. This research investigates key questions: Can we identify a distinct group of transmedia consumers? What elements of the narrative sustain interest across media? A survey of 58 participants reveals that while most enter the franchise through the comic series, a significant number are introduced via the video game. The findings indicate that Fables fans are highly engaged transmedia consumers, with a majority exploring several parts of the franchise wanting to pursue narrative exploration. This study offers insights into how transmedia narratives are consumed, emphasizing the role of familiar story elements in encouraging cross-media engagement.

Understanding Transmedia Storytelling: Reception and Narrative Comprehension in Bill Willingham's Fables Franchise

In this study, we explore the application of large language models (LLMs) to analyze narrative discourse within the framework established by the field of narratology. We develop a set of elementary narrative features derived from prior theoretical work that focus on core dimensions of narrative, including time, setting, and perspective. Through experiments with GPT-4 and fine-tuned open-source models like Llama3, we demonstrate the models' ability to annotate narrative passages with reasonable levels of agreement with human annotators. Leveraging a dataset of human-annotated passages spanning 18 distinct narrative and non-narrative genres, our work provides empirical support for the deictic theory of narrative communication. This theory posits that a fundamental function of storytelling is the focalization of attention on distant human experiences to facilitate social coordination. We conclude with a discussion of the possibilities for LLM-driven narrative discourse understanding.

Using Large Language Models for Understanding Narrative Discourse

Narrative is widely recognized as a powerful tool for structuring information and facilitating comprehension of complex ideas in various domains, such as science communication. This paper investigates whether incorporating narrative elements can assist Large Language Models (LLMs) in solving complex tasks more effectively. We propose a novel approach, Story of Thought (SoT), integrating narrative structures into prompting techniques for problem-solving tasks. This approach involves constructing narratives around problem statements and creating a framework to identify and organize relevant information. We hypothesize that this narrative-based information curation process enhances problem comprehension by contextualizing critical information and highlighting causal relationships within the problem space. Our experimental results show that the SoT approach consistently surpasses Chain of Thought (CoT) and Analogical Reasoning in GPQA tasks, achieving higher accuracy and better solutions in physics, chemistry, and biology problem-solving tasks with all tested OpenAI, Meta, and Mistral LLMs.

Can Stories Help LLMs Reason? Curating Information Space Through Narrative

Stories are a rich representation of the beliefs, practices, and languages of communities, and a source of cultural information for researchers in the social sciences and humanities. Characters – their personalities and behaviour – are a key aspect of this process. Large Language Models (LLMs) are becoming increasingly popular tools for creative text generation. Our work creates the GPT-Writing Prompts dataset, which pairs 97,219 prompts and human-written stories from the Reddit WritingPrompts dataset with corresponding GPT-3.5 generated stories. We quantify the emotional and descriptive features of protagonists in stories along six lexical dimensions: appearance, intellect, power, valence, arousal, dominance. We also quantify biases in character portrayal based on protagonist gender in both human-written and GPT-generated stories.

The GPT-WritingPrompts Dataset: A Comparative Analysis of Character Portrayal in Short Stories

Recent advancements in NLP and language modeling have significantly enhanced LLMs’ ability to extract personal identifiers in text beyond the writing style, including names, locations, and cultural references to infer the writer’s identity. This constitutes an attack on privacy of individuals and could have fatal consequences for some personal narratives (e.g. domestic violence). In this work, we investigate domestic violence narratives in a Reddit community (r/do- mesticviolence), with the aim of identifying common patterns across a large collection of these stories. Our ultimate goal is to leverage the shared structure to develop a privacy preserving framework which assesses the safety of sharing a story. We propose a binary classification approach that utilizes distance vectors as features to classify a story as either ”similar to other domestic violence stories” or ”not similar”. This could serve as a first step toward more sophisticated methods for anonymizing the structure of sensitive narratives.

Is It Safe to Tell Your Story? Towards Achieving Privacy for Sensitive Narratives

This work presents the task of Zero-shot Trajectory Mapping, which focuses on the spatial dimension of narratives. The task consists of two parts: (1) creating a ``map'' with all the locations mentioned in a set of texts, and (2) extracting a trajectory from a single testimony and positioning it within the map. Following recent advances in context length capabilities of large language models, we propose a pipeline for this task in a completely unsupervised manner, without requiring any type of labels. We test the pipeline on a set on 402 Holocaust testimonies and present the resulting map and samples of the trajectory. We find that current long-range models succeed in generating meaningful maps and trajectories. We show how the trajectories can be used to align parallel parts of different testimonies. We conclude that LLMs can serve as beneficial tools for Holocaust research and narrative analysis.

Zero-shot Trajectory Mapping in Holocaust Testimonies

To understand how stories are structured, we would like to be able to analyze the architecture of narratives. This article reviews and compares existing annotation guidelines for scene and narrative level annotation. We propose new guidelines, based on existing ones, and show how these can be effectively extended from general-purpose to specialized contexts, such as mystery novels which feature unique narrative elements like red herrings and plot twists. This provides a controlled environment for examining genre-specific event structuring. Additionally, we present a newly annotated genre-specific dataset of mystery novels, offering valuable resources for training and evaluating models in narrative understanding. This study aims to enhance annotation practices and advance the development of computational models for narrative analysis.

Annotating Mystery Novels: Guidelines and Adaptations

Humans are naturally drawn to narratives, using them to make sense of their lives and engage with information. Even brief social media posts often take narrative forms to capture attention. This underscores the need for advanced computational methods to analyze narratives effectively, recognizing both their potential and limitations. Our study compares traditional techniques like keyword-based measurements and topic modeling with the latest concept-frame-focused narrative triplet extraction method. Using a dataset with competing narratives from two communities, we evaluate these methods' accuracy and objectivity with a Large Language Model. This research offers innovative insights for operationalizing "narrative" in natural language processing, bridging the gap between computer science and social science in narrative understanding.

Evaluating and Enhancing Methods for Narrative Understanding

Evaluating the creativity of large language models (LLMs) in story writing is difficult because LLM-generated stories could seemingly look creative but be very similar to some existing stories in their huge and proprietary training corpus. To overcome this challenge, we introduce a novel benchmark dataset with varying levels of prompt specificity: CS4 (Comparing the Skill of Creating Stories by Controlling the Synthesized Constraint Specificity). By increasing the number of requirements/constraints in the prompt, we can increase the prompt specificity and hinder LLMs from retelling high-quality narratives in their training data. Consequently, CS4 empowers us to indirectly measure the LLMs' creativity without human annotations. Our experiments on LLaMA, Gemma, and Mistral not only highlight the creativity challenges LLMs face when dealing with highly specific prompts but also reveal that different LLMs perform very differently under different numbers of constraints and achieve different balances between the model's instruction-following ability and narrative coherence. Additionally, our experiments on OLMo suggest that Learning from Human Feedback (LHF) can help LLMs select better stories from their training data but has limited influence in boosting LLMs' ability to produce creative stories that are unseen in the training corpora.

CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing Constraints

We present a novel approach to classify causal micro-narratives from text. These narratives are sentence-level explanations of the cause(s) and/or effect(s) of a target subject. The approach requires only a subject-specific ontology of causes and effects, and we demonstrate it with an application to inflation narratives. Using a human-annotated dataset spanning historical and contemporary US news articles for training, we evaluate several large language models (LLMs) on this multi-label classification task. The best-performing model---a fine-tuned Llama 3.1 8B---achieves F1 scores of 0.87 on narrative detection and 0.71 on narrative classification. Comprehensive error analysis reveals challenges arising from linguistic ambiguity and highlights how model errors often mirror human annotator disagreements. This research establishes a framework for extracting causal micro-narratives from real-world data, with wide-ranging applications to social science research.

Downloads

Next from EMNLP 2024

Understanding Transmedia Storytelling: Reception and Narrative Comprehension in Bill Willingham's Fables Franchise

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES