United States

Various aspects of the Elicited Imitation Test (EIT), a sentence repetition task for language assessment, can be automated, for example in terms of test administration or automatic scoring. It is potentially also possible to generate test items with Large Language Models (LLMs). This study investigates the potential of GPT-4o for item creation in the context of EIT, creating a parallel form to two popular and validated tests. We analysed the tests in terms of their linguistic and psychometric properties. While the items created by the LLM show some difference in grammatical structures when compared to human-written items, linguistic complexity results did not differ significantly between tests. Psychometric properties showed only minor differences. These findings lend support to the potential of Automatic Item Generation with LLMs in the context of sentence repetition tasks and might support the process of standardisation in SLA research and testing by enabling parallel test creation.

ACL 2026

Using LLMs for item creation: Validating the potential of automatically generated sentence repetition test items for language assessment

sentence repetition task

automatic item generation

language assessment

### Welcome to ACL 2023, the 61st Annual Meeting of the Association for Computational Linguistics! 
 The conference will be held in Toronto, Canada, July 9-14, 2023. 
Following the succession of the recent conferences in our field, ACL 2023 will adopt a hybrid format.
While the impact of Covid has considerably diminished in terms of traveling, obtaining visas to Canada
entails a very long process. Moreover, the global economic conditions pose challenges for many individuals to travel to conferences. Recognizing these circumstances, we know many participants may not be
able to attend the conference in person. Therefore, we are committed to providing a great virtual platform
so everyone has the opportunity to interact with other participants and enjoy the conference. Based on the
current registered participants, approxiately 30% have chosen to attend the conference virtually. Whether
you join us in person or virtually, we sincerely hope everyone has a remarkable conference experience. 
This General Chair’s message is where I express my gratitude to the many individuals who have made
enormous contributions to the conference over the past year.

Read [**ACL 2023 General Chair's message**](https://docs.google.com/document/d/1WobYM7norbG4dI48s75HfJoD89qgX5a_F-6U8AteLSA/edit?usp=sharing/) in full.

##### **[Conference Handbook](https://2023.aclweb.org/downloads/acl2023-handbook.pdf)**

ACL 2023

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing.

workshop paper

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. 
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) 
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

Young athletes, parents and coaches are increasingly encountering training metrics derived from wearable technology, yet such metrics rarely provide sufficient information on their own. We present a rule-based data-to-text framework designed to promote data literacy in youth football, generating concise, stakeholder-specific summaries of training sessions. A rule layer maps duration-normalised indicators to structured facts about the session profile, internal intensity, speed exposure and movement dynamics. A large language model then verbalises these facts for coaches, parents or players. We compare direct generation from raw metrics, strictly rule-based generation and an extended RULES+CONTEXT implementation that combines validated facts with raw metrics and explicit threshold definitions. In the open-weight setting examined here, the selected models are also adapted using LoRA. When evaluated on 122 player-session records from a U15 environment, it was found that rule grounding improved reliability and audience adaptation compared to direct generation. A school-based expert evaluation also provided preliminary evidence that player-facing explanations in the extended setting can remain accurate, comprehensible, and useful in practice.

From Metrics to Meaning: Rule-Grounded LLM Explanations for Data Literacy in Youth Football

Productive struggle is a critical component of mathematics education, requiring students to actively work through ideas rather than just making errors. However, identifying this struggle from text transcripts is challenging because students often mask confusion with epistemic hedging rather than direct statements. Zero-shot large language models exhibit a conservative bias, systematically under-detecting struggle in classroom discourse. We introduce a two-stage NLP pipeline comprising a lexical heuristic gate and an LLM subtype classifier. Our model achieves 90.0% binary accuracy and 84.0% 4-category accuracy. We demonstrate the pedagogical value of this tool by showing that struggle is uniquely concentrated during explicit mathematical reasoning, offering educators a scalable method for root-cause analysis.

Classification of Student Struggle in Mathematics

Recent advances in the capabilities of conversational agents based on large language models make them a very promising tool for role playing K-12 students in order to train educators in conversational teaching practices, such as eliciting student thinking, explaining disciplinary content, and facilitating a classroom discussion. In fact, such simulations can and have been developed relatively quickly and without data to machine-learn from -- neither classroom data nor human-simulated data. To enhance the usefulness and effectiveness of such teaching simulations, it is necessary to provide pedagogically sound, timely, and personalized feedback to the educator about their simulation performance. In this study, we present experiments on fine-tuning models to evaluate educator performance in an elicitation teaching simulation. The models are developed with data collected during usability testing of the simulation and evaluated on real user data. We show that even with relatively little fine-tuning data, robust performance can be obtained

Data-lean fine-tuning of models for evaluating teacher performance in a GenAI-led elicitation simulation

Recent advances in large language models (LLMs) have led to the development of AI-powered tutoring systems that provide interactive support via dialogue. To enable these tutoring systems to provide personalized support, it is essential to assess student performance at each turn, motivating knowledge tracing (KT) in dialogue settings. However, existing dialogue-based KT approaches often ignore question difficulty modeling and rely on opaque latent representations from LLMs, hindering accurate and interpretable prediction. In this work, we propose an interpretable difficulty-aware conversational KT framework built upon LLMs, which explicitly models students' abilities and the tutor-posed tasks at each turn. The framework incorporates the original textual question and the next tutor-posed task to estimate the student's knowledge state and the difficulty of the upcoming turn. Furthermore, it integrates Item Response Theory to map LLM's outputs into student ability and question difficulty parameters, enabling interpretable prediction of student performance grounded in cognitive theories of learning. We evaluate the framework on two tutor-student dialogue datasets. Both quantitative and qualitative results show that our framework outperforms existing KT baselines, meanwhile generating interpretable outputs consistent with cognitive theory.

Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues

Rubrics are the primary reference for manual scoring of constructed responses, and there is growing interest in their use in automated scoring methodologies. In this work, we propose Aspect-Grounded Rubric–Answer Alignment (AGRAA), a rubric-based end-to-end scoring framework that models rubric descriptors as latent aspect spaces. Concretely, rubric descriptors are represented as low-dimensional subspaces derived from contextualised transformer embeddings, and student responses are scored according to how strongly their representations align with these rubric-induced spaces relative to the residual space outside them. This formulation provides a geometrically grounded interpretation of rubric-based scoring while enabling end-to-end training with standard transformer encoders. We introduce three distinct architectural variants and evaluate them on multiple short-answer and essay scoring datasets. Across these tasks, AGRAA achieves predictive performance highly competitive with strong neural and feature-based baselines. In addition, the framework yields interpretable intermediate representations that expose which rubric-defined aspects contribute to scoring decisions, enabling decision-aligned explanations grounded in rubric descriptors.

Rubrics as Semantic Subspaces: A Unified Approach to Rubric-based Constructed Response Scoring across Short Answers and Essays

This paper proposes a computational framework to operationalize optimal challenge based on Krashen's Input Hypothesis for open-ended conversational English as a Foreign Language (EFL) tutoring. Because conversational EFL tutoring relies on dynamically generated exercises rather than static item banks, traditional difficulty estimation methods cannot verify if a task is appropriately calibrated to a learner. Rather than treating difficulty as an intrinsic item property, we measure it directly from real-time interactional behavior, classifying exercises into three alignment states: Under-Challenged, Optimally Challenged, and Over-Challenged. Using 1,566 exercises from the Teacher-Student Chatroom Corpus, we first demonstrate that a learner's history of these states highly predicts their success on subsequent tasks. We then leverage this micro-level predictive model to construct an in-situ learning gain proxy, tracking shifts in a student's expected success probability across a session. Analyzing these macro-level trajectories, we find that sessions with higher proportions of over-challenging exercises systematically yield lower estimated gains. Crucially, increasing the proportion of optimally challenging interactions drives statistically significantly greater capability improvement than providing under-challenging tasks, providing quantitative evidence consistent with the Input Hypothesis in open-ended tutoring.

Measuring Optimal Challenge: Trajectory-Based Difficulty Alignment in Open-Ended Language Tutoring

Investigating the degree to which large language models (LLMs) affect teaching and learning in universities can help identify strategies for integrating LLMs in a way that supports, rather than undermines, student learning outcomes. This study examined how varying levels of LLM assistance affect writing performance, engagement, and perceived authorship. We report a pilot study in which 24 college students were randomly assigned to write a short essay with no LLM access, limited access (<=3 prompts, responses capped at 100 words), or unlimited access. Overall essay quality was statistically indistinguishable across groups. Yet writing behavior and perceived authorship diverged sharply: students with limited access reported higher ownership (62.5% would submit the essay as independent work, vs. 25% in the unlimited group), stronger organizational gains, and more strategic, revision-focused prompting. The unlimited group spent more time writing, produced essays more similar to LLM output, and reported reduced creative expression. Our findings suggest that constraining, rather than banning, LLM access may preserve authorship confidence while retaining the scaffolding benefits of AI assistance.

Effects of Varying LLM Access on Essay Writing Behavior

Improvements in computing resources and advances made in artificial intelligence have accelerated a systemic shift toward automating educational infrastructure. While the Natural Language Processing community has dedicated significant resources to developing educational technologies (EdTech) to address these pressures, it remains unclear whose interests are being best served among the many stakeholders involved. In this paper, we present a systematic literature review of papers published at venues within the Association for Computational Linguistics' Special Interest Group on Building Educational Applications (ACL SIGEDU) in 2024 and 2025, validated against EdTech papers from the wider ACL Anthology. By examining stakeholder inclusion and the prioritisation of research tasks, our findings reveal a critical tension: a push and pull between private-sector incentives and the foundational needs of educational infrastructure. Our analysis reveals that teachers are systematically under-represented as beneficiaries, that real-world deployment remains rare, and that ethical engagement tends toward acknowledgement rather than action. We identify exemplary research that successfully balances technical innovation with a responsible commitment to stakeholder agency and welfare, and derive from it actionable recommendations to ensure that EduNLP research continues to align with the theoretical goals of teaching and learning.

Incentives Of EdTech: A Systematic Review Of EduNLP Research

Large language models can generate feedback on free-form student writing, but it is unclear whether such feedback is correct and pedagogically useful. We evaluate LLM-generated feedback on 65 undergraduate proof-writing exercises using Hattie and Timperley's feedback framework and a grade agreement metric, comparing two models (GPT-4.1, GPT-5) under two workflow configurations graded by two independent LLM evaluators. A mark-scheme-augmented workflow improves grade correlation with human experts for both models, and its precomputed mark schemes allow instructors to audit the system before deployment. GPT-5 produces higher-quality feedback across all dimensions. The metrics we collect give some evidence that in the setting studied, feedback quality is high, and several sanity checks on our experiments support this finding. However, providing meaningful self-regulation support and controlled tests with students remain to be done. The results in this contribution show that feedback theory provides a useful lens for evaluating automated mathematical feedback.

Evaluating LLM-Generated Formative Feedback for Undergraduate Mathematics Through the Lens of Feedback Theory

Effective tutoring requires distinguishing optimal (expert-path), valid-alternative (valid but suboptimal), and incorrect student solutions, a distinction central to intelligent tutoring systems (ITS) yet untested for LLM-based tutors. As LLMs are increasingly explored as conversational complements to ITS for scalable feedback, evaluating their diagnostic precision is essential. We benchmark seven LLMs as feedback agents in propositional logic using knowledge-graph-derived ground truth across 10,836 solution-feedback pairs \footnote{\url{https://github.com/anonymouspaper64-debug/anonymous_paper.git}} and three prompt-level feedback conditions. Our analysis show that models achieve near-ceiling performance on optimal steps but systematically over-reject valid-alternative reasoning and over-validate incorrect solutions, precisely where adaptive tutoring matters most. These failure modes persist regardless of solution context or model capability, indicating architectural rather than informational limits. Accurate classification does not reliably translate into pedagogically actionable feedback, exposing a gap between diagnostic judgment and instructional effectiveness. LLMs thus cannot resolve the assistance dilemma without ITS-grounded mechanisms; they are better positioned within hybrid systems that combine LLM flexibility with ITS student modeling.

Downloads

Next from ACL 2026

From Metrics to Meaning: Rule-Grounded LLM Explanations for Data Literacy in Youth Football

Similar lecture

Generating Better Items for Cognitive Assessments Using Large Language Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES