United States

Mechanistic interpretability aims to reverse-engineer large language models (LLMs) into human-understandable computational circuits. However, the complexity of pretrained models often obscures the minimal mechanisms required for specific reasoning tasks. In this work, we train small, attention-only transformers from scratch on a symbolic version of the Indirect Object Identification (IOI) task, a benchmark for studying coreference-like reasoning in transformers. Surprisingly, a single-layer model with only two attention heads achieves perfect IOI accuracy, despite lacking MLPs and normalization layers. Through residual stream decomposition, spectral analysis, and embedding interventions, we find that the two heads specialize into additive and contrastive subcircuits that jointly implement IOI resolution. Furthermore, we show that a two-layer, one-head model achieves performance comparable to that of a multi-head model by composing information across layers primarily through query-key interactions. These results demonstrate that task-specific training induces highly interpretable, minimal circuits, offering a controlled testbed for probing the computational foundations of transformer reasoning.

ACL 2026

Emergence of Minimal Circuits for Indirect Object Identification in Attention-Only Transformers

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. <br>
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) <br><br><br>
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>
<br><br><br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

**Online Registration Form**: https://acl.swoogo.com/acl2026

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

technical paper

Logical fallacy detection models frequently over-flag valid reasoning due to reliance on surface-level spurious correlations, a brittleness that persists even in state-of-the-art benchmarks like CoCoLoFa despite strong in-distribution performance. This study introduces a targeted debiasing strategy using 703 LLM-generated counterfactually augmented pairs ---minimally differentiated valid and fallacious arguments matched on topic, structure, and wording---to isolate logical distinctions beyond superficial cues. Fine-tuning DeBERTa-v3-large on CoCoLoFa augmented with these pairs yields only marginal in-distribution improvement (+0.4\% F1 on the test set) but delivers substantial out-of-distribution robustness: a 58\% relative reduction in false positive rate (64\% 26.7\%) on a 300-sample balanced manual evaluation set curated from Reddit and other real-world sources. Zero-shot evaluation of recent LLMs (Llama-3.1-8B and Llama-3.3-70B) shows high performance under carefully optimized prompts (F1 90--94\%), but severe degradation under simple human-like prompts (F1 63--72\%, FPR rising to 54--74\%) and partial recovery with few-shot examples (F1 83--90\%, FPR 21--34\%). In contrast, the proposed lightweight and prompt-invariant approach achieves competitive robustness (F1 85.9\%, FPR 26.7\%) across all prompting regimes without any dependence on prompt engineering. This makes the method particularly stable for environments with unpredictable user input. The 703-pair counterfactual dataset, the fine-tuned model, and an interactive Hugging Face demo are publicly released to facilitate further research in misinformation detection and argumentation analysis.

Debiasing Logical Fallacy Detection for Real-World Robustness via Counterfactually Augmented Data

User-level ADHD-related text classification from social media is methodologically challenging because predictions must aggregate many short posts, performance can be inflated by direct diagnostic leakage, and screening-adjacent settings require calibrated probabilities rather than discrimination alone. We introduce a leakage-aware evaluation framework organized around two controlled axes: evidence budget, i.e., the number of tweets available per user, and leakage control. Within this setup, we compare document-level transformers, strong non-graph embedding-pooling baselines, and heterogeneous graph models combining semantic tweet embeddings, psycholinguistic features, and temporal structure. The main result is regime-dependent: graph aggregation is most useful when user evidence is scarce, whereas simple embedding pooling becomes highly competitive and often slightly stronger as more evidence becomes available. Overall, the main contribution is a controlled benchmarking framework and a clearer account of when graph-based aggregation is actually beneficial.

Leakage-Aware User-Level ADHD Signal Classification from Social Media: When Graph Aggregation Helps, and When It Does Not

Speaker diarization systems produce segmentation errors, such as false splits and boundary misplacements, that degrade transcript readability and downstream applications. We present CBAL (Context-Based Agentic Learning), a post-processing framework that refines segmentation boundaries in diarized scripts through targeted error correction. CBAL detects potential segmentation errors using acoustic and temporal heuristics and employs a lightweight LLM agent to reason about merge decisions, validating corrections through uncertainty-aware filtering with signal-based constraints. CBAL achieves 93.4\% accuracy across 359 applied merges and reduces segment count by 6.1\%. We demonstrate that our framework identifies and corrects high-confidence errors while maintaining 0\% degradation in terms of concatenated minimum-permutation Word Error Rate (cpWER). An ablation study confirms that each component contributes non-redundantly, demonstrating the viability of interpretable refinement frameworks that use the strengths of acoustic models and language understanding without requiring end-to-end retraining.

CBAL: Context-Based Agentic Learning for Speaker Diarization Segmentation Refinement

We study cross-lingual transfer by fine-tuning seven large language models (4B--671B parameters) on Arabic and evaluating zero-shot reading comprehension on Semitic languages and non-Semitic controls. Across dense and Mixture-of-Experts architectures, we find no evidence of Semitic-specific transfer: models with weak baselines improve dramatically across all languages, while strong-baseline models show only marginal gains regardless of language family. A chain-of-thought ablation reinforces this finding---the same models that benefit most from fine-tuning benefit equally from inference-time reasoning, suggesting both mechanisms address task-format alignment rather than cross-lingual knowledge transfer.

Disentangling Linguistic Relatedness from Task Alignment in Cross-Lingual Transfer

Organizations must continuously monitor evolving regulations to maintain compliance. While current tools are limited to surface-level text comparison, existing models lack the fine-grained classification schemes to determine whether small changes impact legal obligations or merely update formatting. To address this gap, we introduce a novel benchmark for change detection in EU regulations. It comprises 4,772 manually annotated pairs of structurally distinct provisions, defined as Atomic Legal Units (ALUs), mapped to a six-class taxonomy. We formalize three core tasks: structural alignment, change classification, and a combined task requiring simultaneous alignment and classification. Evaluating baselines from lexical algorithms to Large Language Models (LLMs), we find LLMs excel at isolated change classification, whereas domain-specific dense encoders offer the most robust combined performance. By providing fine-grained labeled data, this benchmark enables the development of AI systems that can help organizations analyze regulatory shifts and support version-aware retrieval in the legal domain.

RegTrack: A Fine-Grained Benchmark for Multi-Class Legal Change Detection

Reliable automatic evaluation is essential for benchmarking Machine Translation (MT) systems. However, their reliability is often underexplored in morphologically complex, non-English-centric contexts. Current metrics frequently exhibit "semantic blindness" toward subtle morphological shifts in morphologically rich languages like Finnish. This study proposes a framework that utilizes a controlled perturbation suite to systematically isolate and quantify the metric sensitivity to morphosemantic errors in Swedish to Finnish translation. It combines authentic MT outputs with a tiered adversarial corpus, subjecting reference translations. It will quantify discriminative ability and ordinal monotonicity by leveraging a diverse evaluation suite.

Thesis Proposal: Sensitivity of MT Evaluation Metrics to Morphosemantic Errors: A Case Study on Swedish–Finnish Translation

We present **Copyright Detective**, the first interactive forensic system for detecting, analyzing, and visualizing potential copyright risks in LLM outputs. The system treats copyright infringement versus compliance as an **evidence discovery** process rather than a static classification task due to the complex nature of copyright law. It integrates multiple detection paradigms, including content recall testing, paraphrase-level similarity analysis, persuasive jailbreak probing, and unlearning verification, within a unified and extensible framework. Through interactive prompting, response collection, and iterative workflows, our system enables systematic auditing of verbatim memorization and paraphrase-level leakage, supporting responsible deployment and transparent evaluation of LLM copyright risks even with black-box access. In our experiments with GPT-4o-mini, we demonstrate that specific persuasive strategy "Pathos" shift the leakage distribution from about 0.1 (ROUGE-L) to 0.7. Our live system is hosted on [Streamlit](https://copyright-detective.streamlit.app), and the source code will be made publicly available at [GitHub](https://github.com/changhu73/Copyright-Detective) upon acceptance. In addition, a [demo video](https://youtu.be/10gmwV3q_gg) is also included as supplementary material.

Copyright Detective : A Forensic System to Evidence LLMs Flickering Copyright Leakage Risks

We demonstrate Hindsight, a working memory system for AI agents that organizes long-term memory into four logical networks and exposes three core operations. The world, experience, observation, and opinion networks separate objective facts from subjective beliefs, giving developers clear visibility into what an agent knows versus what it believes. The retain, recall, and reflect operations handle memory ingestion, retrieval, and reasoning respectively. The retrieval layer combines vector search, keyword matching, graph traversal, and temporal filtering in a parallel pipeline backed by PostgreSQL with pgvector. Unlike existing memory systems such as MemGPT, Zep, and Mem0, Hindsight is the only system that jointly provides fact-belief separation, temporal entity graphs, evolving opinions with confidence scores, and configurable behavioral profiles. On the LongMemEval and LoCoMo benchmarks, Hindsight with a 20B open-source model achieves 83.6% and 83.2% accuracy, outperforming full-context GPT-4o and all prior memory systems; with Gemini-3 Pro, accuracy reaches 91.4% on LongMemEval. Our interactive demonstration lets users build memory graphs through multi-session conversations, inspect how memories are classified across networks, and observe how agent opinions form and change over time. The system is $\textbf{open-source under the MIT license}$ and available as a Python package (pip install hindsight-all) and Docker image, with $\textbf{1.9K GitHub stars}$ and 191 forks till date.A video demo is at https://youtu.be/4M2wS-yEmVA.

Hindsight: Structured Agent Memory that Retains, Recalls, and Reflects

Code sandboxes have emerged as a critical infrastructure for advancing the coding capabilities of large language models, providing verifiable feedback for both RL training and evaluation. However, existing systems fail to provide accurate verification and efficiency under high-concurrency workloads. We present ScaleBox, a high-fidelity and scalable system designed to address these limitations in large-scale code training. ScaleBox introduces automated special-judge generation and management, fine-grained parallel execution across test cases with seamless multi-node coordination, and a configuration-driven evaluation suite for reproducible benchmarking. A series of experiments demonstrates that ScaleBox significantly enhances code verification accuracy and efficiency. Our further RLVR experiments show that ScaleBox substantially improves both performance on LiveCodeBench and training stability, significantly outperforming heuristic-matching baselines. By providing a reliable and high-throughput infrastructure, ScaleBox facilitates more effective research and development in large-scale code training.

ScaleBox: Enabling High-Fidelity and Scalable Code Verification for Large Language Models

Checklists have emerged as a popular approach for interpretable and fine-grained evaluation, particularly with LLM-as-a-Judge. 
Beyond evaluation, these structured criteria can serve as signals for model alignment, reinforcement learning, and self-correction. To support these endeavors, we present $\texttt{AutoChecklist}$, an open-source library that unifies checklist-based evaluation into composable pipelines. At its core is a taxonomy of five checklist generation abstractions, each encoding a distinct strategy for deriving evaluation criteria. A modular $\textit{Generator $\rightarrow$ Refiner $\rightarrow$ Scorer}$ pipeline connects any generator with a unified scorer, and new configurations can be registered via prompt templates alone. The library ships with eight built-in pipelines implementing published approaches and supports multiple LLM providers (OpenAI, OpenRouter, vLLM). Beyond the Python API, the library includes a CLI for off-the-shelf evaluation and a web interface for interactive exploration. Validation experiments confirm that these checklist methods significantly align with human preferences and quality ratings, and a case study on ICLR peer review rebuttals demonstrates flexible domain adaptation. $\texttt{AutoChecklist}$ is publicly available at https://github.com/ChicagoHAI/AutoChecklist.

Downloads

Next from ACL 2026

Debiasing Logical Fallacy Detection for Real-World Robustness via Counterfactually Augmented Data

Similar lecture

Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2026

Debiasing Logical Fallacy Detection for Real-World Robustness via Counterfactually Augmented Data

Similar lecture

Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads