United States

Vision-Language Models (VLMs) excel at multimodal tasks, but they remain vulnerable to hallucinations that are factually incorrect or ungrounded in the input image. Recent work suggests that hallucination detection using internal representations is more efficient and accurate than approaches that rely solely on model outputs. However, existing internal-representation-based methods typically rely on a single representation or detector, limiting their ability to capture diverse hallucination signals. In this paper, we propose EnsemHalDet, an ensemble-based hallucination detection framework that leverages multiple internal representations of VLMs, including attention outputs and hidden states. EnsemHalDet trains independent detectors for each representation and combines them through ensemble learning. Experimental results across multiple VQA datasets and VLMs show that EnsemHalDet consistently outperforms prior methods and single-detector models in terms of AUC. These results demonstrate that ensembling diverse internal signals significantly improves robustness in multimodal hallucination detection.

ACL 2026

EnsemHalDet: Robust VLM Hallucination Detection via Ensemble of Internal State Detectors

### Welcome to The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

Message from the General Chair: 
*It is my great pleasure and honor to welcome you to the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025), held in beautiful Vienna, Austria, from July 27 to August 1, 2025. ACL2025continues our field’s tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.* 
[Read more](https://drive.google.com/file/d/1GI_hvOpjswAuYdUTromfeDiPpCcqidwg/view?usp=sharing)

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2025". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you need to log in with the **email address you registered with**. 

ACL 2025

Welcome to The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

poster

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. 
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) 
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

**Online Registration Form**: https://acl.swoogo.com/acl2026

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

Lean proofs are built as sequences of tactic-induced state transitions, yet learned models often represent proof steps primarily through tactic strings or raw proof-state text. Building on Delta Tokens, which encode a proof step by the local edit it induces between successive proof states, we train an encoder-only Transformer to learn contextualized representations of Lean proof steps from state changes. We then use these step representations to study complete proofs as trajectories in a learned latent space. We first show that the Delta-based Transformer yields better held-out next-tactic retrieval than a matched surface-syntax control, supporting the representational choice used in the trajectory analysis. We then analyze proof trajectories using path length, endpoint span, directness, curvature, and torsion. Across the LeanWorkbook slice used here, longer proofs become increasingly indirect within a relatively bounded latent span: path length grows sharply with proof length while endpoint span changes little, mean step size decreases, curvature rises modestly, and torsion falls. Qualitative case studies show that these geometric patterns align with recognizable proof organizations, including immediate closure, aligned accumulation, scaffolded enrichment, bookkeeping-heavy restructuring, and repeated local contradiction work. The dataset is small and heavily skewed toward short proofs, so the claims are necessarily limited. Within those limits, the results suggest that learned state-change representations recover nontrivial structure in how proofs unfold and provide a promising basis for future trajectory-aware theorem proving.

Proofs as Trajectories: Learning Lean Proof-Step Representations from State Changes

Large language models (LLMs) exhibit substantial cross-lingual variation in mathematical reasoning performance, but it remains unclear whether these differences reflect language-specific parameters or a shared mechanism that manifests differently by language. We present a cross-lingual mechanistic analysis of mathematical reasoning in LLMs, enabling us to localize and compare model parameters that support mathematical reasoning across languages. We find that mathematical reasoning is supported by a partially shared set of parameters, with the strongest overlap concentrated in intermediate model layers. We further observe that English consistently produces the largest set of math-relevant parameters, whereas lower-resource languages reveal smaller sets of relevant parameters. These results indicate that math-related reasoning in multilingual LLMs is neither fully language-invariant nor fully language-specific, but instead relies on a shared parameter core with systematic cross-lingual differences.

LLM Parameters for Math Across Languages: Shared or Separate?

Large language models show strong capabilities in natural language generation and have been applied to translate complex structured data into human-readable insights. While these models excel at surface-level fluency, they remain unreliable as they produce factually inaccurate outputs and struggle with consistent logical inference beyond surface-level patterns. Moreover, they often lack a clear sense of relevance and produce shallow or uninformative insights. This thesis proposal argues that a key source of these limitations is the fact that tasks are often underspecified, requiring models to make implicit assumptions about missing context. We will investigate how such underspecification leads to unintentional assumptions and how these affect faithfulness and evaluation. We will examine how models can identify missing premises and surface multiple plausible interpretations to improve factuality of their inferences and make their evaluation more rigorous. We will also study how underlying assumptions influence the perceived interestingness of insights. Finally, we will explore how to improve reasoning to enable deeper inferences, focusing on code generation and qualitative reasoning. By shifting focus from surface-level generation to assumption-aware deeper inferences, this work aims to improve reliability, interpretability, and user controlability in NLG.

Thesis Proposal: Intentional Inference for Insight Generation

Large language models and LLM-based agents are widely deployed as personal chat assistants, enterprise copilots, and autonomous workflow agents. Across all these applications, memory (the ability to retain, access, and reason over information accumulated over long contexts and multiple interactions) plays a crucial role in determining the reliability of any agent. We introduce RECON (Reasoning over Extended Contexts with Obfuscated Narratives), a benchmark for evaluating compositional temporal reasoning over long contexts. RECON uses investigation case file (100k tokens) and tests agents on \textbf{six memory-intensive tasks}: reconstructing multi-hop evidence chains, propagating cascading invalidations, resolving source conflicts, counterfactual reasoning, satisfying temporal constraints, and temporal fact retrieval. Recent memory benchmarks evaluate whether agents can retrieve scattered facts or detect if a fact has changed; RECON evaluates what happens after the change, whether agents can trace which downstream conclusions are affected, which survive through independent support, and how alternative timelines would have unfolded. We evaluate systems across three architectural families comprising long-context LLMs, retrieval-augmented generation (RAG) agents, and commercial memory agents and find that existing memory mechanisms are not yet sufficient for reliable reasoning over long, evolving narratives. Our benchmark and code are available at \url{https://anonymous.4open.science/r/recon-benchmark/README.md}.

RECON: Benchmarking Agent Memory for Compositional Reasoning over Long Contexts

To accelerate scientific knowledge acquisition, LLMs are increasingly used to synthesize multiple papers into structured tables by inferring schemas and values. While value generation within a fixed schema can often be reduced to extractive question answering, the schema generation problem, determining which dimensions to compare a set of documents, lacks a formal mapping to standard NLP tasks. In this work, we formulate schema generation as a reinforcement learning problem and investigate whether these dimensions can be induced without access to gold-standard schemas. We design a multi-faceted reward framework capturing schema coverage, non-redundancy, relevance, and format, and train a small language model on a literature review dataset. Our approach yields consistent improvements over the untuned base model across intrinsic, reference-based, and LLM-judge metrics, and remains competitive with supervised fine-tuned models at 5$\times$ the parameter count on structural and diversity dimensions.\footnote{Code will be released upon publication.}

Reference-Free Schema Generation for Literature Review Tables via Multi-Faceted Rewards

The widespread integration of large language models (LLMs) across diverse domains, from entertainment to healthcare and banking, has underscored the pressing need to address the risks associated with artificial intelligence (AI). Indeed, as LLMs have grown in scale, new challenges have emerged, including privacy breaches, dissemination of misinformation, issues related to robustness and alignment, and concerns regarding the societal biases ingrained in these models. In response, there has been a surge in research focused on responsible AI and AI safety. To address these challenges, recent literature has introduced novel techniques for mitigating the risks associated with LLMs and testing the efficacy of safety protocols through red-teaming. Despite these efforts, concerns persist regarding the perpetuation of biases in next-generation LLMs and the lack of robustness in existing safety measures. In this thesis proposal, we argue that some failures in LLM safety arise from flawed evaluation practices and explore ways to more effectively assess and characterize them.

Thesis Proposal: Rethinking Safety Evaluation in Large Language Models

When large language models encounter conflicts between parametric knowledge and contextual evidence, does the parametric circuit's contribution change, or is the conflict resolved at the output distribution? We apply a three-level analysis framework, decomposing conflict resolution into circuit-level (parametric-site logit contribution, $\Phi_P$), pathway-level (MLP vs.\ attention), and output-level (token rank) effects across four transformer architectures spanning three attention types and two scale points. We find that $\Phi_P$ is near-invariant under conflict ($\leq$1.8\% variation), while output token ranks shift substantially, establishing that conflicts resolve through additive logit-space competition. Activation patching confirms parametric sites are causally sufficient and necessary, with contextual heads exerting 30--40$\times$ weaker influence. Layer-by-layer analysis reveals three architecturally determined resolution mechanisms, and parametric--contextual coupling decreases substantially with scale, exceeding random-subspace predictions. Domain generalization shows parametric sites are stratified by knowledge complexity with no overlap, yet the resolution mechanism generalizes across domains.

Dissociating Circuit-Level and Distribution-Level Effects of Knowledge Conflicts in LLMs

Before a tax authority can issue a ruling, it must receive a complete description of the taxpayer's situation---yet no benchmark measures whether language models can systematically elicit all relevant facts through dialogue. We introduce the \textbf{Factual State Discovery Benchmark}, in which a discovery agent questions a simulated taxpayer grounded in a real tax document. The dataset comprises 500 narratives from official Polish tax interpretations, decomposed into 32\,874 atomic facts with validated supported precision (97.6\%), atomicity (93.8\%), and sentence coverage (96.0\%). Experiments with four models show that even the best system recovers only 77\% of facts on easy samples and under 49\% on hard samples after 50 turns. These findings establish conversational fact elicitation as a challenging open problem requiring retrieval-augmented and adaptive questioning strategies.

Factual State Discovery Benchmark: Evaluating Fact Elicitation in Polish Tax Law

Uncertainty quantification (UQ) methods for large language models are predominantly designed by hand based on domain knowledge and heuristics, limiting their scalability and generality. We apply LLM-powered evolutionary search to automatically discover unsupervised UQ methods represented as Python programs. On the task of atomic claim verification, our evolved methods outperform strong manually-designed baselines, achieving up to 6.7\% relative ROC-AUC improvement across 9 datasets while generalizing robustly out-of-distribution. Qualitative analysis reveals that different LLMs employ qualitatively distinct evolutionary strategies: Claude models consistently design high-feature-count linear estimators, while Gpt-oss-120B gravitates toward simpler and more interpretable positional weighting schemes. Surprisingly, only Sonnet 4.5 and Opus 4.5 reliably leverage increased method complexity to improve performance~-- Opus 4.6 shows an unexpected regression relative to its predecessor. Overall, our results hint that LLM-powered evolutionary search is a promising paradigm for automated, interpretable hallucination detector design.

Evolutionary Search for Automated Design of Uncertainty Quantification Methods

Recursive models that iteratively refine latent representations have demonstrated strong performance on variety of reasoning tasks. However, these models only control if and when to stop early, not how computation is distributed. In this work, we introduce shortcut reasoning, a framework for distilling recursive latent reasoning into a multiscale jump model that enables flexible test-time compute. We reinterpret recursive reasoning as a latent-time dynamical process and train a student model to predict the effect of multiple reasoning steps at once. To ensure robustness, we augment shortcut transitions with a repair mechanism, where a denoising base-model update projects approximate latent states back to a valid reasoning manifold. We further introduce stepwise improvement supervision, encouraging each shortcut step to increase the likelihood of the correct answer. Experiments on ARC-AGI show that our approach achieves competitive accuracy compared to recursive baselines while requiring fewer sequential updates.

Downloads

Next from ACL 2026

Proofs as Trajectories: Learning Lean Proof-Step Representations from State Changes

Similar lecture

Prompt-Guided Internal States for Hallucination Detection of Large Language Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2026

Proofs as Trajectories: Learning Lean Proof-Step Representations from State Changes

Similar lecture

Prompt-Guided Internal States for Hallucination Detection of Large Language Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads