Singapore

Transformer-based language models excel at both recall (retrieving memorized facts) and reasoning (performing multistep inference), yet it remains unclear whether these functions rely on overlapping or separable internal circuits. Understanding these mechanisms is critical for building trustworthy and scientifically interpretable AI systems.We address this question through mechanistic interpretability, using controlled linguistic puzzles to probe transformer models at the layer, head, and neuron levels. By combining layer-wise activation tracing, attention-head specialization metrics, and causal activation patching, we identify subnetworks whose perturbation selectively disrupts either factual retrieval or reasoning. Across the Qwen, LLaMA-3 and Mistral families, we find a consistent pattern: the early and middle layers primarily support recall, while the deeper layers and specific MLP pathways enable reasoning. Interventions in distinct components lead to selective impairments: Disabling identified recall circuits reduces the factual precision by up to 15 % while leaving the reasoning intact, while disabling reasoning circuits yields a comparable drop in multistep inference. These findings offer causal, interpretable evidence that recall and reasoning arise from partially distinct but complementary computational processes, advancing the mechanistic foundations of explainable AI.

AAAI 2026

Disentangling Recall and Reasoning in Transformer Models through Layer-wise Attention and Activation Analysis

## Welcome to EMNLP 2024! 
We are excited to welcome you to one of the most prominent conferences in the field of Natural Language Processing. This year, EMNLP 2024 is being held in a hybrid format,
offering both virtual and in-person participation in beautiful Miami. Due to a record-breaking number of submissions, we've expanded the total number of accepted papers to accommodate more cutting-edge research from around the globe.
### [Conference Handbook](https://drive.google.com/file/d/1WPROgxjLAC96AJL7Ugy0tEnYm7dkrbHt/view?usp=sharing)

You are required to register for this event. **Please register [here](https://2024.emnlp.org/registration/).** The EMNLP 2024 event page on Underline will be open to public one week prior to the event.

Please register!

EMNLP 2024

EMNLP 2024 will take place in Miami, Florida from Nov 12th to Nov 16th, 2024, at the Hyatt Regency Miami Hote and on Underline for remote participants.

workshop paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

There is a substantial body of literature examining the mathematical reasoning capabilities of large language models (LLMs), particularly their performance on precise arithmetic operations in autoregressive architectures. However, their ability to perform approximate reasoning in informal, fast-paced mathematical operations has received far less attention, especially among non-autoregressive decoder models. Our work addresses this gap by introducing StreetMath, a benchmark designed to evaluate models’ approximation abilities under real-world approximation scenarios. We conduct extensive evaluations across different LLM architectures: Qwen3-4B-Instruct-2507, Qwen3-4B-Thinking-2507, Dream-v0-Instruct-7B, Falcon-Mamba-7B-Instruct, and Mamba-GPT-3B. Furthermore, we apply mechanistic interpretability techniques to probe their internal computational states. Our analysis reveals that LLMs generally attempt to compute exact values or invoke external tools even in tasks that call for approximation. Moreover, while models sometimes reach the correct answer in early layers or steps, they still consume more tokens when solving approximation tasks. Additional experiments indicate that exact and approximate arithmetic operations rely on largely separate neural components. Drawing upon research on cognitive psychology, we argue that LLMs do not exhibit cognitive miserliness in the same way humans do in street math settings. We open source our work https://anonymous.4open.science/r/StreetMath-1/

StreetMath: Understanding How LLMs Approximate Mathematical Reasoning

Retrieval-Augmented Generation (RAG) systems enhance large language models (LLMs) by grounding responses in external evidence, mitigating hallucinations, and enabling access to up-to-date, domain-specific knowledge. However, existing RAG frameworks often suffer from unstable self-supervised optimization signals and inconsistent factual grounding. We introduce CREAM-RAG (Consistency-Regularized Enhanced Augmented Model for RAG), a unified framework that integrates retrieval, Direct Preference Optimization (DPO)-based self-reward reinforcement learning, and a consistency regularization objective to stabilize reward dynamics during fine-tuning. By enforcing alignment between multiple retrieved contexts and generated responses, CREAM-RAG improves factual faithfulness and semantic coherence without external supervision. Empirical evaluations on the LLaMA-2-7B model demonstrate that CREAM-RAG achieves a 35.04% average improvement over the base model across reasoning and factuality benchmarks, highlighting its effectiveness in reducing hallucinations and enhancing retrieval-grounded reasoning.

CREAM-RAG: Enhanced Retrieval Augmented Generation to Limit Hallucination through Consistency-based Self-RAG

Large language models (LLMs) often generate fluent but factually incorrect statements despite having access to relevant evidence, a failure mode rooted in how they allocate attention between contextual and parametric knowledge. Understanding and steering this internal behavior is key both for trustworthy deployment and for scientific interpretability of model mechanisms. We introduce COMPASS (Context-Modulated PID Attention Steering System), a lightweight, interpretable control framework that embeds a model-based feedback loop directly within decoding. COMPASS quantifies context reliance via a transparent metric, the Context Reliance Score (CRS), which serves as an online probe of how attention heads ground generation in evidence. Using this interpretable signal, a PID controller dynamically modulates attention heads to maintain factual consistency without retraining or multi-pass decoding. Across benchmarks (HotpotQA, XSum, HaluEval, RAGTruth), COMPASS consistently reduces contextual hallucination rates (2.8–5.8\% absolute) while revealing how distinct attention heads contribute to evidence alignment. These results highlight feedback-driven interpretability as a pathway toward scientific understanding of LLM behavior.

COMPASS: Context-Modulated PID Attention Steering System for Hallucination Mitigation

Large reasoning models trained via reinforcement learning (RL) substantially outperform their supervised counterparts on tasks requiring logic and mathematical reasoning, yet the mechanistic basis for these improvements remains unclear. We investigate this phenomenon through an integrated behavioral-mechanistic analysis of mathematical reasoning, asking: what internal differences enable RL models' superior performance? We present three converging lines of evidence that RL models develop higher-quality reasoning representations earlier in their networks. First, linear probes trained on layer-wise hidden states reveal that RL models achieve 11% higher accuracy in predicting answer correctness, with this capability emerging earlier than in base models—suggesting RL training produces more structured, reasoning-relevant representations throughout the network. Second, soft-mean ablation studies show RL models exhibit greater performance degradation when early-layer representations are perturbed, confirming that critical reasoning computations occur earlier and that these representations are functionally important for downstream generation. These representational differences manifest behaviorally in how models allocate computational resources during generation. Analyzing within-problem token generation variability across 50 responses per problem, we find that Qwen3-Thinking maintains remarkably consistent token usage (CV = 0.25-0.33) across all difficulty levels while achieving 97.7% accuracy, suggesting its superior representations enable efficient, reliable solution paths. In contrast, DeepSeek-Math models exhibit elevated variability specifically at capability boundaries (CV >0.8 at 20-60% accuracy), indicating adaptive computational effort where problems strain model capacity—yet DeepSeek's RL and supervised variants show nearly identical variability profiles, suggesting current reward structures may not fully exploit the potential for adaptive token allocation. Together, these findings suggest that RL training fundamentally restructures how models represent and process reasoning problems, building more robust and accessible reasoning representations. This mechanistic understanding provides actionable insights for designing more reliable reasoning systems and highlights the value of probing internal representations to understand capability improvements.

Probing the Origins of Reasoning Performance: Representational Quality for Mathematical Problem-Solving in RL vs SFT Finetuned Models

Recent studies have revealed that LLMs can exhibit behavioral self-awareness — the ability to accurately describe or predict their own learned behaviors without explicit supervision. This capability raises safety concerns, as it may allow models to intentionally conceal their true abilities during evaluation. We attempt to better understand this phenomenon by investigating how and when self-awareness emerges during fine-tuning, and whether it can be mechanistically localized. Through controlled fine-tuning experiments on instruction-tuned LLMs with low-rank adapters (LoRA), we find: (1) that across domains and layers, fine-tuning consistently elicits self-aware behavior early on in the training process; (2) that self-awareness can be reliably induced using a single rank-1 LoRA adapter; and (3) that the learned self-aware behavior is largely captured by a single steering vector in activation space, recovering nearly all of the fine-tune’s behavioral effect. Together, these findings suggest that self-awareness exhibits a rapid transition and is captured by a linear direction rather than a distributed introspective mechanism.

Emergent Mechanisms of Self-Awareness in LLMs

We introduce a diagnostic for symmetry learning in PDE surrogates: the influence function computed across symmetry- related states. On compressible Euler flows, our diagnostic reveals that a UNet exhibits partial but unstable influence across square group actions and translations, whereas a ViT reaches lower prediction error yet shows largely orthogonal updates across orbits. This exposes an optimization-symmetry tradeoff: stronger inductive biases promote data efficiency but can couple updates rigidly; flexible architectures optimize easily but ignore physical structure. Our diagnostic offers a reproducible test for whether training dynamics propagate information across symmetry orbits, a necessary ingredient for robust generalization in scientific machine learning.

Loss Landscape Geometry and the Learning of Symmetries: Or, What Influence Functions Reveal About Memorization and Generalization

Additive manufacturing enables the layer-by-layer fabrication of complex and customizable structures but remains prone to internal defects that are difficult to detect during fabrication. This work presents an anomaly detection framework for real-time defect monitoring using a custom-designed wireless antenna sensor. The sensor consists of an aluminum split-ring resonator patterned with gold conductive ink and mounted on a ceramic substrate. It passively responds to structural and thermal changes during the manufacturing process by producing frequency-domain scattering parameter signals. To process these high-dimensional signals without requiring defect-labeled data, we develop a zero-bias deep neural network trained exclusively on normal-class samples. The model constructs a class-structured latent space using cosine similarity to learned templates and applies Mahalanobis distance to define statistical decision boundaries. Anomalies are identified when inputs deviate significantly from the expected distribution of normal data. The framework is designed around two deployment-oriented principles. Explainability enables users and domain experts to understand detection decisions through latent space visualization, feature attribution, and statistical reasoning that connects model outputs to interpretable signal characteristics. Instructibility allows users to tune detection sensitivity through threshold control without retraining, adapting the system to different quality requirements and process tolerances. The system is evaluated using experimental data collected from the antenna sensor under controlled thermal variation. Results demonstrate that the proposed method accurately detects anomalous signal patterns without using defect supervision, while maintaining interpretability and user control. This approach offers a transparent, adaptive, and data-efficient solution for in situ defect monitoring in additive manufacturing environments.

Anomaly Detection for Antenna-Based Defect Detection in Additive Manufacturing: An Explainable, Instructive Learning Framework

Reports of “emergent” capabilities in transformer-based LLMs (abrupt, non-linear improvements in task performance) remain controversial due to post-hoc measurement and ambiguous definitions. We investigate whether such transitions can be predicted pre-hoc from internal representations. For each training checkpoint, we train sparse autoencoders (SAEs) on model activations, construct a co-activation graph over SAE features, and track graph statistics (e.g. density, clustering, etc.). We analyze a 2-layer grokking-style transformer with aggregate graph metrics over eight SAE initializations, and test lead-lag relationships between changes in graph metrics and subsequent changes in accuracy under a formalized emergence criterion. Across both settings, we find no statistically significant evidence that global co-activation topology forecasts emergent jumps in performance. If pre-hoc indicators exist, they may lie outside the global graph measures analyzed here (e.g., in task-specific circuits or localized subgraphs).

Predicting Emergent Capabilities Using Sparse Features

We present a differentiable framework that leverages the Discrete Empirical Interpolation Method (DEIM) for interpretable deep learning and dynamical system analysis. Although DEIM efficiently approximates nonlinear terms in projection-based reduced-order models (POD-ROM), its fixed interpolation points limit the adaptability to complex and time-varying dynamics. To address this limitation, we first develop a differentiable adaptive DEIM formulation for the one-dimensional viscous Burgers’ equation, which allows neural networks to dynamically select interpolation points in a computationally efficient and physically consistent manner. We then apply DEIM as an interpretable analysis tool for examining the learned dynamics of a pre-trained Neural Ordinary Differential Equation (NODE) on a two-dimensional vortex-merging problem. The DEIM trajectories reveal physically meaningful features in the NODE’s learned dynamics and expose its limitations when extrapolating to unseen flow configurations. These findings demonstrate that DEIM can serve not only as a model reduction tool but also as a diagnostic framework for understanding and improving the generalization behavior of neural differential equation models.

Towards Interpretable Deep Learning and Analysis of Dynamical Systems via the Discrete Empirical Interpolation Method

Classifying non-Hermitian energy spectra under open boundary conditions is an open challenge in physics. This classification is a critical prerequisite for the rational inverse design of systems exhibiting desired dynamics and topological responses. While graph topology has emerged as a promising approach for characterizing these spectra, systematic methods for distilling non-Hermitian spectra into their corresponding graph representations have been lacking. Moreover, the resulting graphs often exhibit complexities that defy manual classification, necessitating machine learning approaches. In this work, we introduce a two-step framework for classifying non-Hermitian spectra based on their graph topologies. The first step employs $\texttt{Poly2Graph}$, an automated, high-performance pipeline that distills non-Hermitian spectra into $\textit{spectral graphs}$ suitable for graph neural networks (GNNs). The second step involves generating a large dataset of these spectral graphs and training a GNN for classification. We propose $\texttt{GnLTransformer}$, a novel architecture featuring dual channels that leverage line graphs to explicitly capture higher-order topological features. $\texttt{GnLTransformer}$ achieves over 99% classification accuracy on our dataset, outperforming standard baselines by 32%. Notably, beyond conventional GNNs, $\texttt{GnLTransformer}$ offers inherent explainability regarding higher-order topology. As a further contribution, we release a new $\textit{multi-graph}$ dataset comprising over 117K spectral graphs.

Content not yet available

Next from AAAI 2026

StreetMath: Understanding How LLMs Approximate Mathematical Reasoning

Similar lecture

Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES