Singapore

Catastrophic forgetting remains a critical challenge in continual learning for large language models (LLMs), where models struggle to retain performance on historical tasks when fine-tuning on new sequential data without access to past datasets. In this paper, we first reveal that the drift of functional directions during the fine-tuning process is a key reason why existing regularization-based methods fail in long-term LLM continual learning. To address this, we propose Dynamic Orthogonal Continual (DOC) fine-tuning, a novel approach that tracks the drift of these functional directions and dynamically updates them during the fine-tuning process. Furthermore, by adjusting the gradients of new task parameters to be orthogonal to the tracked historical function directions, our method mitigates interference between new and old tasks. Extensive experiments on various LLM continual learning benchmarks demonstrate that this approach outperforms prior methods, effectively reducing catastrophic forgetting and providing a robust tool for continuous LLM fine-tuning.

AAAI 2026

Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgetting of LLMs

large language models

continual learning

workshop paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.<br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

Intrinsic self-correction refers to the phenomenon where a language model refines its own outputs purely through prompting, without external feedback or parameter updates. While this approach improves performance across diverse tasks, its internal mechanism remains poorly understood. We analyze intrinsic self-correction from the representation shift induced by prompting. We formalize and introduce the notion of a prompt-induced shift, which is the change in hidden representations caused by a self-correction prompt. Across 5 open-source LLMs, prompt-induced shifts in text detoxification and text toxification align with latent directions constructed from contrastive pairs. In detoxification, the shifts align with the non-toxic direction; in toxification, they align with the toxic direction. These results suggest that intrinsic self-correction functions as representation steering along interpretable latent directions. Our analysis highlights an understanding of model internals can be a direct route to analyzing the mechanisms of prompt-driven LLM behaviors.

Intrinsic Self-Correction in LLMs: Towards Explainable Prompting via Mechanistic Interpretability

We present AI Screener, an end-to-end automated document review system that integrates a 12-billion-parameter pretrained large language model with a Tree-of-Thought reasoning framework to emulate and scale expert-level decision-making. Designed for high-stakes, domain-specific analysis, AI Screener empowers subject matter experts to encode their domain knowledge and reasoning processes in a no-code, efficient manner—enabling rapid customization without technical barriers. The system has been deployed across three different and unrelated mission-critical business functions: (1) accelerating scientific literature reviews to support the development of occupational exposure limits for worker health protection, (2) streamlining patent screening to optimize intellectual property portfolio management, and (3) automating procurement contract analysis to identify value leakage and drive better commercial terms. Across these diverse deployments, subject matter experts encoded their knowledge with AI Screener to transform traditional workflows—significantly reducing manual review time while maintaining expert-grade accuracy and consistency. This work highlights how Tree-of-Thought-augmented LLMs can be pragmatically applied to reshape enterprise document intelligence at scale.

Tree-of-Thought-Augmented LLMs for Automated Document Review Across Industrial Domains

Bias and fairness remain persistent challenges in the responsible deployment of machine learning systems. While most existing metrics are designed for binary classification, fairness evaluation for regression models, widely used in domains such as risk scoring, pricing, and demand forecasting, remains comparatively underexplored. We introduce a quantile-conditioned fairness framework for regression that extends conditional fairness assessment from binary to continuous outcomes. The proposed method partitions target values into quantiles, computes group-to-complement prediction ratios within each segment, and then aggregates these ratios to produce interpretable fairness scores. Through a series of controlled ablation studies on synthetic data, we analyze the effects of bias strength, protected group imbalance, and model performance. We also benchmark our solution against the open-source Dalex fairness toolkit. We further show that the same conditioning principle naturally extends to multiclass classification, treating each class as a conditioning bucket. Real-world case studies on regression and classification datasets demonstrate the practical utility of our approach. Our implementation is lightweight and easily integrable into existing model development workflows, providing a deployable framework for fairness evaluation for all domains.

Quantile-Conditioned Fairness: Extending Binary Fairness Evaluation to Continuous Outcomes

AI inference scaling is often tuned through 1D heuristics (a fixed reasoning passes) or 2D bivariate trade-offs (e.g., performance vs.\ compute), which fail to consider cost and latency constraints. We introduce a 3D optimization framework that jointly calibrates accuracy, cost, and latency within a unified decision space, enabling constraints-aware inference scaling. Using Monte Carlo simulations across three representative scenarios and nine simulated large language models, we evaluate four optimization methods to address the 3D multi-objective optimization (MOO) problem. Framing inference scaling in MOO shapes a feasible space that 1D and 2D optimizations fail to capture, enabling environment-adaptive selection of the inference scaling~$k$. Results show that knee-point optimization achieves the best balance, while accuracy-maximization remains favorable when precision is prioritized. The framework establishes a theoretical foundation for deployment-aware inference scaling across diverse operational contexts.

3D Optimization for AI Inference Scaling: Balancing Accuracy, Cost, and Latency

Vision Language Models (VLMs) are rapidly advancing in multimodal understanding, largely due to increases in training data and model size. However, the computational demands of VLMs are also increasing, presenting challenges for their deployment in resource-constrained environments like smart televisions (TVs). VLMs are particularly useful in smart TVs for applications such as AI summarization, content question-and-answer (QnA), and user interface (UI) understanding and navigation. Previous research has focused on leveraging layer redundancy to accelerate VLM inference, but these methods often depend on extensive training, which requires significant computational resources and time. Recently, some inference-based solutions for layer skipping in Large Language Models (LLMs) have been proposed. In this paper, we demonstrate that the metrics used for layer skipping in LLMs do not always yield favorable results when applied to VLMs. We introduce an inference-only layer skipping strategy for VLMs based on attention to image tokens. Our approach increases overall throughput by up to 21% while maintaining close to baseline performance.

FlashVLM: Exploiting Layer Redundancy via Visual Attention for Efficient Vision-Language Inference

Accurate long-term traffic forecasting remains a critical challenge in intelligent transportation systems, particularly when predicting high-frequency traffic phenomena such as shock waves and congestion boundaries over extended rollout horizons. Neural operators have recently gained attention as promising tools for modeling traffic flow. While effective at learning function space mappings, they inherently produce smooth predictions that fail to reconstruct high-frequency features such as sharp density gradients which results in rapid error accumulation during multi-step rollout predictions essential for real-time traffic management. To address these fundamental limitations, we introduce a unified Diffusion-Enhanced Transformer Neural Operator (DETNO) architecture. DETNO leverages a transformer neural operator with cross-attention mechanisms, providing model expressivity and super-resolution, coupled with a diffusion-based refinement component that iteratively reconstructs high-frequency traffic details through progressive denoising. This overcomes the inherent smoothing limitations and rollout instability of standard neural operators. Through comprehensive evaluation on chaotic traffic datasets, our method demonstrates superior performance in extended rollout predictions compared to traditional and transformer-based neural operators, preserving high-frequency components and improving stability over long prediction horizons.

DETNO: A DIFFUSION-ENHANCED TRANSFORMER NEURALOPERATOR FOR LONG-TERM TRAFFIC FORECASTING

Learned database query optimizers typically optimize over a set of configurations, which limits the attainable plan space. Optimus expands the action space itself by mining novel execution plan rewrites and learns to select among these actions online. Optimus utilizes graph-based inductive matrix completion and a multilayer perceptron with the objective of minimizing latency. Crucially, the system is deployable by design: it requires no engine modifications and its rules include guard-checked compilation. On the extended JOB benchmark, Optimus yields a speedup of 1.16x over vanilla PostgreSQL solely using novel rewrites.

Optimus: Deployable Query Optimization via Novel SQL Rewrites

Although Large Language Models (LLMs) have expanded the capabilities of recommender systems, they are hindered by inherent limitations, including a propensity for factual hallucination and reliance on outdated domain knowledge. These constraints pose significant challenges in contexts requiring high-fidelity recommendations, such as consumer electronics purchases where precise, current specifications are critical. To address these issues, this paper proposes a novel framework termed Dynamic Graph-Retrieval Augmented Generation (Graph-RAG), which integrates LLMs, knowledge graphs, and RAG techniques within a multi-agent architecture. The framework dynamically deciphers complex user purchase intents and prioritizes decision-critical factors through a modular communication protocol that enables cross-agent collaboration, a `Specification Vector Index' that resolves semantic disparities between natural language queries and technical attributes, and a graph-based dynamic retrieval engine that facilitates fact-grounded reasoning. Empirical validation forms a pivotal contribution of this work, with rigorous experimental verification confirming the system's efficacy in minimizing hallucinations through structured knowledge grounding. Quantitative metrics demonstrate statistically significant improvements in recommendation accuracy, such as a +22.7% increase in precision, and reliability, while traceable decision pathways enhance operational transparency. This research delivers a foundational architecture for a possible practical recommender systems, validated through real-world deployment scenarios and test dataset, and establishes a benchmark for empirically substantiated innovation in AI-driven recommendation frameworks. By bridging theoretical innovation and practical deployment, this study marks a critical advancement in the field, offering both a new methodology and robust evidence of its real-world applicability.

Dynamic Graph-Retrieval Augmented Generation for Reliable and Explainable Consumer Electronics Recommendation

Large Language Models require substantial computational resources for inference, posing deployment challenges. While dynamic pruning offers superior efficiency over static methods through adaptive circuit selection, it exacerbates alignment degradation by retaining only input-dependent safety-critical circuit preservation across diverse inputs. As a result, addressing these heightened alignment vulnerabilities remains critical. We introduce Alignment-Aware Probe Pruning (AAPP), a dynamic structured pruning method that adaptively preserves alignment-relevant circuits during inference, building upon Probe Pruning. Experiments on LLaMA 2-7B, Qwen2.5-14B-Instruct, and Gemma-3-12B-IT show AAPP improves refusal rates by 50% at matched compute, enabling efficient yet safety-preserving LLM deployment.

Alignment-Constrained Dynamic Pruning for LLMs: Identifying and Preserving Alignment-Critical Circuits

Aligning proprietary large language models (LLMs) with internal organizational policies has become an urgent priority as organizations increasingly deploy LLMs in sensitive domains such as legal support, finance, and medical services. Beyond generic safety filters, enterprises require reliable mechanisms to detect policy violations within their regulatory and operational frameworks, where breaches can trigger legal and reputational risks. Existing content moderation frameworks, such as guardrails, remain largely confined to
the safety domain and lack the robustness to capture nuanced organizational policies. LLM-as-a-judge and fine-tuning approaches, though flexible, introduce significant latency and lack interpretability. To address these limitations, we propose a training-free and efficient method that treats policy violation detection as an out-of-distribution (OOD) detection problem. Inspired by whitening techniques, we apply a linear transformation to decorrelate the model’s hidden activations and standardize them to zero mean and unit variance, yielding
near-identity covariance matrix. In this transformed space, we use the Euclidean norm as a compliance score to detect policy violations. The method requires only the policy text and a small number of illustrative samples, which makes it lightweight and easily deployable. On a challenging policy benchmark, our approach achieves state-of-the-art results, surpassing both existing guardrails and fine-tuned reasoning models. This work provides organizations with a practical and statistically grounded framework for policy-aware oversight of
LLMs, advancing the broader goal of trustworthy, deployable AI governance. Code implementation will be released upon
publication.

Downloads

Next from AAAI 2026

Intrinsic Self-Correction in LLMs: Towards Explainable Prompting via Mechanistic Interpretability

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAAI 2026

Intrinsic Self-Correction in LLMs: Towards Explainable Prompting via Mechanistic Interpretability

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads