Singapore

We present CRM (Multi-Agent Collaborative Reward Model), a framework that replaces a single black-box reward model with a coordinated team of specialist evaluators to improve robustness and interpretability in RLHF. Conventional reward models struggle to jointly optimize multiple, sometimes conflicting, preference dimensions (e.g., factuality, helpfulness, safety) and offer limited transparency into why a score is assigned.

AAAI 2026

Multi-Agent Collaborative Reward Design for Enhancing Reasoning in Reinforcement Learning

technical paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.<br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

Reinforcement Learning from Human Feedback (RLHF) has become the dominant paradigm for aligning large language models with human preferences. However, when preference data is aggregated from diverse populations, it remains unclear whether the resulting aligned models serve all demographic groups equitably, or support long-term behavior change and mental health and wellness needs in a balanced way. We investigate this question through a controlled experiment using Direct Preference Optimization (DPO), training on preferences collected from our novel synthetic dataset, the 10th Village, comprising 5,000 synthetic villagers with demographics and personality traits modeled on U.S. Census data and validated psychological instruments. Each villager provided preferences across everyday stressors in financial/employment and social/relationship domains. We introduce an alignment fairness evaluation framework that treats RLHF as a behavior-aware recommendation problem, measuring how well the aligned model matches individual villager preferences compared to the base model and analyzing disparities across demographic subgroups. Our results reveal two critical sources of inequality: First, social and relationship problems receive substantially less benefit from alignment than financial concerns (p < .001), despite already generating higher baseline dissatisfaction. Second, more educated villagers gain disproportionate benefit from alignment (p < .001), particularly for social problems, creating a compounding advantage. These findings suggest that standard RLHF practices may systematically disadvantage certain problem domains and demographic groups, highlighting the need for fairness-aware approaches to preference aggregation and model alignment. Our contributions include both the 10th Village dataset and a reusable evaluation protocol for controlled, behavior-aware fairness research, as well as empirical evidence of disparate impact in preference-based alignment to guide the design of more equitable, wellness-oriented RLHF systems.

Not All Stress Is Treated Equal: Fairness Gaps in AI Support for Everyday Problems

As AI systems increasingly mediate human interactions, most alignment frameworks erroneously assume static human preferences. We introduce Socially-Aware Continual Learning (SCL), a framework that maintains ethical alignment with dynamically evolving norms through norm embeddings and Social Elastic Weight Consolidation (SEWC)—a novel algorithm that adapts regularization strength based on measured norm drift. Extensive experiments on longitudinal datasets demonstrate SCL’s superior balance between Alignment Stability (F1 = 0.84) and Normative Plasticity (F1 = 0.87), significantly outperforming state-of-the-art baselines (p < 0.001). Our contributions include: (1) validated drift detection metrics (BDI, DCS) achieving 0.89 F1-score, (2) human evaluations showing 82% trust recovery after norm shifts, (3) formalized fairness-utility trade-offs with 4% versus 20% disparity for baselines, and (4) societal-scale simulation showing 36% polarization reduction. SCL provides both theoretical stability guarantees and practical tools for developing socially responsive AI.

Socially-Aware Continual Learning: Modeling Dynamic Alignment with Evolving Human Norms

Cognitive behavioural therapy (CBT) and exposure therapy (ET) are among the most effective interventions for anxiety and related mental health conditions, yet their traditional delivery remains limited due to issues of scalability, personalisation, evaluation accuracy and constructive user engagement. This position paper proposes a framework for affect-intelligent virtual reality (AIVR), an AI-driven and ethically grounded approach that integrates immersive technology, physiological sensing and automated therapy adaptation to address the core limitations of conventional CBT. By leveraging real-time affective data and behaviour-aware modelling, AIVR systems can dynamically tailor exposure difficulty, provide personalised feedback and support, and offer interpretable feedback to both users and clinicians. The framework outlines how adaptive AI can (1) extend therapy beyond traditionally inefficient conversation-based formats, (2) streamline individualisation through continuous learning, (3) ensure safe and transparent evaluation, and (4) promote long-term behavioural resilience. Emphasising ethical design, transparency and human oversight, AIVR reimagines AI not as a replacement for therapists, but as a collaborative partner in mental health care. This paper calls for interdisciplinary collaboration across AI, human-computer interaction, psychology and ethics to realise trustworthy, behaviour-aware VR interventions that align therapeutic innovation with human values.

Affect-Intelligent Virtual Reality (AIVR): Overcoming the Core Challenges of Cognitive Behavioural Therapy

Automated planning is a long-standing challenge in AI research, which requires domain models that define the state variables and actions in terms of preconditions and effects. However, the handcrafting of such domain models is widely recognized to be a time-consuming and error-prone process. Therefore, theories and algorithms for automatically learning domain models for planning have been developed over the years. The goal of this tutorial is to provide participants with an overview of state-of-the-art approaches for learning planning domain models under different assumptions, and to enable participants to utilize publicly available frameworks and tools for domain model learning.

The tutorial first introduces the basic concepts for planning domain learning, and situates the problem within the broader context of AI research, relating it to other fields such as Reinforcement Learning (RL) and Knowledge Representation (KR). Then, the tutorial provides an overview of state-of-the-art approaches for learning domain models assuming a given symbolic state representation. The focus is on approaches that learn action preconditions and effects from an input set of trajectories generated by executing actions and observing their effects. The discussed approaches are compared w.r.t. their pros and cons, and their theoretical properties. Subsequently, the tutorial discusses domain model learning approaches that do not assume a symbolic state representation. Such approaches additionally learn to encode state subsymbolic observations (e.g., RGB images) into symbolic representations suitable for planning. A hands-on session guides the participants through the usage of open-source frameworks and tools for domain model learning and evaluation. The tools considered are useful and readily accessible resources for practitioners and researchers who aim to apply domain model learning techniques in possibly different research areas. The tutorial ends by describing methods for online domain model learning, and integrations with RL agents and LLM-based agents.

Participants are expected to have only a basic understanding of automated planning and symbolic domain representation, i.e., that a symbolic state is defined by a set of state variables, and a symbolic action is specified by a set of preconditions and effects. Further information about the tutorial is available at https://domain learning.github.io.

Domain Model Learning for Automated Planning: Introduction & Domain Learning Basics 

In this tutorial, we chart a practical path from raw capability to trustworthy reasoning with foundation models. We begin by motivating why trustworthy reasoning is essential: when models bluff multiplications or invent drug interactions, their value collapses and risks increase. We adopt four pillars of trustworthiness, i.e., capability, safety, robustness, and explainability, as the organizing framework for the entire session.

In Part I, we trace the evolution from early language models to today’s foundation models that produce extended chains of thought and act in the world. Through concrete case studies, we dissect jailbreaks, hallucinations, and brittle logic, and we connect these failure modes to regulatory pressure such as the EU AI Act. The takeaway is clear: we must design for trustworthy reasoning from the outset, especially in high-stakes domains such as clinical or financial decision-making.

In Part II, we move from leaderboards to a science of measurement. We show how to build reliable, valid evaluations using psychometric tools, including item response theory, amortized evaluation, and predictability analysis. We implement three open-source pipelines hands-on: TruthfulQA for hallucination detection, HellaSwag for robustness testing, and MATH with formal-verification hooks in Lean4. Along the way, we demonstrate red-teaming stress tests and reasoning-trace metrics that surface subtle errors leaderboards miss, and we practice calibration, dataset curation, and transparent reporting for honest progress tracking.

In Part III, we deliver a compact methodology for trustworthy machine reasoning. We cover training-free prompting methods (chain-of-thought, retrieval-augmented generation, constrained decoding), post-training algorithms (supervised fine-tuning, RLHF, verifiable rewards, self-reward), and test-time techniques (self-consistency, reflection, tree search, tool-augmented verification). We introduce guardrails—safe sampling and semantic filters—that reduce risk without crippling capability. For each technique, we map effects to the four pillars, highlight trade-offs and failure signatures, and summarize when to combine methods for maximum leverage.

In Part IV, we turn to deployment. We walk through real-world agents and workflows, e.g., Lean4-based code verification assistants and bioinformatics pipelines proposing candidate compounds. We share step-by-step recipes, failure checklists, and diagnostics so participants can preserve trust while shipping. We also outline governance artifacts—risk registers, evaluation cards, and incident playbooks—that align technical practice with policy expectations.

We emphasize open, reproducible assets and decision rubrics that translate research into dependable products. Our goal is simple: help you move from compelling demos to trustworthy systems that earn and deserve user trust.

All materials will be available at: https://trustworthy-machine-reasoning.github.io/

Trustworthy Machine Reasoning with Foundation Models

Neuroevolution, or optimization of neural networks through evolutionary computation, is a method for constructing intelligent agents through population-based search. It is particularly useful in partially observable domains with sparse and multiobjective reinforcement; compared to other policy search techniques, its power comes from extensive exploration that allows it to find effective, often surprising solutions.   Prime application domains include robotic control, game-playing agents, and decision-making. More recently, it has also been extended to optimizing deep-learning architectures, understanding how biological intelligence evolved, and optimizing neural networks for hardware implementation.  Several synergies have also emerged with reinforcement learning and large language models. This tutorial introduces participants to the basics of neuroevolution, progresses to several advanced topics that make neuroevolution effective and general, reviews example application areas, and proposes further research questions. An optional hands-on exercise makes these concepts concrete and allows the participants to take advantage of neuroevolution immediately.  For more details, see https://www.cs.utexas.edu/~risto/talks/aaai26-tutorial/.

Evolution of Neural Networks

TH08: Algorithms and Systems for Efficient Inference in Generative AI
State-of-the-art Generative AI (GenAI) models now span diverse modalities including language, vision, and audio, and are being deployed across an increasingly wide range of applications. However, their growing size and complexity pose significant challenges for efficient inference, particularly in real-time or resource-constrained settings. This tutorial will introduce participants to a range of techniques that enable high-performance, scalable inference without compromising model accuracy. It will cover practical tools, open-source serving frameworks, and deployment considerations across different hardware platforms, and the underlying principles will be broadly applicable to emerging model architectures. These methods will be illustrated through concrete scenarios, such as low-latency serving of large language models (LLMs), on-device deployment of quantized models, and performance tuning for cost-sensitive applications.

The tutorial is organized into three sessions that collectively build a comprehensive understanding of efficient inference for GenAI models. Session 1 provides an overview of the diverse architectural classes of GenAI models, ranging from decoder-only LLMs to multimodal models. This is followed by an introduction to the multi-layered inference stack, spanning high-level ML frameworks such as PyTorch to low-level hardware instructions — an understanding of which is essential for systematically uncovering performance bottlenecks. This will be accompanied with a demonstration of profiling techniques to analyze these bottlenecks. Session 2 dives into algorithmic, modeling, and systems-level optimizations, covering techniques like efficient attention variants, quantization, speculative decoding, and kernel fusion — while examining how each of these techniques alleviates the bottlenecks discussed in Session 1. Session 3 concludes the tutorial by focusing on practical tools and frameworks that implement many of the previously discussed optimization techniques with accompanying demos. It will highlight vLLM for high-throughput LLM serving, and TensorRT for hardware-optimized compilation across modalities. Together, these sessions will equip participants with the technical depth and tooling needed to deploy GenAI models efficiently in real-world systems.

This tutorial is designed for researchers and practitioners in AI who are interested in understanding and improving the efficiency of GenAI models. It will be particularly valuable for those with expertise in a specific layer of the inference stack, such as modeling, quantization, or infrastructure, and are seeking to develop a holistic perspective connecting algorithmic techniques with system-level and hardware-aware optimizations. Attendees are expected to have basic familiarity with deep learning frameworks (e.g., PyTorch). Prior experience with LLM architectures and deployment is beneficial, but not required.

Algorithms and Systems for Efficient Inference in Generative AI

In recent years, multi-modal time series analysis has gained increasing attention for integrating insights from diverse data sources, enhancing both predictive performance and explanations across various domains. This growing interest is driven by the rapid production of heterogeneous data in the real world, where temporal signals often come not only in numerical form but are also accompanied by complementary modalities such as text, images, or structured metadata. By effectively leveraging information from different modalities, researchers can discover richer patterns and improve model performance in the application. We plan to present a comprehensive half-day tutorial at AAAI 2026, tailored for researchers and practitioners interested in multi-modal time series analysis. This tutorial provides insights into the theoretical and practical aspects of multi-modal time series, covering data characteristics and cross-modality modeling strategies for various downstream tasks. Attendees will also learn best practices for applying multi-modal time series analysis to real-world domains such as finance, healthcare, and transportation. Through these examples, participants will gain a clearer understanding of how to move from theoretical modeling to impactful deployment. The tutorial offers a comprehensive and in-depth understanding, practical skill development, and networking opportunities, connecting theory with real-world applications.

Multi-modal Time Series Analysis: Methods, Datasets, and Applications

Deterministic search and planning typically operate on graphs where each edge has a given cost, aiming to find a path of minimal cost. Heuristic search has developed tools for solving this path-finding problem, including the A* algorithm. However, the objective of search and planning is often much more complex in real life, for example, because one needs to trade-off between different costs. Multi-objective search and planning operate on graphs where each edge has two or more given costs, aiming to find paths that trade-off between the different costs in the form of a Pareto frontier of paths.

For example, transporting hazardous material requires trading off different costs for each street, such as its length and the number of residents that would be exposed to the hazardous material in case of an accident. Other examples include route planning for vehicles and robots (which involves trading off energy consumption and travel time), planning power transmission lines (which involves trading off power-generation cost and power loss), and scheduling satellites and routing packets in computer networks (which involves trading off profit and fairness).

This tutorial will provide an overview of the multi-objective search problem and summarize recent progress in this fast-moving research area. We will cover theoretical foundations, practical algorithms, and challenges that commonly arise in practice in a way that is accessible to all AI researchers and students. Our target audience is anyone interested in search and planning who wants to learn about this fascinating emerging field. More information on the tutorial and a detailed schedule can be found at https://sites.google.com/usc.edu/aaai24-mos-tutorial/home

Recent Advances in Multi-Objective Search

This tutorial provides a comprehensive and unified overview of the rapidly evolving field of LLM-based Multi-Agent Systems (LaMAS). As LLMs become more capable as autonomous agents, the next frontier is orchestrating their collective intelligence to solve complex problems. This tutorial systematically bridges the foundational principles of multi-agent reinforcement learning (MARL) and game theory with the practical design and engineering of modern LaMAS.

The tutorial begins with the theoretical underpinnings of multi-agent interaction before moving to the core components of a modern LLM agent, including reasoning, planning, and tool use. The central part delves into LaMAS architectures, covering communication protocols, coordination strategies, and collaborative optimization. The tutorial concludes by exploring the frontiers of the field, discussing evaluation methodologies, open-source frameworks, and real-world applications. Attendees will gain a robust theoretical understanding, a map of the current research landscape, and the practical knowledge needed to build and evaluate their own LaMAS.

Premium content

Next from AAAI 2026

Not All Stress Is Treated Equal: Fairness Gaps in AI Support for Everyday Problems

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES