Singapore

Current vision-language models suffer from overconfident predictions and cross-modal hallucinations, lacking principled mechanisms for uncertainty quantification. We introduce a novel architecture that applies the Free Energy Principle from computational neuroscience to multimodal transformers, enabling reliable uncertainty estimation through hierarchical predictive processing. Our approach implements precision-weighted cross-modal prediction, where visual and linguistic representations generate predictions about each other, and prediction errors are weighted by learned precision matrices that capture cross-modal consistency. By minimizing variational free energy across modalities, our model naturally quantifies uncertainty while maintaining task performance. Experimental results demonstrate substantial improvements over standard uncertainty quantification methods, achieving 51.7% better calibration than Monte Carlo Dropout baselines on synthetic evaluation data and 48.6% improvement on the VQA v2 dataset. This work establishes the first principled bridge between the brain&#39;s Bayesian inference mechanisms and practical multimodal AI uncertainty quantification, demonstrating that biologically-inspired architectures can significantly enhance model reliability.

AAAI 2026

Hierarchical Predictive Processing for Uncertainty-Aware Multimodal Transformers

(computational) cognitive architectures

calibration & uncertainty quantification

multimodal learning

Current vision-language models suffer from overconfident predictions and cross-modal hallucinations, lacking principled mechanisms for uncertainty quantification. We introduce a novel architecture that applies the Free Energy Principle from computational neuroscience to multimodal transformers, enabling reliable uncertainty estimation through hierarchical predictive processing. Our approach implements precision-weighted cross-modal prediction, where visual and linguistic representations generate predictions about each other, and prediction errors are weighted by learned precision matrices that capture cross-modal consistency. By minimizing variational free energy across modalities, our model naturally quantifies uncertainty while maintaining task performance. Experimental results demonstrate substantial improvements over standard uncertainty quantification methods, achieving 51.7% better calibration than Monte Carlo Dropout baselines on synthetic evaluation data and 48.6% improvement on the VQA v2 dataset. This work establishes the first principled bridge between the brain's Bayesian inference mechanisms and practical multimodal AI uncertainty quantification, demonstrating that biologically-inspired architectures can significantly enhance model reliability.

workshop paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.<br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

Deep neural networks (DNNs) often “cheat” by relying on shortcut objects (e.g., food⇒kitchen) rather than holistic spatial layout, undermining out-of-distribution (OOD) robustness. We address this issue with Play the (Mis)Match, a diagnostic dataset and brain-aligned fine-tuning framework. Leveraging fMRI recordings from the Natural Scenes Dataset (four participants; bedroom, bathroom, living room, kitchen), we curate MATCH images in which shortcut cues co-occur as usual and MISMATCH images from which those cues are removed. ImageNet-initialised CNN and Transformer backbones are finetuned with an MSE alignment loss that steers their intermediate features toward voxel patterns known to be less sensitive to shortcut cues. Our results found that for ResNet, this procedure narrows the Match–Mismatch accuracy gap by 24 % and redirects Grad-CAM attention from individual objects to holistic scene structure, especially activity from scene-selective cortex (PPA, RSC, OPA), all without explicit shortcut annotations. Our study provides a proof-of-concept indication that human-brain constraints may help steer DNNs toward more semantically grounded and less shortcut-dependent scene representations.

Play the (Mis)Match: Using fMRI-Aligned Feature Fine-Tuning to Reveal Shortcut Bias in Deep Neural Networks

Oscillatory dynamics are an ubiquitous feature of biological neural networks, yet their computational role has remained debated. Recent work has advanced the hypothesis that oscillations are not epiphenomenal but constitute a fundamental mechanism for information processing. Here, we synthesize three lines of research that address this question in conceptual, computational, and physical domains. First, theoretical considerations show that oscillations support functions based on synchrony, resonance and wave interference that are not accessible to non-oscillatory models. Second, the Harmonic Oscillator Recurrent Networks (HORN) model demonstrates that oscillatory dynamics confer advantages in learning speed, robustness, and parameter efficiency compared to non-oscillating recurrent architectures in the field of machine learning. Third, an analog-electronic implementation of the HORN model demonstrates the feasibility of exploiting transient oscillatory dynamics for computation in physical systems. This approach establishes that across neural, artificial, and physical systems, oscillatory transients provide a powerful resource for computation. Recent evidence from biological, photonic, mechanical, and fluid systems shows that wave-based computation extends far beyond neural substrates. Moreover, simulation studies could confirm the functional relevance of wave-based dynamics in artificial neural networks. Together, these results show that transient oscillatory and wave-based dynamics can serve as a unifying substrate for analog computation across a large variety of domains, offering new directions for robust and energy-efficient, biology-inspired information processing systems.

Oscillatory Dynamics as an Universal Substrate for Computation: From Neural Circuits to Artificial Intelligence

Neural decoding from electroencephalography (EEG) remains fundamentally limited by poor generalization to unseen subjects, driven by high inter-subject variability and the lack of large-scale datasets to model it effectively. Existing methods often rely on synthetic subject generation or simplistic data augmentation, but these strategies fail to scale or generalize reliably. We introduce MultiDiffNet, a diffusion-based framework that bypasses generative augmentation entirely by learning a compact latent space optimized for multiple objectives. We decode directly from this space and achieve state-of-the-art generalization across various neural decoding tasks using subject and session disjoint evaluation. We also curate and release a unified benchmark suite spanning four EEG decoding tasks of increasing complexity (SSVEP, Motor Imagery, P300, and Imagined Speech) and an evaluation protocol that addresses inconsistent split practices in prior EEG research. Finally, we develop a statistical reporting framework tailored for low-trial EEG settings. Our work provides a reproducible and open-source foundation for subject-agnostic EEG decoding in real-world BCI systems.

MultiDiffNet: A Multi-Objective Diffusion Framework for Generalizable Brain Decoding

We present a methods-first framework that turns high-dimensional population neural recordings into directly comparable, mechanistic fingerprints at the level of individual subjects. Our pipeline (i) constructs population-universal, shared latent coordinates that align heterogeneous subjects into a common representational space; (ii) fits pairwise maximum-entropy (Ising) models on binarised latent trajectories with rigorous convergence- and uncertainty-diagnostics; (iii) performs energy-landscape analysis (ELA) to obtain interpretable minima, barriers and kinetic descriptors; and (iv) introduces a new, variance-balanced multi-observable phase-diagram analysis (PDA) that places many subjects – including systematically heterogeneous sub-groups – onto a shared Sherrington-Kirkpatrick (SK) reference surface with uncertainty, making cross-subject comparisons direct and faithful. In a cohort of rodent whole-brain imaging time series (sensitive third-party data), our placement costs are typically 10^-6 to 10^-5, with tight bootstrap confidence regions and consistent ordering across pooled and subgroup references; estimated SK parameters fall in sigma ≈ 0.185 - 0.320, mu ≈ -0.013 to +0.031. The result is a compact, uncertainty-aware subject “fingerprint” comprising ELA and kinetic descriptors together with the subject’s location on the phase diagram. We position this as work in progress focusing on methodological reliability and comparability; external replications are planned on public datasets.

Shared Latent Coordinates and Multi-Observable Phase-Diagram Placement Yield Directly Comparable Mechanistic Fingerprints of Whole-Brain Dynamics

The fusion of Spiking Neural Networks (SNNs) and Transformers holds significant promise for energy-efficient on-device Al, particularly for Synthetic Aperture Radar (SAR) imagery in resource-constrained environments like satellites. However, existing models are not optimized for SAR's unique statistical properties, such as speckle noise, and suffer from computational inefficiency due to static inference graphs. To address this, we propose SAR-Spikingformer, a lightweight SNN Transformer. Our model introduces two key contributions: 1) a domain-specific data augmentation strategy to enhance robustness against SAR-specific noise and geometric variations, and 2) a dynamic 'Block-Halting' mechanism that adapts computational depth based on input difficulty, minimizing inference costs. Our results on the MSTAR and EuroSAT datasets show significant energy reductions of up to 27.76% and 46.53%, respectively, without compromising classification accuracy.

Adaptive Spiking Transformer for SAR Image Classification

Spiking Neural Networks (SNNs) are promising for energy-efficient computing on next generation neuromorphic hardware (Nunes et al. 2022). However, scaling them to large Transformer architectures introduces challenges such as training instability and high computational costs associated with multiple timesteps (Hu et al. 2024), which can offset their efficiency benefits. To address these issues, this paper proposes Spiking Swin-B, a spiking version of the large-scale Swin-B Transformer, integrated with a Reinforcement Learning (RL)-based meta-controller for dynamic execution. To ensure stable training and prevent performance degradation, we preserve the original attention mechanism of Swin-B, enabling a robust ANN-to-SNN conversion. Furthermore, the RL meta-controller learns to automatically optimize the accuracy-energy trade-off by dynamically adjusting timesteps and early-exit stages based on input complexity. 
Experimental results on CIFAR-100 show that the proposed Spiking Swin-B reduces computational energy by approximately 40% compared to its ANN counterpart, with only a marginal accuracy drop of about 1%. This work demonstrates the feasibility of applying learnable, dynamic control to large-scale spiking Transformers, paving a practical path toward adaptive and energy-aware SNN inference.

Reinforcement-Learned Dynamic Execution for Spiking Swin-B

Many recent zero or few-shot approaches in Information Extraction rely on large-scale annotated corpora for pretraining. However, while manually annotated corpora are scarce, the current methods for performing data augmentation or data generation remain limited, especially for complex tasks such as event extraction. To address this challenge, we introduce a simple yet efficient annotation method for real-world data that leverages the capabilities of Large Language Models (LLMs) to both generate annotations and evaluate their quality. Our method defines confidence scores to identify passages of interest and to select the best generated annotations. Based on this approach, we propose Omnivent, a large-scale and general-purpose event dataset designed for both event detection and event argument extraction. To the best of our knowledge, our dataset offers the largest coverage of event types to date, comprising 38,859 distinct event types and 9,981 argument roles. To demonstrate the effectiveness of our dataset, we also introduce GLEE, a bidirectional transformer model to perform zero-shot event detection and argument extraction. We conducted evaluations across 9 benchmarks and 7 domains showing that pretraining on Omnivent yields better zero-shot performance than pretraining on human-annotated data. Notably, despite being 25x smaller, GLEE outperforms leading LLM-based approaches in zero-shot event detection and achieves up to 50\% of the performance of fully supervised models in event argument extraction.

Annotating Pretraining Corpora in Information Extraction: An Unsupervised Confidence-Guided Approach

Multimodal document retrieval systems have shown strong progress in aligning visual and textual content for semantic search. However, most existing approaches remain heavily English-centric, limiting their effectiveness in multilingual contexts. In this work, we present M3DR (Multilingual Multimodal Document Retrieval) a framework designed to bridge this gap across languages, enabling applicability across diverse linguistic and cultural contexts. M3DR leverages synthetic multilingual document data and generalizes across different vision-language architectures and model sizes, enabling robust cross-lingual and cross-modal alignment. Using contrastive training, our model learns unified representations for text and document images that transfer effectively across languages. We validate this capability on 22 typologically diverse languages, demonstrating consistent performance and adaptability across linguistic and script variations. We further introduce a comprehensive benchmark that captures real-world multilingual scenarios, evaluating models under monolingual, multilingual, and mixed-language settings. Extensive experiments show that M3DR achieves substantial improvements over existing baselines, with over 150\% relative gains on cross-lingual retrieval tasks.

M3DR - Towards Universal Multilingual Multimodal Document Retrieval

Retrieval-Augmented Generation (RAG) has facilitated the construction of domain-specific question–answering (Q&A) systems by integrating information retrieval (IR) mechanisms with large language models (LLMs). However, real-world RAG systems frequently suffer from domain drift In automotive voice assistants, novice users unfamiliar with technical terminology often paraphrase such terms, which prevents dense retrievers from bridging the semantic gap and consequently causes retrieval failures. The rapid proliferation of LLMs has expanded their user base, increasing the demand for non-expert users to access domain-specific knowledge in specialized fields such as healthcare and law, thereby underscoring the significance of this issue. This study investigates a practical approach to addressing the challenge of understanding paraphrastic expressions of technical terminology, using a dataset of over 10,000 Q&A pairs derived from vehicle manuals, and proposes continual pre-training of embedding models as an effective solution. Experimental results demonstrate that continual pre-training effectively imbues embedding models with domain knowledge, enabling them to associate paraphrased expressions with their corresponding technical terms. Although continual pre-training has been extensively studied for language models, its application to retrieval embedding models within RAG systems remains largely underexplored. Our findings offer insights into designing retrieval systems that achieve both domain fidelity and linguistic generalization—two essential prerequisites for scaling RAG to real-world applications.

Continual Pre-training of Dense Retrievers for Bridging Paraphrases and Technical Terms in Domain-Specific Retrieval-Augmented Generation

Locating and retrieving objects from scene-level point clouds is a challenging problem with broad applications in robotics and augmented reality. This task is commonly formulated as open-vocabulary 3D instance segmentation. Although recent methods demonstrate strong performance, they depend heavily on SAM and CLIP to generate and classify 3D instance masks from images accompanying the point cloud, leading to substantial computational overhead and slow processing that limit their deployment in real-world settings. Open-YOLO 3D alleviates this issue by using a real-time 2D detector to classify class-agnostic masks produced directly from the point cloud by a pretrained 3D segmenter, eliminating the need for SAM and CLIP and significantly reducing inference time. However, Open-YOLO 3D often fails to generalize to object categories that appear infrequently in the 3D training data. In this paper, we propose a method that generates 3D instance masks for novel objects from RGB images guided by a 2D open-vocabulary detector. Our approach inherits the 2D detector’s ability to recognize novel objects while maintaining efficient classification, enabling fast and accurate retrieval of rare instances from open-ended text queries.

Downloads

Next from AAAI 2026

Play the (Mis)Match: Using fMRI-Aligned Feature Fine-Tuning to Reveal Shortcut Bias in Deep Neural Networks

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES