Singapore

We propose Poincaré Gradient Descent (PGD), a first-order optimization method on the Poincaré ball. PGD replaces the exponential map with a projection-based retraction that is first-order equivalent while preserving geodesic structure, thereby reducing per-iteration computational cost. We establish convergence guarantees for smooth geodesically convex and strongly convex objectives, showing that PGD matches the iteration complexity of Riemannian Gradient Descent (RGD). A Möbius isometric initialization and step-size equivalence analysis further ensure stable and efficient updates. Numerical experiments confirm that PGD achieves the same theoretical convergence rate as RGD but runs significantly faster in practice, offering a simple and effective framework for scalable optimization in hyperbolic space.

AAAI 2026

Poincaré Gradient Descent: A Projection-Based Riemannian Method on the Poincaré Ball

geodesic convexity

poincaré disk

riemannian optimization

hyperbolic learning

hyperbolic geometry

gradient descent

The purpose of the AAAI conference is to promote research in artificial intelligence (AI) and scientific exchange among AI researchers, practitioners, scientists, and engineers in affiliated disciplines. 

AAAI 2022

workshop paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.<br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

Spiking neural networks (SNNs) have been proposed as an (energy-)efficient alternative to conventional artificial neural networks. However, the aspired benefits have not yet been realized in practice. To gain a better understanding of why this gap persists, we theoretically study both discrete-time and continuous-time models of leaky integrate-and-fire neurons.
In the discrete-time model, which is a widely used framework due to its amenability to conventional deep learning software and hardware approaches, we analyze the impact of explicit recurrent connections on the network size required to approximate continuously differentiable functions. We contrast this view by investigating the computational efficiency of digital systems that simulate spike-based computations in the continuous-time model. It turns out that even in well-behaved settings, the computational complexity of this task may grow super-polynomially in the prescribed accuracy.
Thereby, we exemplarily highlight the intricacies of realizing potential strengths in the biological context, namely recurrent connections and computational efficiency, of spike-based computations on digital systems.

How to realize efficient Spiking Neural Networks?

Bayesian neural networks (BNNs) require scalable sampling algorithms to approximate posterior distributions over parameters. Existing stochastic gradient Markov Chain Monte Carlo (SGMCMC) methods are highly sensitive to the choice of stepsize and adaptive variants such as pSGLD typically fail to sample the correct invariant measure without addition of a costly divergence correction term. In this work, we build on the recently proposed `SamAdams' framework for timestep adaptation (Leimkuhler, Lohmann, and Whalley 2025), introducing an adaptive scheme: SA-SGLD, which employs time rescaling to modulate the stepsize according to a monitored quantity (typically the local gradient norm). SA-SGLD can automatically shrink stepsizes in regions of high curvature and expand them in flatter regions, improving both stability and mixing without introducing bias. We show that our method can achieve more accurate posterior sampling than SGLD on high-curvature 2D toy examples and in image classification with BNNs using sharp priors.

Adaptive Stepsizing for Stochastic Gradient Langevin Dynamics in Bayesian Neural Networks

We study the fundamental expressivity limits of transformer models. We formalize the notion of accessible sequences—those that a transformer can produce for some prompt—and characterize how accessibility depends on prompt length and model precision. By partitioning the embedding space via the decoder readout into next-token argmax regions and extending transformers to a mean-field map on probability measures, we derive theoretical upper bounds on the number and length of accessible output sequences. We prove that (i) the maximal length of accessible sequences grows linearly with the prompt length, and (ii) beyond a critical threshold, the proportion of reachable sequences decays exponentially with sequence length. These bounds hold even with unbounded context and computation time, linking the expressivity limits of transformers to the geometry of their embedding space and the finiteness of their representational precision. Experiments using a “cramming” procedure confirm both the linear scaling and the post-threshold exponential decay.

The Expressivity Limits of Transformers

Combinatorial and topological structures, such as graphs, simplicial complexes, and cell complexes, form the foundation of geometric and topological deep learning (GDL and TDL) architectures. These models aggregate signals over such domains, integrate local features, and generate representations for diverse real-world applications. However, the distribution and diffusion behavior of GDL and TDL features during training remains an open and underexplored problem. Motivated by this gap, we introduce a cellular sheaf theoretic framework for modeling and analyzing the local consistency and harmonicity of node features and edge weights in graph-based architectures. By tracking local feature alignments and agreements through sheaf structures, the framework offers a topological perspective on feature diffusion and aggregation. Furthermore, a multiscale extension inspired by topological data analysis (TDA) is proposed to capture hierarchical feature interactions in graph models. This approach enables a joint characterization of GDL and TDL architectures based on their underlying geometric and topological structures and the learned signals defined on them, providing insights for future studies on conventional tasks such as node classification, substructure detection, and community detection.

A Sheaf-Theoretic and Topological Perspective on Complex Network Modeling and Attention Mechanisms in Graph Neural Models

Diffusion probabilistic models have been successfully used to generate high resolution data from noise. However, limited interpretability and computationally expensive implementation renders most existing diffusion models unsuitable for applications where the goal is to achieve theoretically accurate results with a fair trade-off on the quality of generated samples. In this work, we propose an adaptive random feature architecture for training a diffusion model that can generate samples from the input distribution using limited parameters, and theoretically quantify the generalization bounds. Our results demonstrate the possibility of building interpretable diffusion model architectures. We validate the obtained theory using numerical results, which perform better (or comparably) to an U-Net architecture having same number of trainable parameters.

Generalization Bound for Diffusion Models using a Adaptive Random Features (DARF)

In this paper, we propose SubLoRA, a rank determination method for Low-Rank Adaptation (LoRA) based on submodular function maximization. In contrast to prior approaches, such as AdaLoRA, that rely on first-order (linearized) approximations of the loss function, SubLoRA utilizes second-order information to capture the potentially complex loss landscape by incorporating the Hessian matrix. We show that the linearization becomes inaccurate and ill-conditioned when the LoRA parameters have been well optimized, motivating the need for a more reliable and nuanced second-order formulation. To this end, we reformulate the rank determination problem as a combinatorial optimization problem with a quadratic objective. However, solving this problem exactly is NP-hard in general. To overcome the computational challenge, we introduce a submodular function maximization framework and devise a greedy algorithm with approximation guarantees. We derive a sufficient and necessary condition under which the rank-determination objective becomes submodular, and construct a closed-form projection of the Hessian matrix that satisfies this condition while maintaining computational efficiency. Our method combines solid theoretical foundations, second-order accuracy, and practical computational efficiency. We further extend SubLoRA to a joint optimization setting, alternating between LoRA parameter updates and rank determination under a rank budget constraint. Extensive experiments on fine-tuning physics-informed neural networks (PINNs) for solving partial differential equations (PDEs) demonstrate the effectiveness of our approach. Results show that SubLoRA outperforms existing methods in both rank determination and joint training performance.

Automatic Rank Determination for Low-Rank Adaptation via Submodular Function Maximization

Mixture-of-Experts (MoE) layers have achieved notable success across various deep learning applications. However, the impact of the number of experts on MoE performance across different task settings remains poorly understood. In this work, we investigate the impact of expert quantity within the MoE architecture composed of multilayer perceptron (MLP) experts. Concretely, we develop a formal MoE model with MLP experts, derive a range of expert counts that optimizes performance and load balance, and validate it on synthetic data. By systematically varying the number of experts, we demonstrate that balancing specialization and effective expert routing is key to maximizing performance.

On the Impact of Expert Count in Mixture of Experts

While existing formal verification tools are increasingly effective at verifying local robustness properties, more complex ones, such as those involving multiple neural networks, remain beyond their capabilities: these properties cannot currently be expressed in their specification languages, nor can they be directly verified. We present CAISAR, an open-source platform for specifying and verifying properties of machine learning models, with particular focus on neural networks and support vector machines. CAISAR provides a high-level language for specifying complex properties and integrates several state-of-the-art verifiers for their automatic verification. Through concrete use cases, we show how CAISAR leverages automated graph-editing techniques to translates high-level specifications into queries for the supported verifiers, bridging the embedding gap between user specifications and the corresponding ones that are actually verified.

The CAISAR Platform: Extending the Reach of Machine Learning Specification and Verification

SHapley Additive exPlanations (SHAP) is a key tool for interpreting decision tree ensembles by assigning contribution values to features. The most accurate variant, Background SHAP, uses a background dataset to estimate feature distributions. We introduce WOODELF, a SHAP algorithm that unifies decision trees, game theory, and Boolean logic within a single framework. For each instance, WOODELF constructs a pseudo-Boolean formula that captures their feature values, the structure of the decision tree ensemble, and the entire background dataset. It then leverages this representation to compute Background SHAP in linear time. WOODELF can also compute Shapley interaction values, Banzhaf values, and Banzhaf interaction values. WOODELF runs efficiently on both CPU and GPU. We demonstrate these advantages empirically on two large, commonly used datasets, where WOODELF achieves at least an order-of-magnitude speedup over state-of-the-art methods.

From Decision Trees to Boolean Logic: A Fast and Unified SHAP Algorithm

Case-based reasoning networks are models that make predictions based on similarity between the input and prototypical parts of training samples. Such models explain each decision by pointing to the prototypes that contributed the most to the final outcome. While promising, such explanations are sometimes misleading, which hampers their usefulness in safety-critical contexts. Several instances may lead to different predictions and yet have the same explanation. Drawing inspiration from formal eXplainable AI, we propose Abductive Latent Explanations (ALEs), a formalism to express sufficient conditions on the latent representation of the instance that imply the prediction. Our approach combines the inherent interpretability of case-based reasoning models and the guarantees provided by formal XAI. We propose a solver-free algorithm for generating ALEs based on three distinct paradigms, compare them, and present the feasibility of our approach on diverse datasets for image classification.

Downloads

Next from AAAI 2026

How to realize efficient Spiking Neural Networks?

Similar lecture

Efficient Riemannian Meta-Optimization by Implicit Differentiation

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES