United States

This paper explores the impact of relational state abstraction on sample efficiency and performance in collaborative Multi-Agent Reinforcement Learning. The proposed abstraction is based on spatial relationships in environments where direct communication between agents is not allowed, leveraging the ubiquity of spatial reasoning in real-world multi-agent scenarios. We introduce MARC (Multi-Agent Relational Critic), a simple yet effective critic architecture incorporating spatial relational inductive biases by transforming the state into a spatial graph and processing it through a relational graph neural network. The performance of MARC is evaluated across four collaborative tasks, including a novel environment with heterogeneous agents. We conduct a comprehensive empirical analysis, comparing MARC against state-of-the-art MARL baselines, demonstrating improvements in both sample efficiency and asymptotic performance, as well as its potential for generalization. Our findings suggest that a minimal integration of spatial relational inductive biases as abstraction can yield substantial benefits without requiring complex designs or task-specific engineering. This work provides insights into the potential of relational state abstraction to address sample efficiency, a key challenge in MARL, offering a promising direction for developing more efficient algorithms in spatially complex environments.

AAAI 2025

Investigating Relational State Abstraction in Collaborative MARL

reinforcement learning

poster

We are pleased to announce the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25), which will be held in Philadelphia, Pennsylvania at the Pennsylvania Convention Center from February 25 to March 4, 2025.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

### [Invited Speakers](https://aaai.org/conference/aaai/aaai-25/aaai-25-invited-speakers/)

Register [here](https://aaai.org/conference/aaai/aaai-25/registration/)

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.



Identifying the Markov properties or conditional independencies of a collection of random variables is a fundamental task in statistics for modeling and inference. Existing approaches often learn the structure of a probabilistic graph, which encodes these dependencies, by assuming the variables follow a distribution with a simple parametric form. Moreover, the computational cost of many algorithms scales poorly for high-dimensional distributions, as they need to estimate all the edges in the graph simultaneously. In this work, we propose a scalable algorithm to infer the conditional independence relationships of each variable by exploiting the local Markov property. The proposed method, named Localized Sparsity Identification for Non-Gaussian Distributions (SING), estimates the graph without restricting the family of conditional distributions for each variable. We show that the localized SING algorithm includes existing approaches, such as neighborhood selection with Lasso, as a special case. We demonstrate the effectiveness of our algorithm in both Gaussian and non-Gaussian settings compared to existing methods. Lastly, we show the scalability of the proposed approach by applying it to high-dimensional non-Gaussian examples, including a biological dataset with more than 150 variables.

Learning Local Neighborhoods of Non-Gaussian Graphical Models

Time Series Forecasting aims at predicting future values for a time series and plays a crucial role in many real-world applications, e.g., finance, disease spread, or weather prediction. However, it is also a very challenging task, especially for long-term forecasting. In this paper, we introduce WaveletMixer, an iterative multi-levels, multi-resolutions and multi-phases approach to effectively capture with long-term dependencies of multivariate time series in both global and local perspectives for improving forecasting accuracy. WaveletMixer fundamentally differs to existing works in the following key aspects. First, it exploits multi-levels properties of Wavelet transformation to create multiple forecasting models for different frequency domains at different level of resolutions. Second, the relationships among different frequency domains are exploited to iteratively adjust all prediction models at all levels simultaneously in both local and global perspectives to reduce prediction errors and biases, thus significantly improving the final prediction accuracy. Third, while WaveletMixer is a general framework that can be used to boost performance of any deep-learning architecture (e.g., MLP, LSTM or Transformer), we additionally introduce TS-Learner, an MLP-based model to further enhance the performance in long-term forecasting. Extensive experiments have conducted on nine real-world datasets to demonstrate the performance of WaveletMixer compared to SOTA methods and to reveal its important characteristics. Code and extended experimental results are available in the supplementary material.

WaveletMixer: A Multi-Resolution Wavelets Based MLP-Mixer for Multivariate Long-Term Time Series Forecasting

Continual Learning (CL) is a highly relevant setting gaining traction in recent machine learning research. Among CL
works, architectural and hybrid strategies are particularly effective due to their potential to adapt the model architecture as
new tasks are presented. However, many existing solutions do not efficiently exploit model sparsity, and are prone to capacity 
saturation due to their inefficient use of available weights, which limits the number of learnable tasks. In this paper, we
propose TinySubNets (TSN), a novel architectural CL strategy that addresses the issues through the unique combination of
pruning with different sparsity levels, adaptive quantization, and weight sharing. Pruning identifies a subset of weights that
preserve model performance, making less relevant weights available for future tasks. Adaptive quantization allows a single 
weight to be separated into multiple parts which can be assigned to different tasks. Weight sharing between tasks boosts
the exploitation of capacity and task similarity, allowing for the identification of a better trade-off between model accuracy
and capacity. These features allow TSN to efficiently leverage the available capacity, enhance knowledge transfer, and reduce
computational resources consumption. Experimental results involving common benchmark CL datasets and scenarios show
that our proposed strategy achieves better results in terms of accuracy than existing state-of-the-art CL strategies. Moreover, 
our strategy is shown to provide a significantly improved model capacity exploitation.

TinySubNets: An Efficient and Low Capacity Continual Learning Strategy

We propose a novel hybrid calibration-free method FreeCap to accurately capture global multi-person motions in open environments. Our system combines a single LiDAR with expandable moving cameras, allowing for flexible and precise motion estimation in a unified world coordinate. In particular, We introduce a local-to-global pose-aware cross-sensor human-matching module that predicts the alignment among each sensor, even in the absence of calibration. Additionally, our coarse-to-fine sensor-expandable pose optimizer further optimizes the 3D human key points and the alignments, it is also capable of incorporating additional cameras to enhance accuracy. Extensive experiments on Human-M3 and FreeMotion datasets demonstrate that our method significantly outperforms state-of-the-art single-modal methods, offering an expandable and efficient solution for multi-person motion capture across various applications.

FreeCap: Hybrid Calibration-Free Motion Capture in Open Environments

We study neural network training (NNT): optimizing a neural network's parameters to minimize the training loss over a given dataset. NNT has been studied extensively under theoretic lenses, mainly on two-layer networks with linear or ReLU activation functions where the parameters can take any real value (here referred to as continuous NNT (C-NNT)). However, less is known about deeper neural networks, which adhere to substantially stronger capabilities in practice. In addition, the complexity of the discrete variant of the problem (D-NNT in short), in which the parameters are taken from a given finite set of options, has remained less explored despite its theoretical and practical significance.     


In this work, we show that the hardness of NNT is dramatically affected by the network depth. Specifically, we show that, under standard complexity assumptions, there is no bounded-error probabilistic polynomial time (BPP) algorithm for D-NNT even on instances with fixed dimensions and dataset size, having a deep architecture. As it is generally assumed that NP $\subseteq$ BPP, our result indicates that D-NNT is unlikely to be in NP. Furthermore, using a polynomial reduction we show that the above result also holds for C-NNT, albeit with more structured instances. We complement these results with a comprehensive list of NP-hardness lower bounds for D-NNT on two-layer networks, showing that fixing the number of dimensions, the dataset size, or the number of neurons in the hidden layer leaves the problem challenging. Finally, we obtain a pseudo-polynomial algorithm for D-NNT on a two-layer network with a fixed dataset size.

On the Hardness of Training Deep Neural Networks Discretely

Identifying the causal pathways of unfairness is a critical objective in improving policy design and algorithmic decision-making. However, prior work in causal fairness analysis requires knowledge of the causal graph, hindering practical applications in complex or low-knowledge domains. Moreover, relying on global discovery methods to learn causal structure from data can result in unstable performance with finite samples, potentially leading to contradictory fairness conclusions. To mitigate these issues, we introduce *local discovery for direct discrimination* (LD3): an algorithm tailored to uncover structural evidence of direct discrimination by identifying the causal parents of an outcome variable. LD3 performs a linear number of conditional independence tests relative to variable set size, and allows for latent confounding under the sufficient condition that no parent of the outcome is latent. LD3 prevents unnecessary adjustment, resulting in more interpretable adjustment sets for assessing unfairness.  We introduce a graphical criterion for identifying the *weighted controlled direct effect* (WCDE), a qualitative indicator of direct discrimination, and show that the knowledge returned by LD3 satisfies this criterion. We deploy LD3 for causal fairness analyses of two complex decision systems: criminal recidivism prediction and liver transplant allocation. Results on real-world data demonstrate more plausible causal relations than baselines, which took 46$\times$ to 5870$\times$ longer to execute.

Local Causal Discovery for Structural Evidence of Direct Discrimination

Beginner musicians often struggle to identify specific errors in their performances, such as playing incorrect notes or rhythms. 
There are two limitations in existing tools for music error detection: (1) Existing approaches rely on automatic alignment, which is error-prone due to small deviations between alignment targets; (2) There is a lack of sufficient data to train music error detection models, resulting in over-reliance on heuristics. 
To address (1), we propose a novel transformer model, \textit{Muse}, that takes audio inputs and outputs annotated music scores. 
This model can be trained end-to-end to implicitly align and compare performance audio with music scores through latent space representations. 
To address (2), we present a novel data generation technique capable of creating large-scale synthetic music error datasets. Our approach achieves a 64.1\% average Error Detection F1 score, improving upon prior work by 40 percentage points across 14 instruments. Compared with existing transcription methods repurposed for music error detection, our model can handle multiple instruments. This allows the model to scale across multiple instruments and generalize across datasets like MAESTRO and CocoChorales.

Detecting Music Performance Errors with Transformers

Large-scale text-to-image diffusion models, (e.g., DALL-E, SDXL) are capable of generating famous persons by simply referring to their names. $\textbf{\textit{Is it possible to make such models generate generic identities as simple as the famous ones, e.g., just use a name?}} $ In this paper, we explore the existence of a ``Name Space'', where any point in the space corresponds to a specific identity. Fortunately, we find some clues in the feature space spanned by text embedding of celebrities' names. Specifically, we first extract the embeddings of celebrities' names in the Laion5B dataset with the text encoder of diffusion models. Such embeddings are used as supervision to learn an encoder that can predict the name (actually an embedding) of a given face image. We experimentally find that such name embeddings work well in promising the generated image with good identity consistency. Note that like the names of celebrities, our predicted name embeddings are disentangled from the semantics of text inputs, making the original generation capability of text-to-image models well-preserved. Moreover, by simply plugging such name embeddings, all variants (e.g., from Civitai) derived from the same base model (i.e., SDXL) readily become identity-aware text-to-image models.

MagicNaming: Consistent Identity Generation by Finding a “Name Space” in T2I Diffusion Models

We study a hinted heterogeneous multi-agent multi-armed bandits problem $\texttt{HMA2B}$, where agents can query low-cost observations (hints) in addition to pulling arms. In this framework, each of the $M$ agents has a unique reward distribution over $K$ arms, and in $T$ rounds, they can observe the reward of the arm they pull only if no other agent pulls that arm.
The goal is to maximize the total utility by querying the minimal necessary hints without pulling arms, achieving time-independent regret. We study $\texttt{HMA2B}$ in both centralized and decentralized setups. Centralized algorithms $\texttt{HCLA}$ and $\texttt{GP-HCLA}$ use a central decision-maker for arm-pulling and hint queries, based on empirical means and $\texttt{kl-UCB}$ index, respectively, achieving $O(M^4K)$ regret with $O(MK\log T)$ adaptive hints. In decentralized setups, we propose two algorithms, $\texttt{HD-ETC}$ and $\texttt{EBHD-ETC}$, that allow agents to choose actions independently through collision-based communication and query hints uniformly until stopping, yielding $O(M^3K^2)$ regret with $O(M^3K\log T)$ hints. 
$\texttt{HD-ETC}$ stops hinting based on an assumed minimum gap $ \Delta_{\min}^\text{G} $, while $\texttt{EBHD-ETC}$ adapts hinting without knowing $ \Delta_{\min}^\text{G} $. 
Last, we establish lower bounds to prove the optimality of our results and confirm them through numerical simulations.

Heterogeneous Multi-Agent Bandits with Parsimonious Hints

Graph Masked AutoEncoder (GMAE) has recently attracted vast interest in handling graph-related tasks by adopting the masking-reconstruction learning paradigm. Most existing GMAE-based methods adhere to the homophily assumption, i.e., connected nodes share the same attributes. However, this assumption is not always right because most graphs from real-world applications are mixed by both homophilic and heterophilic edges. Therefore, it is necessary to distinguish them to improve the representative ability of GMAE. In this paper, we propose a $\textbf{T}$eacher-guided $\textbf{E}$dge $\textbf{D}$iscriminator for personalized graph $\textbf{M}$asked $\textbf{A}$uto$\textbf{E}$ncoder ($\textbf{TEDMAE}$). Specifically, we design a teacher-guided edge discriminator that distinguishes homophilic and heterophilic edges by leveraging the embeddings from teacher models with structure and attribute knowledge. Then, we present a personalized graph masked autoencoder that individually tailors the masking, encoding, and reconstruction processes for each graph. Finally, we optimize the model by minimizing two types of loss functions, i.e., the scaled cosine error (SCE) loss and the InfoNCE loss. Experimental results on 10 datasets demonstrate TEDMAE's superior performance on the tasks of node classification and node clustering.

Premium content

Next from AAAI 2025

Learning Local Neighborhoods of Non-Gaussian Graphical Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES