United States

Building on the success of text-to-image diffusion models (DPMs), image editing is an important application to enable human interaction with AI-generated content.
Among various editing methods, editing within the prompt space gains more attention due to its capacity and simplicity of controlling semantics.
However, since diffusion models are commonly pretrained on descriptive text captions, direct editing of words in text prompts usually leads to completely different generated images, violating the requirements for image editing.
On the other hand, existing editing methods usually consider introducing spatial masks to preserve the identity of unedited regions, which are usually ignored by DPMs and therefore lead to inharmonic editing results.
Targeting these two challenges, in this work, we propose to disentangle the comprehensive image-prompt interaction into several item-prompt interactions, with each item linked to a special learned prompt.
The resulting framework, named \textbf{D-Edit}, is based on pretrained diffusion models with cross-attention layers disentangled and adopts a two-step optimization to build item-prompt associations.
Versatile image editing can then be applied to specific items by manipulating the corresponding prompts.
We demonstrate state-of-the-art results in four types of editing operations including image-based, text-based, mask-based editing, and item removal, covering most types of editing applications, all within a single unified framework.
Notably, D-Edit is the first framework that can (1) achieve item editing through mask editing and (2) combine image and text-based editing.
We demonstrate the quality and versatility of the editing results for a diverse collection of images through both qualitative and quantitative evaluations.

AAAI 2025

An Item Is Worth a Prompt: Versatile Image Editing with Disentangled Contro

autoencoders

deep generative models

poster

We are pleased to announce the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25), which will be held in Philadelphia, Pennsylvania at the Pennsylvania Convention Center from February 25 to March 4, 2025.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

### [Invited Speakers](https://aaai.org/conference/aaai/aaai-25/aaai-25-invited-speakers/)

Register [here](https://aaai.org/conference/aaai/aaai-25/registration/)

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.



As Large Language Models (LLMs) excel in traditional NLP tasks, their application to complex, strategic games like poker poses a new challenge. Poker, an incomplete information game, demands a multitude of skills such as mathematics, reasoning, planning, and understanding of game theory. We evaluate prominent models such as GPT, Llama and Gemma series models, finding that all state-of-the-art LLMs underperform in playing optimal poker. To enhance LLM’s poker-playing capabilities, we formulated strategy learning in a teacher-student framework and performed knowledge distillation from Game Theory Optimal (GTO) strategy solvers. Through aggregating a novel training dataset comprising decisions that represent an unexploitable Nash Equilibrium strategy and supervised fine-tuning (SFT), we distill the GTO solver's output into LLMs, resulting in substantial improvements in model performance. Meanwhile, we introduce PokerBench - a benchmark for evaluating the poker-playing abilities of LLMs. PokerBench consists of a comprehensive compilation of 2,000 essential scenarios in poker. We validate PokerBench by demonstrating that higher test accuracy on PokerBench correlates well with higher expected value of actions taken, leading to a higher win rate in actual gameplay. PokerBench thus presents a unique benchmark for an efficient and reliable evaluation of the poker-playing ability of LLMs as well as a comprehensive benchmark to study the progress of LLMs in complex game-playing scenarios with incomplete information.

PokerBench: Training Large Language Models to Become Professional Poker Players

This paper challenges the prevailing view that convolutional neural network (CNN) filters become increasingly specialized in deeper layers. Motivated by recent observations of clusterable repeating patterns in depthwise separable CNNs (DS-CNNs) trained on ImageNet, we extend this investigation across various domains and datasets. Our analysis of DS-CNNs reveals that deep filters maintain generality, contradicting the expected transition to class-specific features. We demonstrate the generalizability of these filters through transfer learning experiments, showing that frozen filters from models trained on different datasets perform well and can be further improved when sourced from larger, better-performing models. Our findings indicate that spatial features learned by depthwise separable convolutions remain generic across all layers, domains, and architectures. This research provides new insights into the nature of generalization in neural networks, particularly in DS-CNNs, and has significant implications for transfer learning and model design.

The Master Key Filters Hypothesis: Deep Filters Are General

Neural combinatorial optimization (NCO) has gained significant attention due to the potential of deep learning to efficiently solve combinatorial optimization problems. NCO has been widely applied to job shop scheduling problems (JSPs) with the current focus predominantly on deterministic problems. In this paper, we propose a novel attention-based scenario processing module (SPM) to extend NCO methods for solving stochastic JSPs. Our approach explicitly incorporates stochastic information by an attention mechanism that captures the embedding of sampled scenarios (i.e., an approximation of stochasticity). Fed with the embedding, the base neural network is intervened by the attended scenarios, which accordingly learns an effective policy under stochasticity. We also propose a training paradigm that works harmoniously with either the expected makespan or Value-at-Risk objective. Results demonstrate that our approach outperforms existing learning and non-learning methods for the flexible JSP problem with stochastic processing times on a variety of instances. In addition, our approach holds significant generalizability to varied numbers of scenarios and disparate distributions.

Neural Combinatorial Optimization for Stochastic Flexible Job Shop Scheduling Problems

We propose SoundBrush, a model that uses sound as a brush to edit and manipulate visual scenes. We extend the generative capabilities of the Latent Diffusion Model (LDM) to incorporate audio information for editing visual scenes. Inspired by existing image-editing works, we frame this task as a supervised learning problem and leverage various off-the-shelf models to construct a sound-paired visual scene dataset for training. This richly generated dataset enables SoundBrush to learn to map audio features into the textual space of the LDM, allowing for visual scene editing guided by diverse in-the-wild sound. Unlike existing methods, SoundBrush can accurately manipulate the overall scenery or even insert sounding objects to best match the audio inputs while preserving the original content. Furthermore, by integrating with novel view synthesis techniques, our framework can be extended to edit 3D scenes, facilitating sound-driven 3D scene manipulation.

SoundBrush: Sound as a Brush for Visual Scene Editing

Implicit imitation reinforcement learning (IIRL) is a framework that aims to aid a trainee agent’s learning process via observing the state transitions of a mentor, but without access to the latter's action information. Standard IIRL assumes a shared Markov decision process (MDP) between the mentor and trainee, consequently implying an identical action space. This restriction imposes limitations on the applicability of implicit imitation frameworks in real-life scenarios where, possibly due to variations in physical characteristics, the mentor agent may possess distinct own actions, thereby creating a heterogeneous action setting. In this work, we extend the deep implicit imitation Q-networks (DIIQN) method -an online, model-free, deep RL algorithm for implicit imitation- to allow for heterogeneous action sets between mentor and trainee agents. Equipped with our heterogeneous actions DIIQN (HA-DIIQN) method, a trainee agent can harvest the benefits of IIRL even in heterogeneous action settings, achieving accelerated learning and outperforming non-optimal mentor agents.

Deep Implicit Imitation Reinforcement Learning in Heterogeneous Action Settings

Deep learning models with large-scale backbones have been increasingly adopted to tackle complex visual question answering (VQA) problems in real settings. While providing powerful learning capacities to handle the high-dimensional and multimodal VQA data, these models tend to suffer from the memorization effect leading to overconfident predictions. This can significantly limit their applicability in critical domains (e.g., medicine, cyber-security, and public safety), where confidently wrong predictions may lead to severe consequences. In this work, we propose to perform novel low-rank network factorization, resulting in much better-calibrated networks. These low-rank factorized networks are then aggregated into an ensemble guided by a generalized focal loss to further improve the overall performance and calibration. The overall framework, referred to as the Generalized focal Loss Ensemble of low-rank Networks (GLEN), is an important step toward developing well-calibrated VQA models. We demonstrate the effectiveness of our low-rank factorization-based ensemble technique towards calibrating VQA models, theoretically demonstrate that the generalized focal loss provides a more balanced bias-variance trade-off, which guarantees to lower the confidence of the incorrect predictions. Extensive experimentation conducted on benchmark datasets and comparison on various VQA models shows that GLEN leads to much better calibration over both in-distribution and out-of-distribution data without sacrificing the VQA accuracy.

GLEN: Generalized Focal Loss Ensemble of Low-Rank Networks for Calibrated Visual Question Answering

In this paper, we propose a new benchmark called "Archaeological Piece Grouping." In the field of archaeology, it is common for broken archaeological pieces, such as artifact fragments, to be mixed. Archaeologists often spend significant time distinguishing these pieces and categorizing them into different groups. Our benchmark introduces a novel, comprehensive dataset named ArcPie, along with new evaluation metrics for this task. Additionally, we propose a new framework called "3D Probabilistic Graph Search" (3DPGS) to address the problem of grouping mixed archaeological pieces. This framework includes a relation network designed to learn the relationships among all the input 3D pieces. Utilizing the relationships learned, our framework generates a probabilistic matching graph that describes the affinity of any two pieces. We also introduce a novel search algorithm to identify groups according to this matrix. Our framework significantly outperforms other baselines. We commit to releasing our code and dataset if our paper is accepted.

3DPGS: 3D Probabilistic Graph Search for Archaeological Piece Grouping

Wearable Human Action Recognition (wHAR) uses motion sensor data to identify human movements, which is essential for mobile and wearable devices. However, traditional wHAR systems are only trained on a limited set of activities. Hence, they fail to generalize to diverse human motions, prompting Zero-Shot Learning (ZSL). Existing ZSL methods for wHAR focus solely on augmenting labels, such as representing them as attribute matrices, images, videos, or text. We propose ZeroHAR that enhances ZSL by not just focusing on activity labels, but by augmenting motion data with sensor context features. Our approach incorporates information about the sensor type, the Cartesian axis of the data, and the sensor's body position, providing the model with crucial spatial and biomechanical insights. This helps the model generalize better to new actions. First, we train the model by aligning the latent space of the motion time-series with its corresponding sensor context, while distancing it from unrelated sensor contexts. Finally, we train the model using the target activity descriptions. We tested our method against eight baselines on five benchmark HAR datasets with various sensors, placements, and activities. Our model shows exceptional generalizability across 18 motion time series classification benchmark datasets, outperforming the best baselines by 262% in the zero-shot setting.

ZeroHAR: Sensor Context Augments Zero-Shot Wearable Action Recognition

The problem of constrained Markov game has recently attracted interests in the study of multi-agent reinforcement learning (MARL). The existing literature has focused on safe MARL problems where safety constraints are imposed for each agent individually. In this work, we consider Markov potential game (MPG) with a shared constraint, where the cost function with respect to the constraint depends on state and joint actions of all agents. We adopt a primal-dual framework to tackle the problem, and establish the Slater condition to ensure the strong duality. Moreover, we propose our primal-dual learning algorithm for learning approximate Nash equilibrium in MPG with shared constraint. Thanks to the novel design of the dual update, we provide asymptotic convergence on the weighted output policy. Specifically, we prove that both the value function gap and the constraint violation of the output policy converge at the rate $\mathcal{O}(\epsilon+1/\sqrt{T})$, where $\epsilon$ is the accuracy level of the primal update, and $T$ is the number of iterations. We further show that the weighted output policy outperform the existing uniform policy under certain regimes.

Learning Nash Equilibrium of Markov Potential Games with a Shared Constraint via Primal-Dual Optimization

Distributionally robust optimization addresses out-of-sample challenges such as overfitting and distribution shifts by employing an adversarial approach over a range of possible data distributions, known as the ambiguity set. To avoid overly conservative and inaccurate models, ambiguity sets must encompass realistic probability distributions by incorporating all relevant information from the nominal distribution.

Assuming that nominal distributions arise from a structural causal model with a directed acyclic graph $\mathcal{G}$ and structural functions, previous approaches like adapted optimal transport and $\mathcal{G}$-causal optimal transport have only utilized causal graph information in designing ambiguity sets. 

To obtain more realistic distributions, we propose incorporating structural functions into the causal graph when designing the ambiguity set. We define structural causal optimal transport and its corresponding ambiguity set, demonstrating its desirable properties and relation to previous definitions of ambiguity sets. 

One advantage of our approach is the relaxed version, where the hard constraint is replaced by a regularization term. This enables the use of an efficient algorithm with difference-of-convex programming to find solutions for structural causal optimal transport. We show that in real-world applications, estimating structural functions using our methods effectively leads to optimal solutions.

Finally, we discuss the radius of ambiguity sets and demonstrate how our method mitigates the curse of dimensionality common in optimal transport problems. Consequently, we achieve faster shrinkage with dimension-free order as the number of collected samples increases.

Content not yet available

An Item Is Worth a Prompt: Versatile Image Editing with Disentangled Contro

Downloads

Next from AAAI 2025

PokerBench: Training Large Language Models to Become Professional Poker Players

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Content not yet available

An Item Is Worth a Prompt: Versatile Image Editing with Disentangled Contro

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAAI 2025

PokerBench: Training Large Language Models to Become Professional Poker Players

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads