United States

Recently, a number of effective methods have been proposed to tackle the challenging task of Few-Shot Fine-Grained Image Classification (FS-FGIC). However, how to fully leverage the backbone network to discover and extract detailed features to generate more discriminative class prototypes, as well as how to accurately model the similarity relationship between query samples and the class prototypes, are still issues to be further considered. Therefore, we propose a novel progreSsively featUre refInement and conTinuous rElationship moDeling method, SUITED for short, to address these two issues existing in the State-of-the-Art FS-FGIC methods. Specifically, we design the Progressive Feature Refinement Module (PFRM) to fully exploit the backbone network&#39;s progressive feature extraction capabilities, forming multi-scale feature representations to further enhance discriminative features. Then, the Continuous Relationship Modeling Module (CRMM) is proposed to capture the dependencies between query samples and the corresponding class prototypes, achieving precise optimization of the distances among corresponding sample points in the feature space. We conducted extensive experiments on five fine-grained benchmark datasets, and the experimental results demonstrate that the proposed method is comprehensively ahead of the existing State-of-the-Art methods.

AAAI 2025

Few-Shot Fine-Grained Image Classification with Progressively Feature Refinement and Continuous Relationship Modeling

categorization

object detection

Recently, a number of effective methods have been proposed to tackle the challenging task of Few-Shot Fine-Grained Image Classification (FS-FGIC). However, how to fully leverage the backbone network to discover and extract detailed features to generate more discriminative class prototypes, as well as how to accurately model the similarity relationship between query samples and the class prototypes, are still issues to be further considered. Therefore, we propose a novel progreSsively featUre refInement and conTinuous rElationship moDeling method, SUITED for short, to address these two issues existing in the State-of-the-Art FS-FGIC methods. Specifically, we design the Progressive Feature Refinement Module (PFRM) to fully exploit the backbone network's progressive feature extraction capabilities, forming multi-scale feature representations to further enhance discriminative features. Then, the Continuous Relationship Modeling Module (CRMM) is proposed to capture the dependencies between query samples and the corresponding class prototypes, achieving precise optimization of the distances among corresponding sample points in the feature space. We conducted extensive experiments on five fine-grained benchmark datasets, and the experimental results demonstrate that the proposed method is comprehensively ahead of the existing State-of-the-Art methods.

poster

We are pleased to announce the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25), which will be held in Philadelphia, Pennsylvania at the Pennsylvania Convention Center from February 25 to March 4, 2025.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

### [Invited Speakers](https://aaai.org/conference/aaai/aaai-25/aaai-25-invited-speakers/)

Register [here](https://aaai.org/conference/aaai/aaai-25/registration/)

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.



Recent advancements in open-source code large language models (LLMs) have been driven by fine-tuning on the data generated from powerful closed-source LLMs, which are expensive to obtain. This paper explores whether it is possible to use a fine-tuned open-source model to generate additional data to augment its instruction-tuning dataset. We make two observations: (1) A code snippet can serve as the response to different instructions. (2) Instruction-tuned code LLMs perform better at translating code into instructions than the reverse.
Based on these observations, we propose Inverse-Instruct, a data augmentation technique that uses a fine-tuned LLM to generate additional instructions of code responses from its own training dataset. The additional instruction-response pairs are added to the original dataset, and a stronger code LLM can be obtained by fine-tuning on the augmented dataset. We empirically validate Inverse-Instruct on a range of open-source code models (e.g. CodeLlama-Python and DeepSeek-Coder) and benchmarks (e.g., HumanEval(+), MBPP(+), DS-1000 and MultiPL-E), showing it consistently improves the base models.

InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct

Multi-modal Federated Learning (MFL) is a distributed machine learning paradigm that enables multiple participants with multi-modal data to collaboratively train a global model for multi-modal tasks without sharing their local data.
MFL typically deploys the trained global model as an EaaS, allowing participants to obtain embeddings for downstream tasks. However, it increases the risk of unauthorized copying and leakage of the model.
Protecting the ownership of the MFL global model while maintaining model performance is challenging.
In this paper, we propose the first general model ownership protection framework for MFL, named MFL-Owner. 
MFL-Owner decouples the watermark embedding process from the model training process and addresses both ownership verification and traceability, effectively safeguarding the interests of the MFL collective.
MFL-Owner leverages the concept of orthogonal transformations by incorporating a linear transformation matrix with orthogonal constraints into the model, achieving high-quality ownership verification and traceability with minimal impact on model performance.
To enhance the practicality of the watermark and prevent conflicts among multiple clients during tracing, we propose a trigger dataset selection method based on out-of-distribution data combined with Gaussian noise perturbation.
Our experiments on multiple datasets demonstrate that MFL-Owner is effective for model ownership verification and traceability for MFL.
Our code is available at https://anonymous.4open.science/r/MFL-Owner-2A91.

MFL-Owner: Ownership Protection for Multi-modal Federated Learning via Orthogonal Transform Watermark

We consider the contextual combinatorial bandit setting where in each round, the learning agent, e.g., a recommender system, selects a subset of "arms,'' e.g., products, and observes rewards for both the individual base arms, which are a function of known features (called "context''), and the super arm (the subset of arms), which is a function of the base arm rewards. The agent's goal is to simultaneously learn the unknown reward functions and choose the highest-reward arms. For example, the "reward'' may represent a user's probability of clicking on one of the recommended products. Conventional bandit models, however, employ restrictive reward function models in order to obtain performance guarantees. We make use of deep neural networks to estimate and learn the unknown reward functions and propose Neural UCB Clustering (NeUClust), which adopts a clustering approach to select the super arm in every round by exploiting underlying structure in the context space. Unlike prior neural bandit works, NeUClust uses a neural network to estimate the super arm reward and select the super arm, thus eliminating the need for a known optimization oracle. We non-trivially extend prior neural combinatorial bandit works to prove that NeUClust achieves $\widetilde{O}(\widetilde{d}\sqrt{T})$ regret, where $\widetilde{d}$ is the effective dimension of a neural tangent kernel matrix, $T$ the number of rounds. Experiments on real world recommendation datasets show that NeUClust achieves better regret and reward than other contextual combinatorial and neural bandit algorithms.

Neural Combinatorial Clustered Bandits for Recommendation Systems

When it comes to expensive black-box optimization problems, Bayesian Optimization (BO) is a well-known and powerful solution. Many real-world applications involve a large number of dimensions, hence scaling BO to high dimension is of much interest. However, state-of-the-art high-dimensional BO methods still suffer from the curse of dimensionality, highlighting the need for further improvements. In this work, we introduce BOIDS, a novel high-dimensional BO algorithm that guides optimization by a sequence of one-dimensional direction lines using a novel tailored line-based optimization procedure. To improve the efficiency, we also propose an adaptive selection technique to identify most optimal lines for each round of line-based optimization. Additionally, we incorporate a subspace embedding technique for better scaling to high-dimensional spaces. We further provide theoretical analysis of our proposed method to analyze its convergence property. Our extensive experimental results show that BOIDS outperforms state-of-the-art baselines on various synthetic and real-world benchmark problems.

BOIDS: High-Dimensional Bayesian Optimization via Incumbent-Guided Direction Lines and Subspace Embeddings

Variate tokenization, which independently embeds each variate as separate tokens, has achieved remarkable improvements in multivariate time series forecasting. However, employing self-attention with variate tokens incurs a quadratic computational cost with respect to the number of variates, thus limiting its training efficiency for large-scale applications. To address this issue, we propose *VarDrop*, a simple yet efficient strategy that reduces the token usage by omitting redundant variate tokens during training. *VarDrop* adaptively excludes redundant tokens within a given *batch*, thereby reducing the number of tokens used for dot-product attention while preserving essential information. Specifically, we introduce $k$-dominant frequency hashing ($k$-DFH), which utilizes the ranked dominant frequencies in the frequency domain as a hash value to efficiently group variate tokens exhibiting similar periodic behaviors. Then, only representative tokens in each group are sampled through stratified sampling. By performing sparse attention with these selected tokens, the computational cost of scaled dot-product attention is significantly alleviated. Experiments conducted on public benchmark datasets demonstrate that *VarDrop* outperforms existing efficient baselines. The source code is publicly available at https://anonymous.4open.science/r/VarDrop.

VarDrop: Enhancing Training Efficiency by Reducing Variate Redundancy in Periodic Time Series Forecasting

Entrusted with the goal of pixel-level object classification, the
semantic segmentation networks entails the laborious prepa-
ration of pixel-level annotation masks. To obtain pixel-level
annotation masks for a given class without human efforts, re-
cent few works have proposed to generate pairs of images and
annotation masks by employing image and text relationships
modeled by text-to-image generative models, especially Sta-
ble Diffusion. However, these works do not fully exploit the
capability of text-guided Diffusion models and thus require a
pre-trained segmentation network, careful text prompt tuning,
or the training of a segmentation network to generate final an-
notation masks. In this work, we take a closer look at attention
mechanisms of Stable Diffusion, from which we draw con-
nections with classical seeded segmentation approaches. In
particular, we show that cross-attention alone provides very
coarse object localization, which however can provide initial
seeds. Then, akin to region expansion in seeded segmenta-
tion, we utilize the semantic-correspondence-modeling capa-
bility of self-attention to iteratively spread the attention to the
whole class from the seeds using multi-scale self-attention
maps. We also observe that a simple-text-guided synthetic
image often has a uniform background, which is easier to
find correspondences, compared to complex-structured ob-
jects. Thus, we further refine a mask using a more accurate
background mask. Our proposed method, dubbed SeeDiff,
generates high-quality masks off-the-shelf from Stable Dif-
fusion, without additional training procedure, prompt tuning,
or a pre-trained segmentation network.

SeeDiff: Off-the-Shelf Seeded Mask Generation from Diffusion Models

Vision Transformers (ViTs) have achieved remarkable success in various computer vision tasks. However, ViTs have a huge computational cost due to their inherent reliance on multi-head self-attention (MHSA), prompting efforts to accelerate ViTs for practical applications. To this end, recent works aim to reduce the number of tokens, mainly focusing on how to effectively prune or merge them. Nevertheless, since ViT tokens are generated from non-overlapping grid patches, they usually do not convey sufficient semantics, making it incompatible with efficient ViTs. To this end, we propose ImagePiece, a novel re-tokenization strategy for Vision Transformers. Following the MaxMatch strategy of NLP tokenization, ImagePiece groups semantically insufficient yet locally coherent tokens until they convey meaning. This simple retokenization is highly compatible with previous token reduction methods, being able to drastically narrow down relevant tokens, enhancing the inference speed of DeiT-S by 54\% (nearly 1.5$\times$ faster) while achieving a 0.39\% improvement in ImageNet classification accuracy. For hyper-speed inference scenarios (with 251\% acceleration), our approach surpasses other baselines by an accuracy over 8\%.

ImagePiece: Content-aware Re-tokenization for Efficient Image Recognition

This study aims to build a pre-trained Graph Neural Network (GNN) model on molecules without human annotations or prior knowledge. Although various attempts have been proposed to overcome limitations in acquiring labeled molecules, the previous pre-training methods still rely on semantic subgraphs, i.e., functional groups. Only focusing on the semantic subgraphs could overlook the graph-level distinctions. The key challenge to build a pre-trained GNN on molecules is how to (1) generate well-distinguished graph-level representations and (2) automatically discover the functional groups without prior knowledge. To solve it, we propose a novel Subgraph-conditioned Graph Information Bottleneck for pre-training GNNs to recognize core subgraphs (graph cores) and significant subgraphs, named S-CGIB. The key idea is that the graph cores contain compressed and sufficient information that could generate well-distinguished graph-level representations and reconstruct the input graph conditioned on significant subgraphs across molecules. To discover significant subgraphs without prior knowledge about functional groups, we propose generating a set of functional group candidates, i.e., ego networks, and using an attention-based interaction between the graph core and the candidates. Despite being learned from self-supervised learning, our learned subgraphs match the real-world functional groups. Extensive experiments on various molecule datasets across four domains demonstrate the superiority of S-CGIB.

Pre-Training Graph Neural Networks on Molecules by Using Subgraph-Conditioned Graph Information Bottleneck

Compositional generalization is crucial for artificial intelligence agents tackling intricate reasoning over vision and language (V\&L) problems. While neuro-symbolic methods have demonstrated potential in understanding compositional structures, they face challenges such as the need for symbolic domain representations that typically involve a set of predefined predicates, difficulties in deriving domain predicates from raw data, and the requirement for differentiable operations to compose primitive concepts. To address these issues, we propose NeSyCoCo, which is built on the existing neuro-symbolic frameworks that leverage large language models (LLMs) for obtaining symbolic representations of the domain and map them to differentiable neural computations for V\&L reasoning. Our approach a) augments the natural language inputs with their dependency structure to improve the accuracy of symbolic representations, b) utilizes distributed word representations for handling the variety of linguistically motivated logical predicates that are linked to neural modules, and c) utilizes soft composition of normalized predicate scores for better semantic alignment between symbolic compositions and differentiable operations. NeSyCoCo achieves state-of-the-art results on the ReaSCAN and CLEVR-CoGenT compositional generalization benchmarks, as well as the CLEVR vision-language benchmark. It also maintains high accuracy with new, similar concepts in the CLEVR-SYN benchmark.

NeSyCoCo: A Neuro-Symbolic Concept Composer for Compositional Generalization

Real-time object detection is critical for the decision-making process for many real-world applications, such as collision avoidance and path planning in autonomous driving. This work presents an innovative real-time streaming perception method, Transtreaming, which addresses the challenge of real-time object detection with dynamic computational delay. The core innovation of Transtreaming lies in its adaptive delay-aware transformer, which can concurrently predict multiple future frames and select the output that best matches the real-world present time, compensating for any system-induced computation delays.
The proposed model outperforms the existing state-of-the-art methods, even in single-frame detection scenarios, by leveraging a transformer-based methodology. It demonstrates robust performance across a range of devices, from powerful V100 to modest 2080Ti, achieving the highest level of perceptual accuracy on all platforms. Unlike most state-of-the-art methods that struggle to complete computation within a single frame on less powerful devices, Transtreaming meets the stringent real-time processing requirements on all kinds of devices. The experimental results emphasize the system's adaptability and its potential to significantly improve the safety and reliability for many real-world systems, such as autonomous driving.
Our code is open-sourced on https://anonymous.4open.science/r/Transtreaming-7333.

Few-Shot Fine-Grained Image Classification with Progressively Feature Refinement and Continuous Relationship Modeling

Downloads

Next from AAAI 2025

InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Few-Shot Fine-Grained Image Classification with Progressively Feature Refinement and Continuous Relationship Modeling

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAAI 2025

InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads