United States

While Large Language Models (LLMs) have shown exceptional generalization capabilities, their ability to process graph data, such as molecular structures, remains limited. To bridge this gap, this paper proposes Graph2Token, an efficient solution that aligns graph tokens to LLM tokens. The key idea is to represent a graph token with the LLM token vocabulary, without fine-tuning the LLM backbone. To achieve this goal, we first construct a molecule-text paired dataset from multi-sources, including CHEBI and HMDB, to train a graph structure encoder, which reduces the distance between graphs and texts representations in the feature space. Then, we propose a novel alignment strategy that associates a graph token with LLM tokens. To further unleash the potential of LLMs, we collect molecular IUPAC name identifiers, which are incorporated into the LLM prompts. By aligning molecular graphs as special tokens, we can activate LLMs&#39; generalization ability to molecular few-shot learning. Extensive experiments on molecular classification and regression tasks demonstrate the effectiveness of our proposed Graph2Token. Code is available at https://anonymous.4open.science/r/MolGraph-9C0C.

AAAI 2025

Bridging Molecular Graphs and Large Language Models

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-23 is the Thirty-Seventh AAAI Conference on Artificial Intelligence. The theme of this conference is to create collaborative bridges within and beyond AI. Like previous AAAI conferences, AAAI-23 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and two new activities: a Bridge Program and a Lab Program. Many of these activities are tailored to the theme of bridges and all are selected according to the highest standards, with additional programs for students and young researchers. 
AAAI is providing you with a conference planner, which you can use to help organize your itinerary of activities. This includes talks to attend in person, talks to attend remotely, breaks with colleagues and your site seeing activities. To access this conference planner, please go to [https://aaai-2023.takemobi.io](https://aaai-2023.takemobi.io).

In order to access this site, you need to register. If you haven't already, please register [here](https://aaai.org/Conferences/AAAI-23/registration/).


AAAI 2023

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines.

While Large Language Models (LLMs) have shown exceptional generalization capabilities, their ability to process graph data, such as molecular structures, remains limited. To bridge this gap, this paper proposes Graph2Token, an efficient solution that aligns graph tokens to LLM tokens. The key idea is to represent a graph token with the LLM token vocabulary, without fine-tuning the LLM backbone. To achieve this goal, we first construct a molecule-text paired dataset from multi-sources, including CHEBI and HMDB, to train a graph structure encoder, which reduces the distance between graphs and texts representations in the feature space. Then, we propose a novel alignment strategy that associates a graph token with LLM tokens. To further unleash the potential of LLMs, we collect molecular IUPAC name identifiers, which are incorporated into the LLM prompts. By aligning molecular graphs as special tokens, we can activate LLMs' generalization ability to molecular few-shot learning. Extensive experiments on molecular classification and regression tasks demonstrate the effectiveness of our proposed Graph2Token. Code is available at https://anonymous.4open.science/r/MolGraph-9C0C.

poster

We are pleased to announce the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25), which will be held in Philadelphia, Pennsylvania at the Pennsylvania Convention Center from February 25 to March 4, 2025.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

### [Invited Speakers](https://aaai.org/conference/aaai/aaai-25/aaai-25-invited-speakers/)

Register [here](https://aaai.org/conference/aaai/aaai-25/registration/)

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-25 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.



Recent studies have revealed the vulnerability of graph neural networks (GNNs) to adversarial attacks. In practice, effectively attacking GNNs is not easy. Existing attack methods primarily focus on modifying the topology of the graph data. In many scenarios, attackers do not have the authority to manipulate the graph's topology, making such attacks challenging to execute. Although node injection attacks are more feasible than modifying the topology, current injection attacks rely on knowledge of the victim model's architecture. This dependency significantly degrades attack quality when there is inconsistency in the victim models. Moreover, the generation of injected nodes often lacks precise control over features, making it difficult to balance attack effectiveness and stealthiness. In this paper, we investigate a node injection attack under model-agnostic conditions and propose $\underline{\textbf{T}}$argeted $\underline{\textbf{E}}$vasion $\underline{\textbf{A}}$ttack via $\underline{\textbf{N}}$ode $\underline{\textbf{I}}$njection (TEANI). Specifically, TEANI models the generation of adversarial nodes as a Markov process. Without considering the target model's structure, it guides the agent to select features that maximize attack effectiveness within a budget, based solely on the results of queries to a black-box model. Extensive experiments on real-world datasets and mainstream GNN models demonstrate that the proposed TEANI poses more effective and imperceptible threats than state-of-the-art attack methods.

Highly Imperceptible Black-Box Graph Injection Attacks with Reinforcement Learning

Code-switching is a linguistic phenomenon in which differrnt languages are used interactively during conversation. It poses significant performance challenges to natural language processing (NLP) tasks due to the often monolingual nature of the underlying system. We focus on sentence-level semantic associations between the different code-switching expressions. And we propose an innovative task-free semantic learning method based on the semantic property. Specifically, there are many different ways of languages switching for a sentence with the same meaning. We refine this into a semantic computational method by designing the loss of semantic invariant constraint during the model optimization. In this work, we conduct thorough experiments on speech recognition, speech translation, and language modelling tasks. The experimental results fully demonstrate that the proposed method can widely improve the performance of code-switching related tasks.

Code-switching Mediated Sentence-level Semantic Learning

Marked temporal point processes (MTPPs) have been shown to be extremely effective in modeling continuous time event sequences (CTESs). In this work, we present adversarial attacks designed specifically for MTPP models. A key criterion for a good adversarial attack is its imperceptibility. For objects such as images or text, this is often achieved by bounding perturbation in some fixed $L_p$ norm-ball. However, similarly minimizing distance norms between two CTESs in the context of MTPPs is challenging due to their sequential nature and varying time-scales and lengths. We address this challenge by first permuting the events and then incorporating the additive noise to the arrival timestamps. However, the worst case optimization of such adversarial attacks is a hard combinatorial problem, requiring exploration across a permutation space that is factorially large in the length of the input sequence. As a result, we propose a novel differentiable scheme - PermTPP - using which we can perform adversarial attacks by learning to minimize the likelihood, while minimizing the distance between two CTESs. Our experiments on four real-world datasets demonstrate the offensive and defensive capabilities, and lower inference times of PermTPP.

Differentiable Adversarial Attacks for Marked Temporal Point Processes

Large language models (LLMs) have demonstrated significant progress in multilingual language understanding and generation. However, due to the imbalance in training data, their capabilities in non-English languages are limited. Recent studies revealed the English-pivot multilingual mechanism of LLMs, where LLMs implicitly convert non-English queries into English ones at the bottom layers and adopt English for thinking at the middle layers. However, due to the absence of explicit supervision for cross-lingual alignment in the intermediate layers of LLMs, the internal representations during these stages may become inaccurate. In this work, we introduce a deep supervision fine-tuning method (DFT) that incorporates additional supervision in the internal layers of the model to guide its workflow. 
Specifically, we introduce two training objectives on different layers of LLMs: one at the bottom layers to constrain the conversion of the target language into English, and another at the middle layers to constrain reasoning in English. To effectively achieve the guiding purpose, we designed two types of supervision signals: logits and feature, which represent a stricter constraint and a relatively more relaxed guidance. Our method guides the model to not only consider the final generated result when processing non-English inputs but also ensure the accuracy of internal representations. We conducted extensive experiments on typical English-centric large models, LLaMA-2 and Gemma-2, and the results on multiple multilingual datasets show that our method significantly outperforms traditional fine-tuning methods.

Enhancing Non-English Capabilities of English-Centric Large Language Models Through Deep Supervision Fine-Tuning

Neurosymbolic approaches blend the effectiveness of symbolic reasoning with the flexibility of neural networks. In this work, we propose an architecture for the generation of SQL queries that leverages neurosymbolic reasoning as an alternative to simply scaling the neural model size. To this end, we integrate a generic pretrained Language Model (LM) with a symbolic query checker, a neural query checker and a symbolic test and repair module that catch and correct errors made by the LM on SQL queries. We focus on improving the performance of smaller open-source LMs out-of-the-box, and we find that our tool, Xander, increases accuracy by an average of 10.9% and reduces runtime by an average of 28%  compared to the LM without Xander, enabling a smaller LM (with Xander) to outperform its four-times-larger counterpart (without Xander).

Enhancing SQL Query Generation with Neurosymbolic Reasoning

In this work, we introduce Modulated Flows (ModFlows), a novel approach for color transfer between images based on rectified flows. The primary goal of the color transfer is to adjust the colors of a target image to match the color distribution of a reference image. Our technique is based on optimal transport and executes color transfer as an invertible transformation within the RGB color space. The ModFlows utilizes the bijective property of flows, enabling us to introduce a common intermediate color distribution and build a dataset of rectified flows. We train an encoder on this dataset to predict the weights of a rectified model for new images. After training on a set of optimal transport plans, our approach can generate optimal plans for new pairs of distributions without additional fine-tuning. We additionally show that the trained encoder provides an image embedding, associated only with its color style. The presented method is capable of processing 4K images and achieves the state-of-the-art performance in terms of content and style similarity. Our code is avalable at https://anonymous.4open.science/r/modflows-FBB1/.

Color Transfer with Modulated Flows

The fault location task in power grids is crucial for maintaining social order and ensuring public safety. However, existing methods that rely on tabular state records often neglect the intrinsic topological influences of transmission lines, resulting in a segmented approach to fault location that consists of multiple stages. In this paper, we propose an \textbf{D}isentangled \textbf{T}able-\textbf{G}raph representation framework, termed DTG, which integrates fault location tasks at coarse-grained line levels and fine-grained point levels within an end-to-end learning paradigm. Our innovative disentanglement strategy produces interpretable attribution coefficients that connect tabular records and transmission line topology, thereby facilitating fault location at both line- and point-levels. The joint prediction tasks designed around our disentangled tabular graph representation promote mutual information exchange between features and topology of transmission lines in an interpretable manner.Experimental results on the 7-bus system, 36-bus system and a realistic 325-bus system in China demonstrate that the proposed method adapt to different topological structures and handle different types of faults. Compared to traditional methods, DTG4Power achieves high accuracy in both fault lines and fault points.Our code will be made publicly available.

Disentangled Table-Graph Representation for Interpretable Transmission Line Fault Location

Generative information retrieval, which encodes documents into a model and retrieves them by generating document identifiers, is a novel retrieval framework and has attracted much research interest. However, it is a topical problem to design efficient and accurate docid to index documents. Existing methods for designing docids tend to consider only the relevance of docids to the corresponding documents, while neglecting the ability of docids to distinguish the corresponding documents from similar ones, which is crucial for retrieval.
In this paper, we design learnable descriptive and discriminative document Identifiers (DocIDs) for generative retrieval and propose the paired retrieval model D3Gen. The DocIDs can be semantically similar to the documents (descriptive) and distinguish similar documents (discriminativate) in the corpus, thus enhancing retrieval performance.We use a comparative learning assisted generative retrieval task to enable the model to understand the document and complete the generative retrieval. We then design a DocID selection method called NR-IDR to select DocIDs based on the retrieval model's understanding of the document. On the input side of the retrieval model D3Gen, we also design filters to filter diverse, multi-granular pseudo query to help the model fully understand the semantics of the document. Our experiments on the MS MARCO and NQ320k dataset illustrate the effectiveness of the approach.

Descriptive and Discriminative Document Identifiers for Generative Retrieval

The Audio-Visual Question Answering (AVQA) task involves extracting question-related audio-visual clues from both temporal and spatial perspectives to answer questions accurately. Despite the promising performance of existing multi-modal AVQA models, thanks to large-scale pre-trained models, challenges remain in the field. Firstly, aligning audiovisual information across temporal and spatial dimensions is
difficult. Secondly, the fusion of audio-visual information is often weighted inadequately, limiting model performance. To address these issues, we propose the Audio-Visual Adaptive Fusion Network (AVAF-Net), which uses contrastive learning to align audio-visual information temporally and spatially and adaptively adjusts fusion weights based on the question. Specifically, we initially align visual and audio information temporally through a temporal-alignment contrastive loss. This is followed by an audio-visual clue-mining module
that highlights question-related cues, aligning them with the vocal region spatially using spatial alignment contrastive loss. Additionally, a question-oriented adaptive fusion module assigns different weights to audio and visual modalities based on the question content and then fuses them. The fused audio-visual cues are finally used to predict the answer. Extensive experiments on the MUSIC-AVQA dataset show that AVAF-Net surpasses all baseline models, with a maximum improvement of 15.90% in average accuracy and an average improvement of 9.80%.

Audio-Visual Adaptive Fusion Network for Question Answering Based on Contrastive Learning

Few-shot point cloud semantic segmentation aims to accurately segment "unseen" new categories in point cloud scenes using limited labeled data. However, pretraining-based methods not only introduce excessive time overhead but also overlook the local structure representation among irregular point clouds. To address these issues, we propose a pretraining-free local structure fitting network for few-shot point cloud semantic segmentation, named TaylorSeg. Specifically, inspired by Taylor series, we treat the local structure representation of irregular point clouds as a polynomial fitting problem and propose a novel local structure fitting convolution, called TaylorConv. This convolution learns the low-order basic information and high-order refined information of point clouds from explicit encoding of local geometric structures. Then, using TaylorConv as the basic component, we construct two variant of TaylorSeg: a non-parametric TaylorSeg-NN and a parametric TaylorSeg-PN. The former can achieve performance comparable to existing parametric models without pretraining. For the latter, we equip it with an Adaptive Push-Pull (APP) module to mitigate the feature distribution differences between the query set and the support set. Extensive experiments validate the effectiveness of the proposed method. Notably, under the 2-way 1-shot setting, TaylorSeg-PN achieves improvements of +2.28% and +4.37% mIoU on the S3DIS and ScanNet datasets respectively, compared to the previous state-of-the-art methods.

Downloads

Next from AAAI 2025

Highly Imperceptible Black-Box Graph Injection Attacks with Reinforcement Learning

Similar lecture

Conditional Diffusion Based on Discrete Graph Structures for Molecular Graph Generation

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAAI 2025

Highly Imperceptible Black-Box Graph Injection Attacks with Reinforcement Learning

Similar lecture

Conditional Diffusion Based on Discrete Graph Structures for Molecular Graph Generation

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads