Singapore

Retrieval-Augmented Generation (RAG) has facilitated the construction of domain-specific question–answering (Q&amp;A) systems by integrating information retrieval (IR) mechanisms with large language models (LLMs). However, real-world RAG systems frequently suffer from domain drift In automotive voice assistants, novice users unfamiliar with technical terminology often paraphrase such terms, which prevents dense retrievers from bridging the semantic gap and consequently causes retrieval failures. The rapid proliferation of LLMs has expanded their user base, increasing the demand for non-expert users to access domain-specific knowledge in specialized fields such as healthcare and law, thereby underscoring the significance of this issue. This study investigates a practical approach to addressing the challenge of understanding paraphrastic expressions of technical terminology, using a dataset of over 10,000 Q&amp;A pairs derived from vehicle manuals, and proposes continual pre-training of embedding models as an effective solution. Experimental results demonstrate that continual pre-training effectively imbues embedding models with domain knowledge, enabling them to associate paraphrased expressions with their corresponding technical terms. Although continual pre-training has been extensively studied for language models, its application to retrieval embedding models within RAG systems remains largely underexplored. Our findings offer insights into designing retrieval systems that achieve both domain fidelity and linguistic generalization—two essential prerequisites for scaling RAG to real-world applications.

AAAI 2026

Continual Pre-training of Dense Retrievers for Bridging Paraphrases and Technical Terms in Domain-Specific Retrieval-Augmented Generation

automotive voice assistants

information retrieval

## Welcome to 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics!
This year, the conference is in Mexico City. NAACL was actually already planned for Mexico City in 2021, but due to the pandemic the entire conference was moved online. This year, finally, we get to go! So it is my sincere pleasure to welcome you to Mexico City, whether in person or virtually. Having the conference in Mexico City is a good opportunity to emphasize that NAACL is our flagship conference for ACL members not only in North America but also in Central and South America, even though NAACL has been bearing “North” in its name. At this year’s conference, we have a theme to match, with a theme track on the Languages of Latin America to showcase the linguistic diversity of the region.
 
The opportunity to present at NAACL should not depend on a researcher’s travel budget, or their family status. This is why it is so important to make virtual participation at NAACL as good an experience as possible – but we want to also provide a good experience for in-person participants. As a community, we are still working out the best way to do that. This year at NAACL, we are trying out a big virtual poster session ahead of the conference, with the hope that this will make make for a lively and interactive experience. At the same time, we are reducing virtual oral presentations, which seem to be particularly tricky to make to work well. A big thanks to the NAACL program chairs and to Luciana Benotti for all their ideas and work to improve the virtual experience. And participants, virtual as well as in-person: Please let us know what worked for you and what didn’t, so we can continue to improve hybrid conferences. 
 
I have been lucky to work with many amazing people. Without their insight, dedication and patience, and without the many hours of work they put in, NAACL would not have been possible. A huge thank you to the program chairs Helena Gomez, Kevin Duh, and Steve Bethard – you are the best! 
 
Finally, I would like to thank all authors, invited speakers and panelists, area chairs and reviewers, the volunteers organizing and chairing sessions, and all attendees, in-person and virtual. Thank you for helping us make NAACL 2024 come to life. 
 
Welcome and hope you all enjoy the conference! 
Katrin Erk 
The University of Texas at Austin 
NAACL-2024 General Chair 
*You can read the full Welcome message in the [Conference Handbook (downloadable)](https://drive.google.com/file/d/1H1NvW0VASQjkSCYgw3mr-3l4yYDSxz6B/view?usp=sharing)*

To access the event page you need to register [**here.**](http://acl.swoogo.com/naacl2024) 
Your access to the event page is limited based on your registration type. If you registered for workshops only, you will gain access to full workshops content on the day of the workshops program.

Please register!

NAACL 2024

2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics

Retrieval-Augmented Generation (RAG) has facilitated the construction of domain-specific question–answering (Q&A) systems by integrating information retrieval (IR) mechanisms with large language models (LLMs). However, real-world RAG systems frequently suffer from domain drift In automotive voice assistants, novice users unfamiliar with technical terminology often paraphrase such terms, which prevents dense retrievers from bridging the semantic gap and consequently causes retrieval failures. The rapid proliferation of LLMs has expanded their user base, increasing the demand for non-expert users to access domain-specific knowledge in specialized fields such as healthcare and law, thereby underscoring the significance of this issue. This study investigates a practical approach to addressing the challenge of understanding paraphrastic expressions of technical terminology, using a dataset of over 10,000 Q&A pairs derived from vehicle manuals, and proposes continual pre-training of embedding models as an effective solution. Experimental results demonstrate that continual pre-training effectively imbues embedding models with domain knowledge, enabling them to associate paraphrased expressions with their corresponding technical terms. Although continual pre-training has been extensively studied for language models, its application to retrieval embedding models within RAG systems remains largely underexplored. Our findings offer insights into designing retrieval systems that achieve both domain fidelity and linguistic generalization—two essential prerequisites for scaling RAG to real-world applications.

workshop paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

Locating and retrieving objects from scene-level point clouds is a challenging problem with broad applications in robotics and augmented reality. This task is commonly formulated as open-vocabulary 3D instance segmentation. Although recent methods demonstrate strong performance, they depend heavily on SAM and CLIP to generate and classify 3D instance masks from images accompanying the point cloud, leading to substantial computational overhead and slow processing that limit their deployment in real-world settings. Open-YOLO 3D alleviates this issue by using a real-time 2D detector to classify class-agnostic masks produced directly from the point cloud by a pretrained 3D segmenter, eliminating the need for SAM and CLIP and significantly reducing inference time. However, Open-YOLO 3D often fails to generalize to object categories that appear infrequently in the 3D training data. In this paper, we propose a method that generates 3D instance masks for novel objects from RGB images guided by a 2D open-vocabulary detector. Our approach inherits the 2D detector’s ability to recognize novel objects while maintaining efficient classification, enabling fast and accurate retrieval of rare instances from open-ended text queries.

Retrieving Objects from 3D Scenes with Box-Guided Open-Vocabulary Instance Segmentation

The increasing diversity and scale of video data demand retrieval systems capable of multimodal understanding, adaptive reasoning, and domain-specific knowledge integration. This paper presents LLandMark, a modular multi-agent framework for landmark-aware multimodal video retrieval to handle real-world complex queries. The framework features specialized agents that collaborate across four stages: query parsing and planning, landmark reasoning, multimodal retrieval, and reranked answer synthesis. A key component, the Landmark Knowledge Agent, detects cultural or spatial landmarks and reformulates them into descriptive visual prompts, enhancing CLIP-based semantic matching for Vietnamese scenes. To expand capabilities, we introduce an LLM-assisted image-to-image pipeline, where a large language model (Gemini 2.5 Flash) autonomously detects landmarks, generates image search queries, retrieves representative images, and performs CLIP-based visual similarity matching, removing the need for manual image input. In addition, an OCR refinement module leveraging Gemini and LlamaIndex improves Vietnamese text recognition. Experimental results show that LLandMark achieves adaptive, culturally grounded, and explainable retrieval performance. Our code will be released soon.

LLandMark: A Multi-Agent Framework for Landmark-Aware Multimodal Interactive Video Retrieval

Modern generative pre-trained language models excel at open-ended text generation, yet continue to underperform on structure-related tasks such as NER, relation extraction, and semantic role labeling, especially when compared to encoder-only models of similar sizes. While this gap has been attributed to limited structure knowledge, we hypothesize this is also due to the missing connection between the model’s internal representations of linguistic structure and the output space used during supervised fine-tuning. We propose the Structured Language Generation Model (SLGM), a model- and task-agnostic framework that reformulates structured prediction as a classification problem through three components: (1) reinforced input formatting with structural cues, (2) loss design, and (3) format-aware decoding that constrains generation to task-valid outputs. Across 5 tasks and 13 datasets, SLGM substantially improves structure prediction without relying on dataset-specific engineering or additional model parameters. It outperforms baseline fine-tuning on models of the same size, achieves comparable performance to much larger models when used with $<$1B parameter models, and acts as a zero-weight adapter that reproduces the benefits of dataset-specific fine-tuning in low-resource settings.

Structured Language Generation Model: Loss Calibration and Formatted Decoding for Robust Structure Prediction

Optimizing Retrieval-Augmented Generation (RAG) configurations for specific tasks is a complex and resource-intensive challenge. Motivated by this challenge, frameworks for RAG hyper-parameter optimization (HPO) have recently emerged, yet their effectiveness has not been rigorously benchmarked. To fill this gap, we present a comprehensive study involving five HPO algorithms over five datasets from diverse domains, including a newly curated one on real-world product documentation. Our study explores the largest HPO search space considered to date, with three evaluation metrics as optimization targets. Analysis of the results shows that RAG HPO can be done efficiently, either greedily or with random search, and that it significantly boosts RAG performance for all datasets. For greedy HPO approaches, we show that optimizing model selection first is preferable to the common practice of following the RAG pipeline order during optimization.

An Analysis of Hyper-Parameter Optimization Methods for Retrieval Augmented Generation

We present Fisher-LD, a layer-wise knowledge distillation framework for transformer-based recommenders. Fisher-LD uses the Fisher Information Matrix to quantify per-layer importance, allocating supervision where it matters most. On MovieLens-1M, our 6-layer student achieves HR@10=0.978 with 3.3× compression and 3.2× speedup, exceeding the 12-layer teacher (HR@10=0.934) by 4.4 percentage points. Cross-domain validation on Amazon Reviews confirms generalization. Experiments against five baselines show consistent improvements. Our three-phase protocol—Fisher analysis, selective distillation, task fine-tuning—enables effective compression for production LLM-based recommenders. Code will be released.

Fisher-LD: Layer-Wise Knowledge Distillation for LLM Recommenders

jina-reranker-v3 is a 0.6B-parameter multilingual listwise reranker that introduces a novel "last but not late" (LBNL) interaction mechanism. Unlike late interaction models like ColBERT that encode documents separately before multi-vector matching, our approach enables cross-document interactions during encoding by processing queries and all candidate documents simultaneously within shared context windows. We extract contextual embeddings from special tokens at each document's end (the "last" position), but crucially, interactions occur throughout the encoding process ("not late"). The model achieves state-of-the-art BEIR performance with 61.9 nDCG@10 while being significantly smaller than comparable alternatives.

jina-reranker-v3: Last but Not Late Interaction for Listwise Document Reranking

The rapid growth of connected devices has made federated learning (FL) essential for privacy-preserving intelligence at the edge. This, in combination with the growing demand for over-parameterized neural networks, large-scale foundation and reasoning models, creates a fundamental communication and information retrieval bottleneck. Therefore, we aim to provide a solution by alleviating the FL uplink communication bottleneck by proposing \emph{Digital MAC Federated Learning (DigiMAC-FL)}, a novel, fully digital-transmission-based aggregation framework that performs quantized model averaging directly within the multiple access channel. Each client transmits QAM-encoded model increments, enabling simultaneous uplink aggregation and exact arithmetic averaging in the complex baseband. Unlike analog over-the-air schemes, DigiMAC--FL operates within standard digital modulation while preserving the linearity required for theoretical analysis. Experiments on Fashion-MNIST and CIFAR-10 confirm that DigiMAC--FL achieves near--FedAvg accuracy with more than a tenfold reduction in communication cost. The observed Pareto-optimal point at four bits per parameter matches the analytically predicted convergence threshold, demonstrating an exact bit--accuracy trade-off. These results highlight DigiMAC-FL as a practical bridge between wireless communication and distributed reasoning systems, paving the way for scalable, communication-aware retrieval and foundation model training.

DigiMAC-FL: Towards Communication-Aware Federated Optimization for Reasoning-Enhanced Retrieval

Composed Image Retrieval (CIR) aims to find a target image that aligns with user intent, expressed through a reference image and a modification text. While Zero-shot CIR (ZS-CIR) methods sidestep the need for labeled training data by leveraging pretrained vision-language models, they often rely on a single fused query that merges all descriptive cues of what the user wants—tending to dilute key information and failing to account for what they wish to avoid. Moreover, current CIR benchmarks assume a single correct target per query, overlooking the ambiguity in modification texts. To address these challenges, we propose Soft Filtering with Textual constraints (SoFT), a training-free, plug-and-play filtering module for ZS-CIR. SoFT leverages multimodal large language models (LLMs) to extract two complementary constraints from the reference-modification pair: prescriptive (must-have) and proscriptive (must-avoid) constraints. These serve as semantic filters that reward or penalize candidate images to re-rank results, without modifying the base retrieval model or adding supervision. In addition, we construct a two-stage dataset pipeline that refines CIR benchmarks. We first identify multiple plausible targets per query to construct multi-target triplets, capturing the open-ended nature of user intent. Then guide multimodal LLMs to rewrite the modification text to focus on one target, while referencing contrastive distractors to ensure precision. This enables more comprehensive and reliable evaluation under varying ambiguity levels. Applied on top of CIReVL—a ZS-CIR retriever—SoFT raises $R@5$ to 65.25 on CIRR (+12.94), $mAP@50$ to 27.93 on CIRCO (+6.13), and $R@50$ to 58.44 on FashionIQ (+4.59), demonstrating broad effectiveness.

Soft Filtering : Guiding Zero-shot Composed Image Retrieval with Prescriptive and Proscriptive Constraints

Large Language Models (LLMs) are increasingly employed in enterprise question-answering (QA) systems, requiring adaptation to domain-specific knowledge. Among the most prevalent methods for incorporating such knowledge are Retrieval-Augmented Generation (RAG) and fine-tuning (FT). Yet, from a cost–accuracy trade-off perspective, it remains unclear which approach best suits industry scenarios. This study investigates the impact of RAG and FT across two closed, industry-specific datasets, evaluating answer quality and operational costs.We extend the Cost-of-Pass framework proposed by Erol et al. (2025) to jointly assess output quality, generation cost, and user interaction cost. Our findings reveal that while premium models perform best out of the box, open-source models can achieve comparable quality when enhanced with RAG. Overall, RAG emerges as the most effective and cost-efficient adaptation method for closed and open source.

Assessment of RAG and Fine-Tuning for Industrial Question-Answering-Applications

Multi-hop question-answering tasks pose a significant challenge for retrieval-augmented generation (RAG) models, which must effectively retrieve, infer, and reason across multiple documents to provide accurate answers. Despite the support of RAG, large language models (LLMs) often struggle to consistently retrieve the most relevant information from large collections of documents. To address this issue, we propose a novel approach, BrowseNet involving query-specific traversal within a knowledge graph (KG) for multi-hop information retrieval. This method encodes unstructured text data into a KG, where nodes represent document chunks and edges capture lexical relationships between these chunks. By dynamically traversing the knowledge graph based on the type of multi-hop query, the proposed method enhances retrieval performance by leveraging intrinsic network parameters. We evaluate this method against RAG baselines on publicly available multi-hop query datasets. Experimental results demonstrate that BrowseNet establishes a new state-of-the-art retrieval performance in multi-hop retrieval.

Next from AAAI 2026

Retrieving Objects from 3D Scenes with Box-Guided Open-Vocabulary Instance Segmentation

Similar lecture

Optimizing LLM Based Retrieval Augmented Generation Pipelines in the Financial Domain

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES