Singapore

Optimizing Retrieval-Augmented Generation (RAG) configurations for specific tasks is a complex and resource-intensive challenge. Motivated by this challenge, frameworks for RAG hyper-parameter optimization (HPO) have recently emerged, yet their effectiveness has not been rigorously benchmarked. To fill this gap, we present a comprehensive study involving five HPO algorithms over five datasets from diverse domains, including a newly curated one on real-world product documentation. Our study explores the largest HPO search space considered to date, with three evaluation metrics as optimization targets. Analysis of the results shows that RAG HPO can be done efficiently, either greedily or with random search, and that it significantly boosts RAG performance for all datasets. For greedy HPO approaches, we show that optimizing model selection first is preferable to the common practice of following the RAG pipeline order during optimization.

AAAI 2026

An Analysis of Hyper-Parameter Optimization Methods for Retrieval Augmented Generation

retreival augmented generation

hyper-parameter optimization

question answering

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. 
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) 
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

Registration Required

ACL 2026

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

workshop paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

We present Fisher-LD, a layer-wise knowledge distillation framework for transformer-based recommenders. Fisher-LD uses the Fisher Information Matrix to quantify per-layer importance, allocating supervision where it matters most. On MovieLens-1M, our 6-layer student achieves HR@10=0.978 with 3.3× compression and 3.2× speedup, exceeding the 12-layer teacher (HR@10=0.934) by 4.4 percentage points. Cross-domain validation on Amazon Reviews confirms generalization. Experiments against five baselines show consistent improvements. Our three-phase protocol—Fisher analysis, selective distillation, task fine-tuning—enables effective compression for production LLM-based recommenders. Code will be released.

Fisher-LD: Layer-Wise Knowledge Distillation for LLM Recommenders

jina-reranker-v3 is a 0.6B-parameter multilingual listwise reranker that introduces a novel "last but not late" (LBNL) interaction mechanism. Unlike late interaction models like ColBERT that encode documents separately before multi-vector matching, our approach enables cross-document interactions during encoding by processing queries and all candidate documents simultaneously within shared context windows. We extract contextual embeddings from special tokens at each document's end (the "last" position), but crucially, interactions occur throughout the encoding process ("not late"). The model achieves state-of-the-art BEIR performance with 61.9 nDCG@10 while being significantly smaller than comparable alternatives.

jina-reranker-v3: Last but Not Late Interaction for Listwise Document Reranking

The rapid growth of connected devices has made federated learning (FL) essential for privacy-preserving intelligence at the edge. This, in combination with the growing demand for over-parameterized neural networks, large-scale foundation and reasoning models, creates a fundamental communication and information retrieval bottleneck. Therefore, we aim to provide a solution by alleviating the FL uplink communication bottleneck by proposing \emph{Digital MAC Federated Learning (DigiMAC-FL)}, a novel, fully digital-transmission-based aggregation framework that performs quantized model averaging directly within the multiple access channel. Each client transmits QAM-encoded model increments, enabling simultaneous uplink aggregation and exact arithmetic averaging in the complex baseband. Unlike analog over-the-air schemes, DigiMAC--FL operates within standard digital modulation while preserving the linearity required for theoretical analysis. Experiments on Fashion-MNIST and CIFAR-10 confirm that DigiMAC--FL achieves near--FedAvg accuracy with more than a tenfold reduction in communication cost. The observed Pareto-optimal point at four bits per parameter matches the analytically predicted convergence threshold, demonstrating an exact bit--accuracy trade-off. These results highlight DigiMAC-FL as a practical bridge between wireless communication and distributed reasoning systems, paving the way for scalable, communication-aware retrieval and foundation model training.

DigiMAC-FL: Towards Communication-Aware Federated Optimization for Reasoning-Enhanced Retrieval

Composed Image Retrieval (CIR) aims to find a target image that aligns with user intent, expressed through a reference image and a modification text. While Zero-shot CIR (ZS-CIR) methods sidestep the need for labeled training data by leveraging pretrained vision-language models, they often rely on a single fused query that merges all descriptive cues of what the user wants—tending to dilute key information and failing to account for what they wish to avoid. Moreover, current CIR benchmarks assume a single correct target per query, overlooking the ambiguity in modification texts. To address these challenges, we propose Soft Filtering with Textual constraints (SoFT), a training-free, plug-and-play filtering module for ZS-CIR. SoFT leverages multimodal large language models (LLMs) to extract two complementary constraints from the reference-modification pair: prescriptive (must-have) and proscriptive (must-avoid) constraints. These serve as semantic filters that reward or penalize candidate images to re-rank results, without modifying the base retrieval model or adding supervision. In addition, we construct a two-stage dataset pipeline that refines CIR benchmarks. We first identify multiple plausible targets per query to construct multi-target triplets, capturing the open-ended nature of user intent. Then guide multimodal LLMs to rewrite the modification text to focus on one target, while referencing contrastive distractors to ensure precision. This enables more comprehensive and reliable evaluation under varying ambiguity levels. Applied on top of CIReVL—a ZS-CIR retriever—SoFT raises $R@5$ to 65.25 on CIRR (+12.94), $mAP@50$ to 27.93 on CIRCO (+6.13), and $R@50$ to 58.44 on FashionIQ (+4.59), demonstrating broad effectiveness.

Soft Filtering : Guiding Zero-shot Composed Image Retrieval with Prescriptive and Proscriptive Constraints

Large Language Models (LLMs) are increasingly employed in enterprise question-answering (QA) systems, requiring adaptation to domain-specific knowledge. Among the most prevalent methods for incorporating such knowledge are Retrieval-Augmented Generation (RAG) and fine-tuning (FT). Yet, from a cost–accuracy trade-off perspective, it remains unclear which approach best suits industry scenarios. This study investigates the impact of RAG and FT across two closed, industry-specific datasets, evaluating answer quality and operational costs.We extend the Cost-of-Pass framework proposed by Erol et al. (2025) to jointly assess output quality, generation cost, and user interaction cost. Our findings reveal that while premium models perform best out of the box, open-source models can achieve comparable quality when enhanced with RAG. Overall, RAG emerges as the most effective and cost-efficient adaptation method for closed and open source.

Assessment of RAG and Fine-Tuning for Industrial Question-Answering-Applications

Multi-hop question-answering tasks pose a significant challenge for retrieval-augmented generation (RAG) models, which must effectively retrieve, infer, and reason across multiple documents to provide accurate answers. Despite the support of RAG, large language models (LLMs) often struggle to consistently retrieve the most relevant information from large collections of documents. To address this issue, we propose a novel approach, BrowseNet involving query-specific traversal within a knowledge graph (KG) for multi-hop information retrieval. This method encodes unstructured text data into a KG, where nodes represent document chunks and edges capture lexical relationships between these chunks. By dynamically traversing the knowledge graph based on the type of multi-hop query, the proposed method enhances retrieval performance by leveraging intrinsic network parameters. We evaluate this method against RAG baselines on publicly available multi-hop query datasets. Experimental results demonstrate that BrowseNet establishes a new state-of-the-art retrieval performance in multi-hop retrieval.

Structured Traversal of Knowledge Graphs for Multi-hop Question Answering

Large language models (LLMs) often produce fluent but factually incorrect statements, even when relevant evidence is available, due to misallocation of attention between contextual inputs and parametric knowledge. Ensuring that models actively reason over context and retrieve relevant information is critical for trustworthy and interpretable AI. We introduce COMPASS (Context-Modulated PID Attention Steering System), a lightweight, interpretable framework that dynamically steers attention to retrieved context during generation. Using the Context Reliance Score (CRS), COMPASS identifies which attention heads are underutilizing context, and a PID controller adjusts them in real time to improve evidence grounding and factual consistency. This mechanism enables the model to demonstrate advanced reasoning by actively returning to context and retrieving supporting information when needed, without retraining or multi-pass decoding. Across benchmarks including HotpotQA, XSum, HaluEval, and RAGTruth, COMPASS reduces hallucinations by 2.8–5.8% absolute while revealing how attention heads contribute to context-aligned reasoning. These results show that feedback-driven, interpretable control can enhance reasoning, retrieval, and evidence-based generation in LLMs.

COMPASS: Context-Modulated PID Attention Steering System for Hallucination Mitigation

Retrieval-Augmented Generation (RAG) based on Large Language Models (LLMs) is a powerful solution to understand and query the industry's closed-source documents. However, basic RAG often struggles with complex QA tasks in legal and regulatory domains, particularly when dealing with numerous government documents. The traditional top-$k$ strategy frequently misses golden chunks, leading to incomplete or inaccurate answers. To address these retrieval bottlenecks, we explore two strategies to improve evidence coverage and answer quality. The first is a One-SHOT retrieval method that adaptively selects chunks based on a token budget, allowing as much relevant content as possible to be included within the LLM's context window.Additionally, we design modules to further filter and refine the chunks. The second is an iterative retrieval strategy built on a Reasoning Agentic RAG framework, where a reasoning LLM dynamically issues search queries, evaluates retrieved results, and progressively refines the context over multiple turns. We identify query drift and retrieval laziness issues and further design two modules to tackle them. Through extensive experiments on a dataset of government documents, we aim to offer practical insights and guidance for real-world applications in legal and regulatory domains. The code and dataset will be released upon acceptance.

Fishing for Answers: Exploring One-shot vs. Iterative Retrieval Strategies for Retrieval Augmented Generation

Consumer-to-consumer (C2C) marketplaces pose distinct retrieval challenges: short, ambiguous queries; noisy, user- generated listings; and strict production constraints. This pa- per reports our experiment to build a domain-aware Japanese text-embedding approach to improve the quality of search at Mercari, Japan’s largest C2C marketplace. We experimented with fine-tuning on purchase-driven query–title pairs, using role-specific prefixes to model query–item asymmetry. To meet production constraints, we apply Matryoshka Representation Learning to obtain compact, truncation-robust embed- dings. Offline evaluation on historical search logs shows consistent gains over a strong generic encoder, with particularly large improvements when replacing PCA compression with Matryoshka truncation. A manual assessment further high- lights better handling of proper nouns, marketplace-specific semantics, and term-importance alignment. Additionally, an initial online A/B test demonstrates statistically significant improvements in revenue per user and search-flow efficiency, with transaction frequency maintained. Results show that domain-aware embeddings improve relevance and efficiency at scale and form a practical foundation for richer LLM-era search experiences.

Towards Better Search with Domain-Aware Text Embeddings for C2C Marketplaces

Large Language Models (LLMs) have shown remarkable capabilities in document processing, but their inability to provide visual grounding without OCR dependencies poses significant challenges in business-critical applications. Current solutions either require model fine-tuning or rely on external OCR services, introducing additional costs, latency, and limitations in handling derived information. This paper presents ViG-LLM, a novel framework that enables closed-box LLMs to generate localization information through a multi-agent system combining U-Net-based layout deconstruction with viewport identification tasks. Evaluated on the FATURA and CORD dataset, our framework achieves perfect accuracy over spatial reasoning tuned LLM like Amazon Nova Pro, while demonstrating superior template-specific consistency. The framework maintains robust performance across LLM architectures while reducing operational costs by 60% compared to Textract-based solutions. In real-world document processing applications, the framework helps retain the high reasoning capabilities of the system in document information extraction tasks while improving explainability, reliability and human interaction for information verification. Through human-in-the-loop learning and closed-box prompt alignment techniques, ViG-LLM provides a robust, adaptable solution for visual grounding tasks in document processing workflows.

Downloads

Next from AAAI 2026

Fisher-LD: Layer-Wise Knowledge Distillation for LLM Recommenders

Similar lecture

Efficient Hyperparameter Optimization for LLM Reinforcement Learning

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES