Singapore

In online chat platforms, multiple dialogues are often entangled in a complex manner. Disentangling them into coherent dialogues is essential for understanding conversations. Supervised models have long dominated this task. The application of large language models (LLMs) to this problem remains underexplored; however, preliminary studies have shown their performance to be substantially inferior to that of machine-learning methods. This raises fundamental questions: Do LLMs inherently lack the capability for dialogue disentanglement, and what approaches are necessary to enhance their performance? We answer these questions by introducing two novel methods: dialogue-level assignment (DLA), which tasks LLMs with assigning an utterance to a dialogue, and subsequent context (SC), which provides subsequent context as auxiliary evidence. Experiments on the benchmark IRC dataset show that our method, equipped with DLA+SC, achieves new state-of-the-art results across all metrics. These results prove that LLMs possess a strong capability for the studied task. Furthermore, our ablation study on open-source models investigates the effectiveness of DLA and SC, revealing that it is model- dependent. Nevertheless, it also demonstrates that both methods are key factors in improving performance. This research marks a paradigm shift for dialogue disentanglement, transitioning from machine-learning approaches to applying LLMs.

AAAI 2026

Rethinking Dialogue Disentanglement for LLMs via Dialogue-Level Assignment and Subsequent Context

nlp: conversational ai/dialog systems

nlp: prompt engineering / prompting

nlp: (large) language models

workshop paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.<br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

Large Language Models (LLMs) have demonstrated remarkable success in conversational systems by generating human-like responses. However, they can fall short, especially when required to account for personalization or specific knowledge. In real-life settings, it is impractical to rely on users to detect these errors and request a new response. One way to address this problem is to refine the response before returning it to the user. While existing approaches focus on refining responses within a single LLM, this method struggles to consider diverse aspects needed for effective conversations. In this work, we propose refining responses through a multi-agent framework, where each agent is assigned a specific role for each aspect. We focus on three key aspects crucial to conversational quality: factuality, personalization, and coherence. Each agent is responsible for reviewing and refining one of these aspects, and their feedback is then merged to improve the overall response. To enhance collaboration among them, we introduce a dynamic communication strategy. Instead of following a fixed sequence of agents, our approach adaptively selects and coordinates the most relevant agents based on the specific requirements of each query. We validate our framework on challenging conversational datasets, demonstrating that ours significantly outperforms relevant baselines, particularly in tasks involving knowledge or user’s persona, or both.

Adaptive Multi-Agent Response Refinement in Conversational Systems

Large Language Models (LLMs) make it easy to generate convincing phishing emails, but most research on their detection focuses on English. This leaves users of other languages, such as Japanese, more vulnerable to targeted attacks. This study first demonstrates how LLMs can be prompted to produce realistic Japanese phishing emails, creating a new dataset for testing security defenses. We then use this dataset to evaluate the performance of both multilingual and Japanese-specific transformer models on a phishing detection task. The results show that models trained specifically for Japanese achieve the highest accuracy. In contrast, multilingual models perform less effectively, showing limitations in handling Japanese text. Our findings reveal two critical points: how easily LLMs can be misused to create non-English malicious content, and the urgent need for better security models designed for languages that do not use Roman scripts.

PhishLLM: Generating Japanese Phishing Emails to Test Cross-Lingual Generalization in English-Trained Models

With the spread of the internet, the volume of reviews on services, shopping, and word-of-mouth websites has grown annually, becoming a significant source of information for decision-making. However, not all reviews are equally useful to potential buyers. A certain number of reviews are low-value or spam. Therefore, presenting only highly useful reviews to users would support more effective and efficient decision-making. In this study, we address this issue by analyzing the features of useful reviews and proposing a method to support user decision- making. First, after surveying existing research on review helpfulness classification, we propose a scoring method that focuses on the amount of information in a document (text volume) as a key feature of useful reviews. We also examined the concept of helpfulness of reviews in detail, and created a dataset containing reviews with annotations of subjectively perceived helpfulness, and objective features related to it. In a binary classification experiment using Transformers-based models to categorize review helpfulness, we obtained strong results, with an F1 score exceeding 80%. Furthermore, to show users which parts of a review are useful, we built a multi-label classification model to automatically extract the features of helpfulness. This model demonstrated its ability to effectively capture the characteristics of useful reviews, achieving an F1 score of over 80% for four of the core helpfulness-related features defined in this research.

End-to-End Assessment of Product Review Helpfulness Using Subjective and Objective Information

We present CID-GraphRAG (Conversational Intent-Driven Graph Retrieval Augmented Generation), a novel framework that addresses the limitations of existing dialogue systems in maintaining both contextual coherence and goal-oriented progression in multi-turn customer service conversations. Unlike traditional RAG systems that rely solely on semantic similarity (Conversation RAG) or standard knowledge graphs (GraphRAG), CID-GraphRAG constructs dynamic intent transition graphs from goal-achieved historical dialogues and implements a dual- retrieval mechanism that adaptively balances intent-based graph traversal with semantic search. This approach enables the system to simultaneously leverage both conversational intent flow patterns and contextual semantics, significantly improving retrieval quality and response quality. In extensive experiments on real-world customer service dialogues, we demonstrated that CID-GraphRAG significantly outperforms both semantic-based and intent-based baselines across automatic metrics, LLM-as-Judge evaluations and human evaluations, with relative gains of 11.4% in BLEU, 4.9% in ROUGE, 5.9% in METEOR, and most notably, a 58.0% improvement in response quality according to LLM-as-judge evaluations. These results demonstrate that integrating intent transition structures with semantic retrieval creates a synergistic effect that neither approach achieves independently, establishing CID-GraphRAG as an effective framework for addressing the challenges of maintaining contextual coherence and goal-oriented progression in knowledge-intensive multi-turn dialogues.

Conversational Intent-Driven GraphRAG: Enhancing Multi-Turn Dialogue Systems through Adaptive Dual-Retrieval of Flow Patterns and Context Semantics

We present iCARE, a modular multi-agent reasoning framework that integrates iCARE-Onto with Case-Based Reasoning (CBR) to support cognitively grounded, LLM-based dialogue management. The framework addresses intent disambiguation and intent routing within an agentic architecture, where specialised agents handle domain- specific tasks. Agent outputs are combined to produce accurate responses through a composite strategy that couples few-shot prompting with Retrieval-Augmented Generation (RAG) and a CBR-based fallback mechanism. Conversational content is first grounded in ontological concepts that capture situational, personal, and dialogue knowledge. These are stored as short-term memory entries and subsequently organised into semantic, procedural, and episodic summaries that form long-term memory. This structured memory ensures factual accuracy, coherence, and contextual relevance beyond surface fluency in generated responses. While the framework encompasses multiple reasoning components, in this paper we focus on how the system reasons over user utterances to form clarified intents for routing to appropriate agents. Preliminary experiments on a synthetic dataset show a 20% relative improvement in routing accuracy for the LLM-as-Router when ontology-based disambiguation is applied. Complementary evaluations with ProtoKNN demonstrate its potential as a low-latency, offline alternative, suggesting opportunities for hybrid routing in cognitively grounded dialogue systems.

iCARE: Ontology-Guided Intent Routing for Multi-Agent LLM-Based Dialogue Systems

Tool-augmented large language models (LLMs) are increas- ingly employed in real-world applications, but tool usage er- rors still hinder their reliability. We introduce ToolCritic, a di- agnostic framework that evaluates and improves LLM behav- ior in multi-turn, tool-augmented dialogues. ToolCritic de- tects eight distinct error types specific to tool-calling (e.g., premature invocation, argument misalignment, and misinter- pretation of tool outputs) and provides targeted feedback to the main LLM. The main LLM, assumed to have strong reasoning, task understanding and orchestration capabilities, then revises its response based on ToolCritic’s feedback. We systematically define these error categories and construct a synthetic dataset to train ToolCritic. Experimental results on the Schema-Guided Dialogue (SGD) dataset demonstrate that ToolCritic improves tool-calling accuracy by up to 13% over baselines, including zero-shot prompting and self-correction techniques. This represents a promising step toward more ro- bust LLM integration with external tools in real-world dia- logue applications.

ToolCritic: Detecting and Correcting Tool-Use Errors in Dialogue Systems

The use of Large Language Models (LLMs) for reasoning and planning tasks has drawn increasing attention in Artifi- cial Intelligence research. Despite their remarkable progress, these models still exhibit limitations in multi-step inference scenarios, particularly in mathematical and logical reasoning. We introduce PREGU (Partial Reasoning Guided by Uncer- tainty). PREGU monitors the entropy of the output distribu- tion during autoregressive generation and halts the process whenever entropy exceeds a defined threshold, signaling un- certainty. From that point, a localized search is performed in the latent space to refine the partial reasoning and select the most coherent answer, using the Soft Reasoning method. Experiments conducted with LLaMA-3-8B, Mistral-7B, and Qwen2-7B across four reasoning benchmarks (GSM8K, GSM-Hard, SVAMP, and StrategyQA) showed greater or similar compared to Soft Reasoning, confirming that entropy can serve as an effective signal to guide selective reasoning processes.

Partial Reasoning in Language Models Search and Refinement Guided by Uncertainty

Characterized by elevated hostility, disputes often result in disputants being unable to resolve their differences. This emphasizes the role of a mediator; however, mediators are typically highly specialized, difficult to find, and expensive. While alternatives exist, such as acquaintances or online moderators, these prove less effective. This raises the question of whether AI can effectively facilitate contentious disputes. We assess the potential of LLMs as mediators. First (Study 1), we focus on a large open-source corpus of customer service disputes that are objectively classified in terms of two key reasons for mediation: (1) whether or not the dispute ended in success or failure, and (2) to what extent the participants reported frustration with each other. We examine whether LLMs could be prompted to predict when a mediator should intervene in advance, and show that the decision to intervene is correctly sensitive to these two factors. Finally, we conducted a user study that compared AI to human suggestions on when and how to intervene. We find that the LLMs were rated significantly better in predicting when to intervene, rated as providing a better rationale for intervening, and rated as providing a more effective mediation message to the disputants. Remarkably, observers preferred the AI 2-to-1 over the human mediation decisions. Second (Study 2), we collect a small corpus of mediated dispute dialogues and analyze the effectiveness of novice human mediators. Quantifying the similarity of novice mediators’ behaviors to those of an LLM, we find the closer one mediates to an LLM, the better the outcome — both subjectively and objectively. These results indicate LLMs not only outperform novice mediators but may serve to identify effective mediators.

Can LLMs Mediate Synchronous Dispute Dialogues?

This paper introduces a novel neuro-symbolic method for irony detection that offers both high accuracy and interpretability. Our two-stage approach first uses a Transformer model to translate sentences into symbolic sequences representing their core linguistic components, such as sentiment expressions and irony targets. A machine learning classifier then uses this symbolic representation for the final classification. By explicitly modeling the internal structure of ironic statements, our method outperforms strong end-to-end baselines while providing a transparent, human-readable decision process.

A Novel Neuro-symbolic Approach to Irony Detection Based on Structural Components of Ironic Statements

The proliferation of specialized Large Language Model (LLM) dialogue systems necessitates robust defense mechanisms against unrelated prompts that introduce functional misalignment and unnecessary costs. While LLM-based filtering approaches usually work well, using them requires payments, and that motivated us to develop simple, high-speed, pre-inference input validation techniques. In this paper, we evaluate the efficacy of two semantic pre-filtering strategies applied to a Japanese export control (trade security) application domain: (1) a Centroid-based similarity guardrail (utilizing the mean vector of on-topic embeddings) and (2) a supervised Support Vector Machine (SVM) classifier. Using various multilingual embedding models, we demonstrate that the Centroid-based Similarity Guardrail exhibits superior robustness against adversarial keyword augmentation, effectively maintaining high refusal rates despite injected domain-related terms intended to shift the query’s semantic vector. Furthermore, our analysis of cross-lingual zero-shot transfer confirms that while the strongest multilingual embedding models successfully maintain topic alignment when processing English queries against a Japanese-trained filter, the efficacy is highly model-dependent, underscoring the necessity of model-specific cross-lingual validation for deployment in multilingual environments.

Downloads

Next from AAAI 2026

Adaptive Multi-Agent Response Refinement in Conversational Systems

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES