Singapore

Tool-augmented large language models (LLMs) are increas- ingly employed in real-world applications, but tool usage er- rors still hinder their reliability. We introduce ToolCritic, a di- agnostic framework that evaluates and improves LLM behav- ior in multi-turn, tool-augmented dialogues. ToolCritic de- tects eight distinct error types specific to tool-calling (e.g., premature invocation, argument misalignment, and misinter- pretation of tool outputs) and provides targeted feedback to the main LLM. The main LLM, assumed to have strong reasoning, task understanding and orchestration capabilities, then revises its response based on ToolCritic’s feedback. We systematically define these error categories and construct a synthetic dataset to train ToolCritic. Experimental results on the Schema-Guided Dialogue (SGD) dataset demonstrate that ToolCritic improves tool-calling accuracy by up to 13% over baselines, including zero-shot prompting and self-correction techniques. This represents a promising step toward more ro- bust LLM integration with external tools in real-world dia- logue applications.

AAAI 2026

ToolCritic: Detecting and Correcting Tool-Use Errors in Dialogue Systems

verifier

tool-use

conversational

dialog

## Welcome to EMNLP 2024! 
We are excited to welcome you to one of the most prominent conferences in the field of Natural Language Processing. This year, EMNLP 2024 is being held in a hybrid format,
offering both virtual and in-person participation in beautiful Miami. Due to a record-breaking number of submissions, we've expanded the total number of accepted papers to accommodate more cutting-edge research from around the globe.
### [Conference Handbook](https://drive.google.com/file/d/1WPROgxjLAC96AJL7Ugy0tEnYm7dkrbHt/view?usp=sharing)

You are required to register for this event. **Please register [here](https://2024.emnlp.org/registration/).** The EMNLP 2024 event page on Underline will be open to public one week prior to the event.

Please register!

EMNLP 2024

EMNLP 2024 will take place in Miami, Florida from Nov 12th to Nov 16th, 2024, at the Hyatt Regency Miami Hote and on Underline for remote participants.

workshop paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The use of Large Language Models (LLMs) for reasoning and planning tasks has drawn increasing attention in Artifi- cial Intelligence research. Despite their remarkable progress, these models still exhibit limitations in multi-step inference scenarios, particularly in mathematical and logical reasoning. We introduce PREGU (Partial Reasoning Guided by Uncer- tainty). PREGU monitors the entropy of the output distribu- tion during autoregressive generation and halts the process whenever entropy exceeds a defined threshold, signaling un- certainty. From that point, a localized search is performed in the latent space to refine the partial reasoning and select the most coherent answer, using the Soft Reasoning method. Experiments conducted with LLaMA-3-8B, Mistral-7B, and Qwen2-7B across four reasoning benchmarks (GSM8K, GSM-Hard, SVAMP, and StrategyQA) showed greater or similar compared to Soft Reasoning, confirming that entropy can serve as an effective signal to guide selective reasoning processes.

Partial Reasoning in Language Models Search and Refinement Guided by Uncertainty

Characterized by elevated hostility, disputes often result in disputants being unable to resolve their differences. This emphasizes the role of a mediator; however, mediators are typically highly specialized, difficult to find, and expensive. While alternatives exist, such as acquaintances or online moderators, these prove less effective. This raises the question of whether AI can effectively facilitate contentious disputes. We assess the potential of LLMs as mediators. First (Study 1), we focus on a large open-source corpus of customer service disputes that are objectively classified in terms of two key reasons for mediation: (1) whether or not the dispute ended in success or failure, and (2) to what extent the participants reported frustration with each other. We examine whether LLMs could be prompted to predict when a mediator should intervene in advance, and show that the decision to intervene is correctly sensitive to these two factors. Finally, we conducted a user study that compared AI to human suggestions on when and how to intervene. We find that the LLMs were rated significantly better in predicting when to intervene, rated as providing a better rationale for intervening, and rated as providing a more effective mediation message to the disputants. Remarkably, observers preferred the AI 2-to-1 over the human mediation decisions. Second (Study 2), we collect a small corpus of mediated dispute dialogues and analyze the effectiveness of novice human mediators. Quantifying the similarity of novice mediators’ behaviors to those of an LLM, we find the closer one mediates to an LLM, the better the outcome — both subjectively and objectively. These results indicate LLMs not only outperform novice mediators but may serve to identify effective mediators.

Can LLMs Mediate Synchronous Dispute Dialogues?

This paper introduces a novel neuro-symbolic method for irony detection that offers both high accuracy and interpretability. Our two-stage approach first uses a Transformer model to translate sentences into symbolic sequences representing their core linguistic components, such as sentiment expressions and irony targets. A machine learning classifier then uses this symbolic representation for the final classification. By explicitly modeling the internal structure of ironic statements, our method outperforms strong end-to-end baselines while providing a transparent, human-readable decision process.

A Novel Neuro-symbolic Approach to Irony Detection Based on Structural Components of Ironic Statements

The proliferation of specialized Large Language Model (LLM) dialogue systems necessitates robust defense mechanisms against unrelated prompts that introduce functional misalignment and unnecessary costs. While LLM-based filtering approaches usually work well, using them requires payments, and that motivated us to develop simple, high-speed, pre-inference input validation techniques. In this paper, we evaluate the efficacy of two semantic pre-filtering strategies applied to a Japanese export control (trade security) application domain: (1) a Centroid-based similarity guardrail (utilizing the mean vector of on-topic embeddings) and (2) a supervised Support Vector Machine (SVM) classifier. Using various multilingual embedding models, we demonstrate that the Centroid-based Similarity Guardrail exhibits superior robustness against adversarial keyword augmentation, effectively maintaining high refusal rates despite injected domain-related terms intended to shift the query’s semantic vector. Furthermore, our analysis of cross-lingual zero-shot transfer confirms that while the strongest multilingual embedding models successfully maintain topic alignment when processing English queries against a Japanese-trained filter, the efficacy is highly model-dependent, underscoring the necessity of model-specific cross-lingual validation for deployment in multilingual environments.

Evaluating Lightweight Embedding Guardrails for Cost-Effective Misalignment Mitigation in Export Control Dialog Systems

In recent years, research on role-playing agents that leverage the powerful conversational capabilities of large language models (LLMs) has been actively conducted. Role-playing agents are designed to converse as real or fictional characters, reproducing dialogues that resemble those of the target persona. While many studies have explored how LLMs can generate utterances that reflect a character’s individuality, the examination of utterance generation conditioned on the target character’s emotions has not been sufficiently conducted. Because it is difficult to collect ground-truth data for the target character’s emotions, it has not yet been fully investigated whether conditioning on emotional information can improve the perceived character-likeness of generated utterances. To address this gap, we used scenario data from a visual novel game in which facial expressions, one of the key components of emotional information, are explicitly annotated. We trained an LLM to generate responses conditioned on facial expression labels by incorporating these labels into the input during fine-tuning, and evaluated the resulting utterances in terms of their character-likeness. Our experiments revealed three key findings: (1) conditioning on facial expression labels improves the perceived character-likeness of the generated utterances; (2) providing the LLM with facial expression labels that do not correspond to the character’s actual expressions can still enhance the perceived character-likeness of the generated utterances; and (3) there remains a gap between how humans and LLMs process emotional information in dialogue generation.

Evaluation of Emotion-Conditioned Response Generation for Role-Playing Agents Using Large Language Models: A Case Study with Facial Expression Labels from Visual Novel Data

Content not yet available

Next from AAAI 2026

Partial Reasoning in Language Models Search and Refinement Guided by Uncertainty

Similar lecture

Enhancing Tool Retrieval with Iterative Feedback from Large Language Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES