Singapore

Large Language Models (LLMs) make it easy to generate convincing phishing emails, but most research on their detection focuses on English. This leaves users of other languages, such as Japanese, more vulnerable to targeted attacks. This study first demonstrates how LLMs can be prompted to produce realistic Japanese phishing emails, creating a new dataset for testing security defenses. We then use this dataset to evaluate the performance of both multilingual and Japanese-specific transformer models on a phishing detection task. The results show that models trained specifically for Japanese achieve the highest accuracy. In contrast, multilingual models perform less effectively, showing limitations in handling Japanese text. Our findings reveal two critical points: how easily LLMs can be misused to create non-English malicious content, and the urgent need for better security models designed for languages that do not use Roman scripts.

AAAI 2026

PhishLLM: Generating Japanese Phishing Emails to Test Cross-Lingual Generalization in English-Trained Models

phishing detection; large language models; japanese emails; multilingual transformers; text generation; cyberse-curity; cross-lingual evaluation; llm misuse prevention.

### Welcome to 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics

Welcome to the 2025 meeting of the Nations of the Americas Chapter of the Association for Computational Linguistics! I am proud to help organize the first NAACL conference to carry the new name of our organization, one that emphasizes inclusion for all of the Americas. I am also pleased to welcome you to Albuquerque, New Mexico, a state whose unique blend of cultural influences will make for an excellent backdrop for NAACL 2025, especially with this year’s special theme on NLP in a Multicultural World. 
**[Continue reading...](https://drive.google.com/file/d/1jX-qGhqVSZZCIrAnJaz798pflu5Irrdn/view?usp=sharing)**

*- Colin Cherry, Google, NAACL 2025 General Chair* 

[![](https://assets.underline.io/markdown_image/1/image/b087f8a4dc5816d6e1a6514e59c59ac3.png)](https://drive.google.com/file/d/1T96GzPqObXrMln2BMByCSXTSizjTg69P/view?usp=sharing)

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

To access NAACL 2025 event page you are required to register. Please follow [**this link**](https://2025.naacl.org/registration/registration/) to register. Access will depend on your registration type.

Please register!

NAACL 2025

Welcome to 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics

workshop paper

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

With the spread of the internet, the volume of reviews on services, shopping, and word-of-mouth websites has grown annually, becoming a significant source of information for decision-making. However, not all reviews are equally useful to potential buyers. A certain number of reviews are low-value or spam. Therefore, presenting only highly useful reviews to users would support more effective and efficient decision-making. In this study, we address this issue by analyzing the features of useful reviews and proposing a method to support user decision- making. First, after surveying existing research on review helpfulness classification, we propose a scoring method that focuses on the amount of information in a document (text volume) as a key feature of useful reviews. We also examined the concept of helpfulness of reviews in detail, and created a dataset containing reviews with annotations of subjectively perceived helpfulness, and objective features related to it. In a binary classification experiment using Transformers-based models to categorize review helpfulness, we obtained strong results, with an F1 score exceeding 80%. Furthermore, to show users which parts of a review are useful, we built a multi-label classification model to automatically extract the features of helpfulness. This model demonstrated its ability to effectively capture the characteristics of useful reviews, achieving an F1 score of over 80% for four of the core helpfulness-related features defined in this research.

End-to-End Assessment of Product Review Helpfulness Using Subjective and Objective Information

We present CID-GraphRAG (Conversational Intent-Driven Graph Retrieval Augmented Generation), a novel framework that addresses the limitations of existing dialogue systems in maintaining both contextual coherence and goal-oriented progression in multi-turn customer service conversations. Unlike traditional RAG systems that rely solely on semantic similarity (Conversation RAG) or standard knowledge graphs (GraphRAG), CID-GraphRAG constructs dynamic intent transition graphs from goal-achieved historical dialogues and implements a dual- retrieval mechanism that adaptively balances intent-based graph traversal with semantic search. This approach enables the system to simultaneously leverage both conversational intent flow patterns and contextual semantics, significantly improving retrieval quality and response quality. In extensive experiments on real-world customer service dialogues, we demonstrated that CID-GraphRAG significantly outperforms both semantic-based and intent-based baselines across automatic metrics, LLM-as-Judge evaluations and human evaluations, with relative gains of 11.4% in BLEU, 4.9% in ROUGE, 5.9% in METEOR, and most notably, a 58.0% improvement in response quality according to LLM-as-judge evaluations. These results demonstrate that integrating intent transition structures with semantic retrieval creates a synergistic effect that neither approach achieves independently, establishing CID-GraphRAG as an effective framework for addressing the challenges of maintaining contextual coherence and goal-oriented progression in knowledge-intensive multi-turn dialogues.

Conversational Intent-Driven GraphRAG: Enhancing Multi-Turn Dialogue Systems through Adaptive Dual-Retrieval of Flow Patterns and Context Semantics

We present iCARE, a modular multi-agent reasoning framework that integrates iCARE-Onto with Case-Based Reasoning (CBR) to support cognitively grounded, LLM-based dialogue management. The framework addresses intent disambiguation and intent routing within an agentic architecture, where specialised agents handle domain- specific tasks. Agent outputs are combined to produce accurate responses through a composite strategy that couples few-shot prompting with Retrieval-Augmented Generation (RAG) and a CBR-based fallback mechanism. Conversational content is first grounded in ontological concepts that capture situational, personal, and dialogue knowledge. These are stored as short-term memory entries and subsequently organised into semantic, procedural, and episodic summaries that form long-term memory. This structured memory ensures factual accuracy, coherence, and contextual relevance beyond surface fluency in generated responses. While the framework encompasses multiple reasoning components, in this paper we focus on how the system reasons over user utterances to form clarified intents for routing to appropriate agents. Preliminary experiments on a synthetic dataset show a 20% relative improvement in routing accuracy for the LLM-as-Router when ontology-based disambiguation is applied. Complementary evaluations with ProtoKNN demonstrate its potential as a low-latency, offline alternative, suggesting opportunities for hybrid routing in cognitively grounded dialogue systems.

iCARE: Ontology-Guided Intent Routing for Multi-Agent LLM-Based Dialogue Systems

Tool-augmented large language models (LLMs) are increas- ingly employed in real-world applications, but tool usage er- rors still hinder their reliability. We introduce ToolCritic, a di- agnostic framework that evaluates and improves LLM behav- ior in multi-turn, tool-augmented dialogues. ToolCritic de- tects eight distinct error types specific to tool-calling (e.g., premature invocation, argument misalignment, and misinter- pretation of tool outputs) and provides targeted feedback to the main LLM. The main LLM, assumed to have strong reasoning, task understanding and orchestration capabilities, then revises its response based on ToolCritic’s feedback. We systematically define these error categories and construct a synthetic dataset to train ToolCritic. Experimental results on the Schema-Guided Dialogue (SGD) dataset demonstrate that ToolCritic improves tool-calling accuracy by up to 13% over baselines, including zero-shot prompting and self-correction techniques. This represents a promising step toward more ro- bust LLM integration with external tools in real-world dia- logue applications.

ToolCritic: Detecting and Correcting Tool-Use Errors in Dialogue Systems

The use of Large Language Models (LLMs) for reasoning and planning tasks has drawn increasing attention in Artifi- cial Intelligence research. Despite their remarkable progress, these models still exhibit limitations in multi-step inference scenarios, particularly in mathematical and logical reasoning. We introduce PREGU (Partial Reasoning Guided by Uncer- tainty). PREGU monitors the entropy of the output distribu- tion during autoregressive generation and halts the process whenever entropy exceeds a defined threshold, signaling un- certainty. From that point, a localized search is performed in the latent space to refine the partial reasoning and select the most coherent answer, using the Soft Reasoning method. Experiments conducted with LLaMA-3-8B, Mistral-7B, and Qwen2-7B across four reasoning benchmarks (GSM8K, GSM-Hard, SVAMP, and StrategyQA) showed greater or similar compared to Soft Reasoning, confirming that entropy can serve as an effective signal to guide selective reasoning processes.

Partial Reasoning in Language Models Search and Refinement Guided by Uncertainty

Characterized by elevated hostility, disputes often result in disputants being unable to resolve their differences. This emphasizes the role of a mediator; however, mediators are typically highly specialized, difficult to find, and expensive. While alternatives exist, such as acquaintances or online moderators, these prove less effective. This raises the question of whether AI can effectively facilitate contentious disputes. We assess the potential of LLMs as mediators. First (Study 1), we focus on a large open-source corpus of customer service disputes that are objectively classified in terms of two key reasons for mediation: (1) whether or not the dispute ended in success or failure, and (2) to what extent the participants reported frustration with each other. We examine whether LLMs could be prompted to predict when a mediator should intervene in advance, and show that the decision to intervene is correctly sensitive to these two factors. Finally, we conducted a user study that compared AI to human suggestions on when and how to intervene. We find that the LLMs were rated significantly better in predicting when to intervene, rated as providing a better rationale for intervening, and rated as providing a more effective mediation message to the disputants. Remarkably, observers preferred the AI 2-to-1 over the human mediation decisions. Second (Study 2), we collect a small corpus of mediated dispute dialogues and analyze the effectiveness of novice human mediators. Quantifying the similarity of novice mediators’ behaviors to those of an LLM, we find the closer one mediates to an LLM, the better the outcome — both subjectively and objectively. These results indicate LLMs not only outperform novice mediators but may serve to identify effective mediators.

Can LLMs Mediate Synchronous Dispute Dialogues?

This paper introduces a novel neuro-symbolic method for irony detection that offers both high accuracy and interpretability. Our two-stage approach first uses a Transformer model to translate sentences into symbolic sequences representing their core linguistic components, such as sentiment expressions and irony targets. A machine learning classifier then uses this symbolic representation for the final classification. By explicitly modeling the internal structure of ironic statements, our method outperforms strong end-to-end baselines while providing a transparent, human-readable decision process.

A Novel Neuro-symbolic Approach to Irony Detection Based on Structural Components of Ironic Statements

The proliferation of specialized Large Language Model (LLM) dialogue systems necessitates robust defense mechanisms against unrelated prompts that introduce functional misalignment and unnecessary costs. While LLM-based filtering approaches usually work well, using them requires payments, and that motivated us to develop simple, high-speed, pre-inference input validation techniques. In this paper, we evaluate the efficacy of two semantic pre-filtering strategies applied to a Japanese export control (trade security) application domain: (1) a Centroid-based similarity guardrail (utilizing the mean vector of on-topic embeddings) and (2) a supervised Support Vector Machine (SVM) classifier. Using various multilingual embedding models, we demonstrate that the Centroid-based Similarity Guardrail exhibits superior robustness against adversarial keyword augmentation, effectively maintaining high refusal rates despite injected domain-related terms intended to shift the query’s semantic vector. Furthermore, our analysis of cross-lingual zero-shot transfer confirms that while the strongest multilingual embedding models successfully maintain topic alignment when processing English queries against a Japanese-trained filter, the efficacy is highly model-dependent, underscoring the necessity of model-specific cross-lingual validation for deployment in multilingual environments.

Evaluating Lightweight Embedding Guardrails for Cost-Effective Misalignment Mitigation in Export Control Dialog Systems

In recent years, research on role-playing agents that leverage the powerful conversational capabilities of large language models (LLMs) has been actively conducted. Role-playing agents are designed to converse as real or fictional characters, reproducing dialogues that resemble those of the target persona. While many studies have explored how LLMs can generate utterances that reflect a character’s individuality, the examination of utterance generation conditioned on the target character’s emotions has not been sufficiently conducted. Because it is difficult to collect ground-truth data for the target character’s emotions, it has not yet been fully investigated whether conditioning on emotional information can improve the perceived character-likeness of generated utterances. To address this gap, we used scenario data from a visual novel game in which facial expressions, one of the key components of emotional information, are explicitly annotated. We trained an LLM to generate responses conditioned on facial expression labels by incorporating these labels into the input during fine-tuning, and evaluated the resulting utterances in terms of their character-likeness. Our experiments revealed three key findings: (1) conditioning on facial expression labels improves the perceived character-likeness of the generated utterances; (2) providing the LLM with facial expression labels that do not correspond to the character’s actual expressions can still enhance the perceived character-likeness of the generated utterances; and (3) there remains a gap between how humans and LLMs process emotional information in dialogue generation.

Evaluation of Emotion-Conditioned Response Generation for Role-Playing Agents Using Large Language Models: A Case Study with Facial Expression Labels from Visual Novel Data

Downloads

Next from AAAI 2026

End-to-End Assessment of Product Review Helpfulness Using Subjective and Objective Information

Similar lecture

Towards Reliable and Practical Phishing Detection

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES