United States

This paper describes our first-place submission to Task B (generation with reference passages) of the SemEval-2026 Task 8 MTRAGEval shared task on multi-turn retrieval-augmented generation. We propose a heterogeneous ensemble of seven LLMs organised into two groups with distinct prompting strategies, and use a GPT-4o-mini judge to select the best candidate response for each instance. Our system ranked first among 26 teams, achieving a conditioned harmonic mean score of 0.78 and substantially outperforming the strongest 
organiser baseline (0.64). Ablation experiments show that diversity across model families, scales, and prompting strategies is critical: the ensemble consistently outperforms any individual model. We also include Meno-Lite-0.1, a 7B domain-adapted model with a favourable cost--performance trade-off, and present an analysis of MTRAGEval that highlights annotation limitations and directions for benchmark improvement.

ACL 2026

RaguTeam at SemEval-2026 Task 8: Meno and Friends in a Judge-Orchestrated LLM Ensemble for Faithful Multi-Turn Response Generation

domain-adapted llm

muti-turn dialogue

llm ensemble

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. <br>
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) <br><br><br>
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>
<br><br><br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

**Online Registration Form**: https://acl.swoogo.com/acl2026

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

workshop paper

Large language models suffer from content effects in reasoning tasks, particularly in multilingual contexts. We introduce a novel method that reduces these biases through explicit structural abstraction that transforms syllogisms into canonical logical representations and applies deterministic parsing to determine validity. Evaluated on the SemEval-2026 Task 11 multilingual benchmark, our approach achieves top-5 rankings across all subtasks while substantially reducing content effects and offering a competitive alternative to complex fine-tuning or activation-level interventions.

ITLC at SemEval-2026 Task 11: Normalization and Deterministic Parsing for Formal Reasoning in LLMs

This paper presents an uncertainty-aware adversarial learning framework developed for SemEval-2026 Task 3, a shared task focusing on Dimensional Aspect-Based Sentiment Analysis (ABSA). Our framework involves three key components: Uncertainty modeling, Heterogeneous Mixture-of-Experts (HMoE) architecture, and embedding-level adversarial training. Experimental results demonstrate that our framework effectively reduces the Root Mean Square Error (RMSE), thereby validating the synergistic advantages of uncertainty modeling and heterogeneous fusion strategies in fine-grained sentiment regression tasks.

Scmhl5 at SemEval-2026 Task 3: Uncertainty-Aware Adversarial Learning for Embedding Enhancement in Dimensional Aspect-Based Sentiment Analysis

The Dimensional ABSA (DimABSA) shared
task extends traditional aspect-based sentiment
analysis from categorical polarity to continuous
valence–arousal (VA) prediction. We present
our system for all three subtasks: Dimensional
Aspect Sentiment Regression (DimASR),
Dimensional Aspect Sentiment Triplet Extrac-
tion (DimASTE), and Dimensional Aspect
Sentiment Quad Prediction (DimASQP).
Due to the cascading nature of the different
subtasks, we built a modular interlocking
pipeline that uses classical Machine Learning
and NLP methods.
Experiments across domains show consistent
gains in regression accuracy and structured
extraction performance. Our results demon-
strate the effectiveness of distribution-aware
regression, retrieval-augmented generation, and
contrastive prototype learning for dimensional
sentiment analysis.

SokraTUM at SemEval-2026 Task 3: A hybrid cascade of Label Distribution Learning, RAG supported generative extraction and contrastive metric learning for dimensional sentiment analysis

This paper presents our winning system for SemEval-2026 Task 5 on rating the plausibility of word senses in ambiguous stories. Unlike traditional Word Sense Disambiguation, the task requires predicting continuous plausibility scores that reflect human variability rather than selecting a single correct sense. We propose a multi-target fine-tuning framework for decoder-only large language models that jointly optimizes regression for score prediction and text generation for interpretable explanations. To address inter-annotator variability, we adopt objective-level strategies to enhance robustness. Our system achieves first place, demonstrating the effectiveness of unified regressive–generative modeling for fine-grained plausibility estimation.

SRCB at SemEval-2026 Task 5 A Multi-Target Finetuning Framework for Large Language Models with Joint Regression and Text Generation

We propose a CodeBERT-based system for detecting AI-generated code under severe cross-language and cross-domain distribution shift. Our approach conducts a controlled comparison between a fully frozen backbone and a partially fine-tuned configuration that unfreezes only the final transformer layer with discriminative learning rates. While partial fine-tuning substantially improves in-domain performance, the frozen backbone demonstrates stronger robustness under out-of-distribution evaluation. Our results highlight a trade-off between task adaptation and cross-language generalization in machine-generated code detection.

Team_Omega at SemEval-2026 Task 13: Frozen vs. Trainable Representations for Out-of-Distribution AI-Generated Code Detection: A CodeBERT Fine-Tuning Study

The Narrative Story Similarity and Narrative Representation Learning (NSNRL) task measures the narrative similarity between two stories based on three core aspects: the abstract theme, the course of action, and the outcomes. Our system leverages LLMs both for extracting high-level aspects and to encode them with state-of-the-art generalist embedding models. We then apply a series of embedding post-processing steps and learn to fit the embedding space with a Mahalanobis-like diagonal metric. We show that some of these techniques should not be applied universally, as they do not necessarily increase performance or overfit, depending on the base encoder. Our system outperforms the baseline only in Track B, ranking twelfth out of twenty-seven on the final leaderboard, while performing lower than the baseline accuracy in Track A.

Team UBSE at SemEval-2026 Task 4: Adapting Generalist Embeddings for Narrative Representations

This paper presents a neuro-symbolic ensemble for determining narrative similarity by moving beyond surface-level text matching toward structural and causal alignment. The architecture fuses three primary signals: action-focused neural embeddings that isolate event trajectories , a symbolic Structural Survival Ratio (SSR) that measures the preservation of discrete event tuples via dependency parsing , and high-level structural comparisons conducted by the gpt-5-mini model. Evaluated on the SemEval-2026 Task 4 test set, the integrated ensemble achieved an accuracy of 68.25%.

harapalb at SemEval-2026 Task 4: Multi-Signal Neuro-Symbolic Ensembles for Narrative Similarity

We describe the Seals-NLP system for SemEval-2026 Task 9 (POLAR) Subtask 1, binary polarization detection. Our study compares (i) fully fine-tuned encoder-only transformers, (ii) QLoRA-based fine-tuned open-weight LLMs, and (iii) zero-shot prompted LLMs. ModernBERT-large emerges as the most cost-effective option, matching or surpassing larger fine-tuned and zero-shot LLMs in macro-F1 while requiring substantially less memory and lower latency. An error analysis by failure mode and polarization subtype reveals systematic over-triggering on political cue words and under-detection of sarcastic vilification and multifaceted attacks in the POLAR dataset across all models.

Seals-NLP at SemEval-2026 Task 9: A Comparative Study of Transformer Architectures for Polarization Detection

This paper describes our submission to SemEval-2026 Task 8 on multi-turn retrieval-augmented generation (RAG). We propose a hybrid multi-stage pipeline that combines high-recall lexical retrieval, dual-embedding dense re-ranking with reciprocal rank fusion, LLM-based relevance judging, and strictly constrained evidence-grounded generation. Our design emphasizes robustness and faithfulness across the full retrieval-to-generation pipeline. Our results suggest that relevance-aware filtering and constrained generation are important for improving faithfulness and overall RAG performance.

GenAIus at SemEval-2026 Task 8: Beyond Retrieval with Relevance-Aware RAG for Faithful Multi-Turn Generation

We first use Qwen2.5-32B-Instruct model to generate hard negatives from three
narrative dimensions. We then train a Qwen3-Embedding-8B model with a multi-negative
contrastive objective and use self-distllation.

Downloads

Next from ACL 2026

ITLC at SemEval-2026 Task 11: Normalization and Deterministic Parsing for Formal Reasoning in LLMs

Similar lecture

SemEval-2026 Task 8: MTRAGEval: Evaluating Multi-Turn RAG Conversations

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2026

ITLC at SemEval-2026 Task 11: Normalization and Deterministic Parsing for Formal Reasoning in LLMs

Similar lecture

SemEval-2026 Task 8: MTRAGEval: Evaluating Multi-Turn RAG Conversations

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads