FROMANNUAL REVIEWS

EMNLP 2026

•

•

Budapest, Hungary

Automatic machine-translation (MT) metrics collapse heterogeneous errors into a single score. For Arabic, no fine-grained resource exists for diagnosing error types and severity. We introduce QUANTA, a 9,032-instance En to Ar MT-error dataset (738 real UNPC + 7,856 synthetic training instances, both structurally validated, plus a held-out 438-instance real Gold test set) annotated with Mizan (ميزان, "scale"), a new 23-class MQM-compatible taxonomy organised in a four-level hierarchy: 23 leaf classes rolling up to 14 subtypes, 5 parent groups, and 4 TQA dimensions. To make the resource usable, we release the Raqeeb (رقيب, "observer") classifier, an AraBERT-v2 model for predicting Mizan 23-class labels (5-fold Gold macro-F1 = 0.614 ± 0.017). Data quality is supported by ETCA (Error Taxonomy Cross-vendor Audit), an LLM-as-judge check in which a different-vendor model audits QUANTA instances for anchor validity and label fidelity, curbing same-family self-preference bias, and by dual-annotator validation (Gwet's AC1 = 0.97 synthetic; 0.83 real) on core taxonomy criteria. Ablations show that anchor-guided framing is decisive (without it, macro-F1 collapses to 0.114); that 738 structurally validated real samples are ≈12× more sample-efficient than synthetic data; and that, restricted to the classes with real-tier training support, combining both tiers outperforms real (0.656) or synthetic (0.312) alone. Code, data, prompts, and checkpoints are publicly available.

Downloads

Paper

Next from EMNLP 2026

On the Value of Open Research in Language Modeling - An Adversarial Dialog
keynote

On the Value of Open Research in Language Modeling - An Adversarial Dialog

EMNLP 2026

Graham Neubig
Graham Neubig

26 October 2026

Similar lecture

Scaling Low-Resource MT via Synthetic Data Generation with LLMs
poster

Scaling Low-Resource MT via Synthetic Data Generation with LLMs

EMNLP 2025

Ona de Gibert
+5
Joseph Attieh and 7 other authors

07 November 2025

Stay up to date with the latest Underline news!

Select topic of interest (you can select more than one)

PRESENTATIONS

  • All Presentations
  • For Librarians
  • Resource Center
  • Free Trial
Underline Science, Inc.
1216 Broadway, 2nd Floor, New York, NY 10001, USA

© 2026 Underline - All rights reserved

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.