Lecture image placeholder

Premium content

Access to this content requires a subscription. You must be a premium user to view this content.

Monthly subscription - $9.99Pay per view - $4.99Access through your institutionLogin with Underline account
Need help?
Contact us
Lecture placeholder background
VIDEO DOI: https://doi.org/10.48448/fj96-qt02

poster

ACL 2024

August 13, 2024

Bangkok, Thailand

Fine-Tuned Machine Translation Metrics Struggle in Unseen Domains

keywords:

domain bias

evaluation metrics

evaluation

machine translation

We introduce a new, extensive multidimensional quality metrics (MQM) annotated dataset covering 11 language pairs in the biomedical domain. We use this dataset to investigate whether machine translation (MT) metrics which are fine-tuned on human-generated MT quality judgements are robust to domain shifts between training and inference. We find that fine-tuned metrics exhibit a substantial performance drop in the unseen domain scenario relative to both metrics that rely on the surface form and pre-trained metrics that are not fine-tuned on MT quality judgments.

Downloads

SlidesTranscript English (automatic)

Next from ACL 2024

Hyperspherical Multi-Prototype with Optimal Transport for Event Argument Extraction
poster

Hyperspherical Multi-Prototype with Optimal Transport for Event Argument Extraction

ACL 2024

+3Jiye LiangRu Li
Guangjun Zhang and 5 other authors

13 August 2024

Stay up to date with the latest Underline news!

Select topic of interest (you can select more than one)

PRESENTATIONS

  • All Lectures
  • For Librarians
  • Resource Center
  • Free Trial
Underline Science, Inc.
1216 Broadway, 2nd Floor, New York, NY 10001, USA

© 2023 Underline - All rights reserved