Dominican Republic

This work introduces a simple regressive ensemble for evaluating machine translation quality based on a set of novel and established metrics. 
We evaluate the ensemble using a correlation to expert-based MQM scores of the WMT 2021 Metrics workshop.
In both monolingual and zero-shot cross-lingual settings, we show a significant performance improvement over single metrics.
In the cross-lingual settings, we also demonstrate that an ensemble approach is well-applicable to unseen languages.
Furthermore, we identify a strong reference-free baseline that consistently outperforms the commonly-used BLEU and METEOR measures and significantly improves our ensemble&#39;s performance.

EMNLP 2021

Regressive Ensemble for Machine Translation Quality Evaluation

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible. The experience for on-site participants would closely approximate a normal pre-COVID *ACL conference, with 5-6 thematically organized parallel sessions and live Q/A and interactive discussion immediately after the talks. Presentations by virtual participants will be equitably interleaved with those of on-site participants, projected on the auditorium screens as if on-site, and also followed immediately by live Q/A and interactive discussion at a time during reasonable waking hours for the virtual presenter. For all participants, on-site and virtual, who are unable to attend a session due to either time-zone issues or because they are participating in another session live, talk recordings and slides will be available online at a minimum after the live presentation (and in many cases before as well), and questions may be submitted in advance on session-specific discussion boards and answered live in session with the usual visual aids if desired.

<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/ceby/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>

Please Note: The EMNLP registration system is not currently connected to the underline site as we are still in the process of building out EMNLP 2021. You will receive access instructions from underline the week of November 1st. 

Access is given only to EMNLP upon registration, if you have not registered please do so [here](https://2021.emnlp.org/registration).

Registered attendees will receive access the week of November 1st.

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible.

This work introduces a simple regressive ensemble for evaluating machine translation quality based on a set of novel and established metrics. 
We evaluate the ensemble using a correlation to expert-based MQM scores of the WMT 2021 Metrics workshop.
In both monolingual and zero-shot cross-lingual settings, we show a significant performance improvement over single metrics.
In the cross-lingual settings, we also demonstrate that an ensemble approach is well-applicable to unseen languages.
Furthermore, we identify a strong reference-free baseline that consistently outperforms the commonly-used BLEU and METEOR measures and significantly improves our ensemble's performance.

workshop paper

Argument pair extraction (APE) aims to extract interactive argument pairs from two passages of a discussion. Previous work studied this task in the context of peer review and rebuttal, and decomposed it into a sequence labeling task and a sentence relation classiﬁcation task. However, despite the promising performance, such an approach obtains the argument pairs implicitly by the two decomposed tasks, lacking explicitly modeling of the argument-level interactions between argument pairs. In this paper, we tackle the APE task by a mutual guidance framework, which could utilize the information of an argument in one passage to guide the identiﬁcation of arguments that can form pairs with it in another passage. In this manner, two passages can mutually guide each other in the process of APE. Furthermore, we propose an inter-sentence relation graph to effectively model the interrelations between two sentences and thus facilitates the extraction of argument pairs. Our proposed method can better represent the holistic argument-level semantics and thus explicitly capture the complex correlations between argument pairs. Experimental results show that our approach signiﬁcantly outperforms the current state-of-the-art model.

Argument Pair Extraction with Mutual Guidance and Inter-sentence Relation Graph

Opinion Role Labeling (ORL), aiming to identify the key roles of opinion, has received increasing interest. Unlike most of the previous works focusing on the English language, in this paper, we present the first work of Chinese ORL. We construct a Chinese dataset by manually translating and projecting annotations from a standard English MPQA dataset. Then, we investigate the effectiveness of cross-lingual transfer methods, including model transfer and corpus translation. We exploit multilingual BERT with Contextual Parameter Generator and Adapter methods to examine the potentials of unsupervised cross-lingual learning and our experiments and analyses for both bilingual and multilingual transfers establish a foundation for the future research of this task.

Chinese Opinion Role Labeling with Corpus Translation: A Pivot Study

Aspect-based sentiment analysis (ABSA) predicts the sentiment polarity towards a particular aspect term in an input sentence, which is significantly important in real-world applications. To perform ABSA, one faces a long-standing challenge that ABSA requires the model to have a good understanding of the contextual information, especially the particular patterns that suggests the sentiment polarity. However, these patterns vary in different sentences, especially when the sentences come from different sources (domains). Although leveraging extra labeled data from different sources (domains) is always a promising solution to address the challenge, in practical applications, these labeled data are always stored at different locations and might be inaccessible to each other due to privacy or legal concerns (e.g., the data are owned by different companies). To address the issue and benefit from extra labeled data, we propose a novel ABSA model with federated learning (FL) adopted to overcome the data isolation limitations and topic memory (TM) proposed to enhance FL by accommodating the cases of data from diverse sources (domains). Particularly, TM contributes to overcoming difficulties in identifying domains due to data inaccessibility by providing useful categorical information for localized predictions. Experimental results on a simulated environment for FL with three nodes demonstrate the effectiveness of our approach, where TM-FL outperforms different baselines including some well-designed FL frameworks.

Improving Federated Learning for Aspect-based Sentiment Analysis via Topic Memories

Aspect term extraction aims to extract aspect terms from a review sentence that users have expressed opinions on. One of the remaining challenges for aspect term extraction resides in the lack of sufficient annotated data. While self-training is potentially an effective method to address this issue, the pseudo-labels it yields on unlabeled data could induce noise. In this paper, we use two means to alleviate the noise in the pseudo-labels. One is that inspired by the curriculum learning, we refine the conventional self-training to progressive self-training. Specifically, the base model infers pseudo-labels on a progressive subset at each iteration, where samples in the subset become harder and more numerous as the iteration proceeds. The other is that we use a discriminator to filter the noisy pseudo-labels. Experimental results on four SemEval datasets show that our model significantly outperforms the previous baselines and achieves state-of-the-art performance.

Progressive Self-Training with Discriminator for Aspect Term Extraction

This paper describes the SEBAMAT contribution to the 2021 WMT Similar Language Translation shared task. Using the Marian neural machine translation toolkit, translation systems based on Google’s transformer architecture were built in both directions of Catalan–Spanish and Portuguese–Spanish. The systems were trained in two contrastive parameter settings (different vocabulary sizes for byte pair encoding) using only the parallel but not the comparable corpora provided by the shared task organizers. According to their official evaluation results, the SEBAMAT system turned out to be competitive with rankings among the top teams and BLEU scores between 38 and 47 for the language pairs involving Portuguese and between 76 and 80 for the language pairs involving Catalan.

Similar Language Translation for Catalan, Portuguese and Spanish Using Marian NMT

In this paper, we present the joint contribution of Unbabel and IST to the WMT 2021 Metrics Shared Task. With this year's focus on Multidimensional Quality Metric (MQM) as the ground-truth human assessment, our aim was to steer COMET towards higher correlations with MQM. We do so by first pre-training on Direct Assessments and then fine-tuning on z-normalized MQM scores. In our experiments we also show that reference-free COMET models are becoming competitive with reference-based models, even outperforming the best COMET model from 2020 on this year's development data. Additionally, we present COMETinho, a lightweight COMET model that is 19x faster on CPU than the original model, while also achieving state-of-the-art correlations with MQM. Finally, in the "QE as a metric'' track, we also participated with a QE model trained using the OpenKiwi framework leveraging MQM scores and word-level annotations.

Are References Really Needed? Unbabel-IST 2021 Submission for the Metrics Shared Task

This paper describes Lingua Custodia's submission to the WMT21 shared task on machine translation using terminologies. 
We consider three directions, namely English to French, Russian, and Chinese.
We rely on a Transformer-based architecture as a building block, and we explore a method which introduces two main changes to the standard procedure to handle terminologies. The first one consists in augmenting the training data in such a way as to encourage the model to learn a copy behavior when it encounters terminology constraint terms. The second change is constraint token masking, whose purpose is to ease copy behavior learning and to improve model generalization. 
Empirical results show that our method satisfies most terminology constraints while maintaining high translation quality.

Lingua Custodia's participation at the WMT 2021 Machine Translation using Terminologies shared task

We describe the EdinSaar submission to the shared task of Multilingual Low-Resource Translation for North Germanic Languages at the Sixth Conference on Machine Translation (WMT2021). We submit multilingual translation models for translations to/from Icelandic (is), Norwegian-Bokmal (nb), and Swedish (sv). We employ various experimental approaches, including multilingual pre-training, back-translation, fine-tuning, and ensembling. In most translation directions, our models outperform other submitted systems.

EdinSaar@WMT21: North-Germanic Low-Resource Multilingual NMT

We present the joint contribution of IST and Unbabel to the WMT 2021 Shared Task on Quality Estimation. Our team participated on two tasks: Direct Assessment and Post-Editing Effort, encompassing a total of 35 submissions. For all submissions, our efforts focused on training multilingual models on top of OpenKiwi predictor-estimator architecture, using pre-trained multilingual encoders combined with adapters. We further experiment with and uncertainty-related objectives and features as well as training on out-of-domain direct assessment data.

IST-Unbabel 2021 Submission for the Quality Estimation Shared Task

This paper describes Charles University sub-mission for Terminology translation shared task at WMT21. The objective of this task is to design a system which translates certain terms based on a provided terminology database, while preserving high overall translation quality. We competed in English-French language pair. Our approach is based on providing the desired translations alongside the input sentence and training the model to use these provided terms. We lemmatize the terms both during the training and inference, to allow the model to learn how to produce correct surface forms of the words, when they differ from the forms provided in the terminology database.

Content not yet available

Regressive Ensemble for Machine Translation Quality Evaluation

Next from EMNLP 2021

Argument Pair Extraction with Mutual Guidance and Inter-sentence Relation Graph

Similar lecture

Allegro.eu submission to WMT21 News Translation Task

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES