United Arab Emirates

Most vision-and-language pretraining research focuses on English tasks. However, the creation of multilingual multimodal evaluation datasets (e.g. Multi30K, xGQA, XVNLI, and MaRVL) poses a new challenge in finding high-quality training data that is both multilingual and multimodal. In this paper, we investigate whether machine translating English multimodal data can be an effective proxy for the lack of readily available multilingual data. We call this framework TD-MML: Translated Data for Multilingual Multimodal Learning, and it can be applied to any multimodal dataset and model. We apply it to both pretraining and fine-tuning data with a state-of-the-art model. In order to prevent models from learning from low-quality translated text, we propose two metrics for automatically removing such translations from the resulting datasets. In experiments on five tasks across 20 languages in the IGLUE benchmark, we show that translated data can provide a useful signal for multilingual multimodal learning, both at pretraining and fine-tuning.

EMNLP 2022

Multilingual Multimodal Learning with Machine Translated Text

vision-and-language pretraining

multilingual multimodal learning

machine translation

findings / work in progress

## Welcome to EMNLP 2022!
I am delighted to welcome you to EMNLP 2022! I believe this conference has been complicated beyond any precedent. Over the past year, it’s been thrilling to see the organization team approach each new puzzle with creativity and enthusiasm. We hope that those participating in Abu Dhabi as well as those joining remotely will leave the conference feeling newly inspired by the program and newly connected to our ever-growing community. Following EMNLP 2021 and major NLP conferences since, EMNLP 2022 is “hybrid,” serving both virtual and in-person participants.

Our key innovations for EMNLP 2022 include:

* EMNLP 2022 is “hybrid” in a second sense, as well: we allowed both direct and rolling review paper submissions, building on the pilot experiment of EMNLP 2021, which considered a small number of ARR submissions. 
* Familiar from NAACL but new to EMNLP, we’ve added an industry track.
* During the conference, “portals” will link virtual poster sessions to in-person conference participants during poster sessions each day.
* The first ACL-family conference in the United Arab Emirates.

 *Message from Noah A. Smith, University of Washington and Allen Institute for AI, Seattle, Washington, USA* 
***EMNLP 2022 General Chair***
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/9eec7d4a287ee18c278b08229290aa83.png)](https://drive.google.com/file/d/1OlPv6QBeo62VVTughj2jkiLeyHd1WnUt/view)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/a3db7a768409f05192210d98601edb25.png)](https://emnlp2022.rocket.chat/)

To access this site you need to register. Please register [here](https://2022.emnlp.org/registration/).

Register here

Welcome!
EMNLP 2022 will take place in Abu Dhabi from December 7th to December 11th, 2022. And it will be held in hybrid mode, both online and offline.

Timely and effective response to humanitarian crises requires quick and accurate analysis of large amounts of text data – a process that can highly benefit from expert-assisted NLP systems trained on validated and annotated data in the humanitarian response domain. To enable creation of such NLP systems, we introduce and release HumSet, a novel and rich multilingual dataset of humanitarian response documents annotated by experts in the humanitarian response community. The dataset provides documents in three languages (English, French, Spanish) and covers a variety of humanitarian crises from 2018 to 2021 across the globe. For each document, HUMSET provides selected snippets (entries) as well as assigned classes to each entry annotated using common humanitarian information analysis frameworks. HUMSET also provides novel and challenging entry extraction and multi-label entry classification tasks. In this paper, we take a first step towards approaching these tasks and conduct a set of experiments on Pre-trained Language Models (PLM) to establish strong baselines for future research in this domain. The dataset is available at https://blog.thedeep.io/humset/.

HumSet: Dataset of Multilingual Information Extraction and Classification for Humanitarian Crises Response

JamPatoisNLI provides the first dataset for natural language inference in a creole language, Jamaican Patois.
Many of the most-spoken low-resource languages are creoles. These languages commonly have a lexicon derived from a major world language and a distinctive grammar reflecting the languages of the original speakers and the process of language birth by creolization. This gives them a distinctive place in exploring the effectiveness of transfer from large monolingual or multilingual pretrained models. While our work, along with previous work, shows that transfer from these models to low-resource languages that are unrelated to languages in their training set is not very effective, we would expect stronger results from transfer to creoles. Indeed, our experiments show considerably better results from few-shot learning of JamPatoisNLI than for such unrelated languages, and help us begin to understand how the unique relationship between creoles and their high-resource base languages affect cross-lingual transfer. JamPatoisNLI, which consists of naturally-occurring premises and expert-written hypotheses, is a step towards steering research into a traditionally underserved language and a useful benchmark for understanding cross-lingual NLP.

JamPatoisNLI: A Jamaican Patois Natural Language Inference Dataset

Zero-resource cross-lingual transfer approaches aim to apply supervised models
from a source language to unlabelled target languages. In this paper we perform
an in-depth study of the two main techniques employed so far for cross-lingual
zero-resource sequence labelling, based either on data or model transfer.
Although previous research has proposed translation and annotation projection
(data-based cross-lingual transfer) as an effective technique for cross-lingual
sequence labelling, in this paper we experimentally demonstrate that high
capacity multilingual language models applied in a zero-shot (model-based
cross-lingual transfer) setting consistently outperform data-based
cross-lingual transfer approaches. A detailed analysis of our results suggests
that this might be due to important differences in language use. More
specifically, machine translation often generates a textual signal which is
different to what the models are exposed to when using gold standard data,
which affects both the fine-tuning and evaluation processes. Our results also
indicate that data-based cross-lingual transfer approaches remain a competitive
option when high-capacity multilingual language models are not available.

Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings

This paper deals with the problem of incre-
mental dialect identification. Our goal is to
reliably determine the dialect before the full
utterance is given as input. The major part
of the previous research on dialect identification has been model-centric, focusing on performance. We address a new question: How much input is needed to identify a dialect? Our
approach is a data-centric analysis that results
in general criteria for finding the shortest input
needed to make a plausible guess. Working
with three sets of language dialects (Swiss German, Indo-Aryan and Arabic languages), we
show that it is possible to generalize across dialects and datasets with two input shortening
criteria: model confidence and minimal input
length (adjusted for the input type). The source
code for experimental analysis can be found at
Github.

Early Guessing for Dialect Identification

Accurate alignment between languages is fundamental for improving cross-lingual pre-trained language models (XLMs). Motivated by the natural phenomenon of code-switching (CS) in multilingual speakers, CS has been used as an effective data augmentation method that offers language alignment at word- or phrase-level, in contrast to sentence-level via parallel instances. Existing approaches either use dictionaries or parallel sentences with word-alignment to generate CS data by randomly switching words in a sentence. However, such methods can be suboptimal as dictionaries disregard semantics, and syntax might become invalid after random word switching. In this work, we propose EntityCS, a method that focuses on Entity-level Code-Switching to capture fine-grained cross-lingual semantics without corrupting syntax. We use Wikidata and the English Wikipedia to construct an entity-centric CS corpus by switching entities to their counterparts in other languages. We further propose entity-oriented masking strategies during intermediate model training on the EntityCS corpus for improving entity prediction. Evaluation of the trained models on four entity-centric downstream tasks shows consistent improvements over the baseline with a notable increase of 10% in Fact Retrieval. We release the corpus and models to assist research on code-switching and enriching XLMs with external knowledge.

EntityCS: Improving Zero-Shot Cross-lingual Transfer with Entity-Centric Code Switching

The goal of this paper is to present a methodology that allows social scientists and domain experts in Latin America to explore  social biases present in word embeddings, and to define the kinds of biasses they want to explore. In particular, we investigate how domain experts and social scientists can transfer their domain knowledge through representations of lexical resources that they can manipulate intuitively, resorting to their expertise to assess both hypothesized and emergent biases.

Lexical methods for bias exploration from a Latin American perspective

In this paper, we present a new approach to overcome the problem of language resources that share significant amount of linguistic resource in the Ometo language family. The dataset is collected from religious domain consisting of four Ometo (Wolaita, Gamo, Gofa, and Dawuro) languages paired with English sentence. Among the experiments, the Wolaita, Dawuro and Gamo paired with English sentence combination for training and Gofa for testing gives highest BLEU score 4.5 than the other combinations. The BLEU score shows a promising result despite the language differences, the morphological richness, and complexity of the Ometo languages which has high impact on the performance of the Ometo-English machine translation.

Low Resourced Multilingual Neural Machine Translation for Ometo-English

In spite of many proposals to solve the neural machine translation problem of low-resource languages, it continues to be difficult for languages with few resources. The issue becomes even more complicated when few resources cover only a single domain. In our attempt to combat this issue, we propose a new approach to improve NMT for low-resource languages. The proposed approach using the transformer model shows 5.3, 5.0, and 3.7 BLEU score improvement for Gamo-English, Gofa-English, and Dawuro-English language pairs, respectively. We discuss our contributions and envisage future steps in this challenging research area.

Improving neural machine translation for low-resource languages using related language resources

Word embeddings powered the early days of neural network-based NLP research. Their effectiveness in small data regimes makes them still relevant in low-resource environments. However, they are limited in two critical ways: linearly increasing memory requirement based on the number of tokens and out-of-vocabulary token handling. In this work, we present a distillation technique of word embeddings into a CNN network using contrastive learning. This method allows embeddings to be regressed given the characters of a token. Low resource languages are the primary beneficiary of these distilled embeddings and hence, we show the effectiveness of such a model on Amharic, Semitic languages that is spoken in Ethiopia.

DistillEmb: Distilling word embeddings via contrastive learning

This paper introduces the Forgotten Realms Wiki (FRW) data set and uses it for domain specific natural language generation. The data set was extracted from the Forgotten Realms Fandom wiki dedicated to the eponymous Dungeons and Dragons (D\&D) game setting. The FRW data set is constituted of 11 sub-data sets in varying degrees of pre-processing including plain text, annotated text, linked graphs, info-box dictionary, and several embedding models. This is the first data set of this size for the D\&D domain. We then present a pairwise similarity comparison benchmark and perform D\&D domain specific natural language gener- ation using the corpus and evaluate the named entity classification with respect to the lore of Forgotten Realms.

Multilingual Multimodal Learning with Machine Translated Text

Downloads

Next from EMNLP 2022

HumSet: Dataset of Multilingual Information Extraction and Classification for Humanitarian Crises Response

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES