United Arab Emirates

In this work, we propose a flexible multimodal transformer architecture for classifying depression from social media posts containing text and images. We show that our model is able to surpass other methods by a large margin on a popular multimodal depression dataset from Twitter, obtaining state-of-the-art results of 95\% F1 score. Our model is flexible and can easily incorporate temporal information by manipulating the transformer positional encodings. Moreover, it can operate both on randomly sampled and unordered sets of posts, to be more robust to dataset noise, and on ordered collections of posts, accommodating the relative posting intervals without any major architectural modifications.

EMNLP 2022

Detecting Depression on Twitter with a Time-Aware Multimodal Transformer

time-aware methods

mental health

social media

workshop paper

## Welcome to EMNLP 2022!
I am delighted to welcome you to EMNLP 2022! I believe this conference has been complicated beyond any precedent. Over the past year, it’s been thrilling to see the organization team approach each new puzzle with creativity and enthusiasm. We hope that those participating in Abu Dhabi as well as those joining remotely will leave the conference feeling newly inspired by the program and newly connected to our ever-growing community. Following EMNLP 2021 and major NLP conferences since, EMNLP 2022 is “hybrid,” serving both virtual and in-person participants.

Our key innovations for EMNLP 2022 include:

* EMNLP 2022 is “hybrid” in a second sense, as well: we allowed both direct and rolling review paper submissions, building on the pilot experiment of EMNLP 2021, which considered a small number of ARR submissions. 
* Familiar from NAACL but new to EMNLP, we’ve added an industry track.
* During the conference, “portals” will link virtual poster sessions to in-person conference participants during poster sessions each day.
* The first ACL-family conference in the United Arab Emirates.

 *Message from Noah A. Smith, University of Washington and Allen Institute for AI, Seattle, Washington, USA* 
***EMNLP 2022 General Chair***
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/9eec7d4a287ee18c278b08229290aa83.png)](https://drive.google.com/file/d/1OlPv6QBeo62VVTughj2jkiLeyHd1WnUt/view)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/a3db7a768409f05192210d98601edb25.png)](https://emnlp2022.rocket.chat/)

To access this site you need to register. Please register [here](https://2022.emnlp.org/registration/).

Register here

Welcome!
EMNLP 2022 will take place in Abu Dhabi from December 7th to December 11th, 2022. And it will be held in hybrid mode, both online and offline.

Out-of-Vocabulary (OOV) is a problem for Neural Machine Translation (NMT). OOV refers to words with a low occurrence in the training data or those absent from the training data. Word or phrase-based Data Augmentation (DA) techniques have been used to alleviate this. However, existing DA techniques have addressed only one of these OOV types and are limited to considering either syntactic or semantic constraints. We present a word/phrase replacement-based DA technique that considers both types of OOV, considering both syntactic and semantic properties of the words to guarantee fluency in the synthetic sentences. This technique was experimented with low resource language pair Sinhala-English.

Data Augmentation to Address Out-of-Vocabulary Problem in Low-Resource Sinhala-English Neural Machine Translation

Microaggressions (MAs) are subtle, often unintentional, form of prejudice. They often impact people of minority or marginalized backgrounds. Gender MAs are common in India but not well understood. To the best of my knowledge, there is no existing MAs dataset in Hindi. To tackle the problem of MAs, a dataset needs to be created with which a machine learning model can be trained. Enabling a model to learn MAs would allow for MAs detection and possibly correction in text. In this paper, I build (i) a small gender MAs dataset and crowdsource its labeling, and (ii) a novel pipeline to detect MAs in Hindi.

Tackling Gender Microaggressions in Hindi Text

Lexicon-based method for sentiment analysis provides explainable results compared to deep learning methods. However, the approach does not capture contextual information and provides sub-optimal performance. In this paper, we build domain-specific lexicons and shifter patterns that capture contextual information. We show that symbolic approaches can achieve competitive performance as deep neural networks for sentiment classification tasks with limited data. Our approach is useful in cases where an explanation is the most important criterion and not the accuracy. 



Domain-Specific Lexicon-Based Sentiment Analysis using Contextual Shifter Patterns

Large language models appear to learn facts from the large text corpora they are trained on. Such facts are encoded implicitly within their many parameters, making it difficult to verify or manipulate what knowledge has been learned. Language models have recently been extended to multilingual language models (MLLMs), enabling knowledge to be learned across hundreds of languages. Meanwhile, knowledge graphs contain facts in an explicit triple format, which require careful and costly curation and are only available in a few high-resource languages, restricting their research and application. To address these issues, we propose to enhance MLLMs with knowledge from multilingual knowledge graphs (MLKGs) so as to tackle language and knowledge graph tasks across many languages, including low-resource ones. Specifically, we introduce a lightweight adapter set to enhance MLLMs with cross-lingual entity alignment and facts from MLKGs for many languages. Experiments on common benchmarks show that such enhancement benefits both MLLMs and MLKGs, achieving: (1) comparable or improved performance for knowledge graph completion and entity alignment relative to baselines, especially for low-resource languages (for which knowledge graphs are unavailable); and (2) improved MLLM performance on language understanding tasks that require multilingual factual knowledge; all while maintaining performance on other general language tasks.

Adapters for Enhanced Modeling of Multilingual Knowledge and Text

Bilingual lexicon induction (BLI) with limited bilingual supervision is a crucial yet challenging task in multilingual NLP. Current state-of-the-art BLI methods rely on the induction of cross-lingual word embeddings (CLWEs) to capture cross-lingual word similarities; such CLWEs are obtained 1) via traditional static models (e.g., VecMap), or 2) by extracting type-level CLWEs from multilingual pretrained language models (mPLMs), or 3) through combining the former two options. In this work, we propose a novel semi-supervised post-hoc reranking method termed BLICEr (BLI with Cross-Encoder Reranking), applicable to any precalculated CLWE space, which improves their BLI capability. The key idea is to 'extract' cross-lingual lexical knowledge from mPLMs, and then combine it with the original CLWEs. This crucial step is done via 1) creating a word similarity dataset, comprising positive word pairs (i.e., true translations) and hard negative pairs induced from the original CLWE space, and then 2) fine-tuning an mPLM (e.g., mBERT or XLM-R) in a cross-encoder manner to predict the similarity scores. At inference, we 3) combine the similarity score from the original CLWE space with the score from the BLI-tuned cross-encoder. BLICEr establishes new state-of-the-art results on two standard BLI benchmarks spanning a wide spectrum of diverse languages: it substantially outperforms a series of strong baselines across the board. We also validate the robustness of BLICEr with different CLWEs. Our code is available at https://github.com/cambridgeltl/BLICEr.

Improving Bilingual Lexicon Induction with Cross-Encoder Reranking

In an effort to study the inductive biases of language models, numerous studies have attempted to use linguistically motivated tasks as a proxy of sorts, wherein performance on these tasks would imply an inductive bias towards a specific linguistic phenomenon. In this study, we attempt to analyse the inductive biases of language models with respect to natural language phenomena
in the context of building multilingual embedding spaces.
We sample corpora from 2 sources in 15 languages and train language models on pseudo-bilingual variants of each corpus, created by duplicating each corpus and shifting token indices for half the resulting corpus. We evaluate the cross-lingual capabilities of these LMs, and show that while correlations with language families tend to be weak, other corpus-level characteristics, such as type-token ratio, tend to be more strongly correlated. Finally, we show that multilingual spaces can be built, albeit less effectively, even when additional destructive perturbations are applied to the training corpora, implying that (effectively) bag-of-words models also have an inductive bias that is sufficient for inducing multilingual spaces.

The Effects of Corpus Choice and Morphosyntax on Multilingual Space Induction

Most vision-and-language pretraining research focuses on English tasks. However, the creation of multilingual multimodal evaluation datasets (e.g. Multi30K, xGQA, XVNLI, and MaRVL) poses a new challenge in finding high-quality training data that is both multilingual and multimodal. We investigate whether machine translating English multimodal data can be an effective proxy for the lack of readily available multilingual data. We call this framework TD-MML: Translated Data for Multilingual Multimodal Learning, and it can be applied to any multimodal dataset and model. We apply it to both pretraining and fine-tuning data with a state-of-the-art model. In order to prevent models from learning from low-quality translated text, we propose two metrics for automatically removing such translations from the resulting datasets. In experiments on five tasks across 20 languages in the IGLUE benchmark, we show that translated data can provide a useful signal for multilingual multimodal learning, both at pretraining and fine-tuning.

Multilingual Multimodal Learning with Machine Translated Text

Timely and effective response to humanitarian crises requires quick and accurate analysis of large amounts of text data - a process that can highly benefit from expert-assisted NLP systems trained on validated and annotated data in the humanitarian response domain. To enable creation of such NLP systems, we introduce and release HumSet, a novel and rich multilingual dataset of humanitarian response documents annotated by experts in the humanitarian response community. The dataset provides documents in three languages (English, French, Spanish) and covers a variety of humanitarian crises from 2018 to 2021 across the globe. For each document, HumSet provides selected snippets (entries) as well as assigned classes to each entry annotated using common humanitarian information analysis frameworks. HumSet also provides novel and challenging entry extraction and multi-label entry classification tasks. In this paper, we take a first step towards approaching these tasks and conduct a set of experiments on Pre-trained Language Models (PLM) to establish strong baselines for future research in this domain. The dataset is available at https://blog.thedeep.io/humset/.

HUMSET: Dataset of Multilingual Information Extraction and Classification for Humanitarian Crisis Response

JamPatoisNLI provides the first dataset for natural language inference in a creole language, Jamaican Patois. Many of the most-spoken low-resource languages are creoles. These languages commonly have a lexicon derived from a major world language and a distinctive grammar reflecting the languages of the original speakers and the process of language birth by creolization. This gives them a distinctive place in exploring the effectiveness of transfer from large monolingual or multilingual pretrained models. While our work, along with previous work, shows that transfer from these models to low-resource languages that are unrelated to languages in their training set is not very effective, we would expect stronger results from transfer to creoles. Indeed, our experiments show considerably better results from few-shot learning of JamPatoisNLI than for such unrelated languages, and help us begin to understand how the unique relationship between creoles and their high-resource base languages affect cross-lingual transfer. JamPatoisNLI, which consists of naturally-occurring premises and expert-written hypotheses, is a step towards steering research into a traditionally underserved language and a useful benchmark for understanding cross-lingual NLP.

JamPatoisNLI: A Jamaican Patois Natural Language Inference Dataset

Zero-resource cross-lingual transfer approaches aim to apply supervised models
from a source language to unlabelled target languages. In this paper we perform
an in-depth study of the two main techniques employed so far for cross-lingual
zero-resource sequence labelling, based either on data or model transfer.
Although previous research has proposed translation and annotation projection
(data-based cross-lingual transfer) as an effective technique for cross-lingual
sequence labelling, in this paper we experimentally demonstrate that high
capacity multilingual language models applied in a zero-shot (model-based
cross-lingual transfer) setting consistently outperform data-based
cross-lingual transfer approaches. A detailed analysis of our results suggests
that this might be due to important differences in language use. More
specifically, machine translation often generates a textual signal which is
different to what the models are exposed to when using gold standard data,
which affects both the fine-tuning and evaluation processes. Our results also
indicate that data-based cross-lingual transfer approaches remain a competitive
option when high-capacity multilingual language models are not available.

Detecting Depression on Twitter with a Time-Aware Multimodal Transformer

Downloads

Next from EMNLP 2022

Data Augmentation to Address Out-of-Vocabulary Problem in Low-Resource Sinhala-English Neural Machine Translation

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES