We propose a simple method to align multilingual contextual embeddings as a postpretraining step for improved cross-lingual transferability of the pretrained language models. Using parallel data, our method aligns embeddings on the word level through the recently proposed Translation Language Modeling objective as well as on the sentence level via contrastive learning and random input shuffling. We also perform sentence-level code-switching with English when finetuning on downstream tasks. On XNLI, our best model (initialized from mBERT) improves over mBERT by 4.7% in the zero-shot setting and achieves comparable result to XLM for translate-train while using less than 18% of the same parallel data and 31% fewer model parameters. On MLQA, our model outperforms XLM-RBase, which has 57% more parameters than ours.

Multilingual BERT Post-Pretraining Alignment

Walk through Challenges of intent detection iin real world setting and performance benchmark of open-source solutions and commercial solutions.


Benchmarking Commercial Intent Detection Services with Practice-Driven Evaluations

**Session Chair: ** Valia Kordoni

1C-Oral: Machine Translation and Multilinguality

technical paper

2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics.

**Whova App** 
Stay in touch with your fellow conference attendees via the [Whova App](https://whova.com/portal/webapp/nacon_202106/)

**Conference Structure**
https://2021.naacl.org/blog/conference-structure/

**Walkthrough video of how to NAACL 2021** 

Please take a moment to view this video explaining how to navigate the platform, attend sessions network with other attendees. 


<figure class="video_container">
 <iframe src="https://screencast-o-matic.com/watch/crhwbGVh3vx?v=6&ff=1&title=0&controls=1" width=640 height=350 frameborder="0" allowfullscreen="true"> </iframe>
</figure>

NAACL 2021

**Session Chair: ** Avneesh Saluja

Haode Qi

2

3

SHORT BIO

Presentations

Multilingual BERT Post-Pretraining Alignment

Benchmarking Commercial Intent Detection Services with Practice-Driven Evaluations

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES