Real-world NLP applications often deal with nonstandard text (e.g., dialectal, informal, or misspelled text). However, language models like BERT deteriorate in the face of dialect variation or noise. How do we push BERT’s modeling capabilities to encompass nonstandard text? Fine-tuning helps, but it is designed for specializing a model to a task and does not seem to bring about the deeper, more pervasive changes needed to adapt a model to nonstandard language. In this paper, we introduce the novel idea of sandwiching BERT's encoder stack between additional encoder layers trained to perform masked language modeling on noisy text. We find that our approach, paired with recent work on including character-level noise in fine-tuning data, can promote zero-shot transfer to dialectal text, as well as reduce the distance in the embedding space between words and their noisy counterparts.

BERTwich: Extending BERT’s Capabilities to Model Dialectal and Noisy Text

In this work, we induce character-level noise in various forms when fine-tuning BERT to enable zero-shot cross-lingual transfer to unseen dialects and languages. We fine-tune BERT on three sentence-level classification tasks and evaluate our approach on an assortment of unseen dialects and languages. We find that character-level noise can be an extremely effective agent of cross-lingual transfer under certain conditions, while it is not as helpful in others. Specifically, we explore these differences in terms of the nature of the task and the relationships between source and target languages, finding that introduction of character-level noise during fine-tuning is particularly helpful when a task draws on surface level cues and the source-target cross-lingual pair has a relatively high lexical overlap with shorter (i.e., less meaningful) unseen tokens on average.

Fine-Tuning BERT with Character-Level Noise for Zero-Shot Transfer to Dialects and Closely-Related Languages

**Are you attending this poster session virtually?** 
 Virtual posters are presented in Gather Town. Connect by clicking on the link that you can find in the 'Abstracts' tab of the posters.
 
**Are you attending this poster session in person?** 
Virtual posters are presented in Gather Town. Connect by clicking on the link that you can find in the 'Abstracts' tab of the posters.

PS6 (GatherTown) Posters, Demo, Industry, Findings

poster

**Welcome to EMNLP 2023!**

On behalf of the EMNLP 2023 Organizing Committee, I extend a warm and heartfelt welcome to all of you to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). It is with immense pleasure and excitement that we gather here in Singapore, a vibrant hub of innovation and technological advancement.

The conference program is packed with insightful presentations, thought-provoking workshops, and engaging networking opportunities. In addition to the technical sessions, we have also planned several social events that will provide you with the opportunity to connect with your colleagues.

The conference is held in person at the Resorts World Convention Centre in Singapore, and available on-line with the help of Underline.

For those in Singapore, I hope that you will find time to explore the exciting Sentosa Island, Gardens by the Bay, Singapore Botanic Gardens and the many other attractions unique to Singapore.

Of course, we are grateful to our sponsors and partners for their generous support of EMNLP 2023, whose contributions make it possible for us to host this world-class event.

Yuji Matsumoto (RIKEN AIP) 
EMNLP 2023 General Chair

To access the **EMNLP 2023** event page on Underline, you need to register for the Conference. 
Please follow **[this link](https://2023.emnlp.org/registration/)** for more details.

EMNLP 2023

EMNLP 2023 took place in Singapore from Dec 6th to Dec 10th, 2023.

**Workshop Organizers:** Duygu Ataman, Sebastian Ruder, Gözde Gül Şahin, Genta Indra Winata, Orhan Firat, Hila Gonen, Omer Goldman, Arya McCarthy, Chris Chinenye Emezue, David Ifeoluwa Adelani and Francesco Tinner
 *Coffee Break 10:30 am* 
*Lunch Break 12:30 pm* 
*Coffee Break 3:30 pm* [**Workshop Website**](https://sigtyp.github.io/ws2023-mrl.html) 
This workshop aims to bring together the research community consisting of scientists studying different aspects in multilingual representation learning, currently the most promising approach to improve the NLP in low-resource or underrepresented languages, and provide the rapidly growing number of researchers working on the topic with a means of communication and an opportunity to present their work and exchange ideas. The main objectives of the workshop will be: 
* To construct and present a wide array of multi-lingual representation learning methods, including their theoretical formulation and analysis, practical aspects such as the application of current state-of-the-art approaches in transfer learning to different tasks or studies on adaptation into previously under-studied context;
 
* To provide a better understanding on how the language typology may impact the applicability of these methods and motivate the development of novel methods that are more generic or competitive in different languages;
 
* To promote collaborations in developing novel software libraries or benchmarks in implementing or evaluating multi-lingual models that would accelerate progress in the field.
 [**Invited Speakers**](https://sigtyp.github.io/ws2023-mrl.html) [**Schedule**](https://sigtyp.github.io/ws2023-mrl.html)
 [**Go to Gather**](https://app.gather.town/app/ROH1myZvp9ba7a3C/EMNLP2023?spawnToken=JJmXxxFGTOGjyZlKSJiT)

W18: The Third Workshop on Multi-lingual Representation Learning(MRL)

workshop paper

**Organizers:** Yves Scherrer - University of Helsinki (Finland); Tommi Jauhiainen - University of Helsinki (Finland); Nikola Ljubešić - Jožef Stefan Institute (Slovenia) and University of Zagreb (Croatia); Preslav Nakov - Mohamed bin Zayed University of Artificial Intelligence (UAE); Jörg Tiedemann - University of Helsinki (Finland); Marcos Zampieri - George Mason University (USA) For more details, please visit our [workshop website](https://sites.google.com/view/vardial-2023?pli=1)

Tenth Workshop on NLP for Similar Languages, Varieties and Dialects

### Welcome to the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL) 
Continuing its mission of expanding and involving the science community of all European countries, EACL had selected the Ukrainian community for the 16th EACL, which took place online due to the COVID pandemic. Unfortunately, the ongoing war made the organisation in Kyiv impossible. Considering the importance of physical interaction among researchers, especially after the restrictions imposed by the COVID pandemic, in addition to an online mode, the conference will be held in Dubrovnik, Croatia, from 2 to 6 of May, 2023. The original aim of strengthening the connection with the Ukrainian community will not change as our program will feature a dedicated session and a workshop to highlight work on Ukrainian language technologies. As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

You need to register for the conference in order to access this site. Please visit https://2023.eacl.org/registration for more information.

EACL 2023

As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

Aarohi Srivastava

3

SHORT BIO

Presentations

BERTwich: Extending BERT’s Capabilities to Model Dialectal and Noisy Text

BERTwich: Extending BERT’s Capabilities to Model Dialectal and Noisy Text

Fine-Tuning BERT with Character-Level Noise for Zero-Shot Transfer to Dialects and Closely-Related Languages

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES