United States

Though recently there have been an increased interest in how pre-trained language models encode different linguistic features, there is still a lack of systematic comparison between languages with different morphology and syntax. In this paper, using BERT as an example of a pre-trained model, we compare how three typologically different languages (English, Korean, and Russian) encode morphology and syntax features across different layers. In particular, we contrast languages which differ in a particular aspect, such as flexibility of word order, head directionality, morphological type, presence of grammatical gender, and morphological richness, across four different tasks.

NAACL 2022

Cross-linguistic comparison of linguistic feature encoding in BERT models
for typologically different languages

linguistic features encoding

cross-lingual

pre-trained models

**Welcome to ACL-IJCNLP 2021!**

The great event is jointly organized by the Association for Computational Linguistics (ACL) and Asian Federation of Natural Language Processing (AFNLP). 

As in previous years, the program of the conference includes a poster session, tutorials, workshops and demonstrations in addition to the main conference.


We were able to keep the registration fees similar to those charged for the virtual ACL 2020. The one fee allows attendance at the main conference and any/all tutorials and workshops. These fees would be $125 Regular Early and $175 Regular Late; $50 Student Early and $75 Student Late. Early registration closes at midnight July 11, 2021 (Eastern Daylight Time).

**Reminder:** It is ACL’s policy that at least one author of each accepted paper (including ACL Finding papers) must register for the conference.

**Reminder2:** Underline site will open closer to the event. If you already registered you will receive access detail

Registration is now open

ACL-IJCNLP 2021

The great event is jointly organized by the Association for Computational Linguistics (ACL) and Asian Federation of Natural Language Processing (AFNLP).

workshop paper

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

We study the extent to which emoji can be used to add interpretability to embeddings of text and emoji. To do so, we extend the POLAR-framework that transforms word embeddings to interpretable counterparts and apply it to word-emoji embeddings trained on four years of messaging data from the Jodel social network. We devise a crowdsourced human judgement experiment to study six usecases, evaluating against words only, what role emoji can play in adding interpretability to word embeddings. That is, we use a revised POLAR approach interpreting words and emoji with words, emoji or both according to human judgement. We find statistically significant trends demonstrating that emoji can
be used to interpret other emoji very well.

Interpreting Emoji with Emoji: 🏖️ => ☀️😎🌊

This paper presents a new iconic language, the IKON language, and its philosophical, linguis­tic, and graphical principles. We examine some case studies to highlight the semantic complex­ity of the visual representation of meanings. We also introduce the Iconometer test to vali­date our icons and their application to the med­ical domain, through the creation of iconic sen­tences.

Beyond Emoji: an insight into the IKON language

Identifying sarcasm is a challenging research problem owing to its highly contextual nature. Several researchers have attempted numerous mechanisms to incorporate context, linguistic aspects, and supervised and semi-supervised techniques to determine sarcasm. It has also been noted that emojis in a text may also hold key indicators of sarcasm. However, the availability of sarcasm datasets with emojis is scarce. This makes it challenging to effectively study the sarcastic nature of emojis. In this work, we present SarcOji which has been compiled from five publicly available sarcasm datasets. SarcOji contains labeled English texts which all have emojis. We also analyze SarcOji to determine if there is an incongruence in the polarity of text and emojis used therein. Further, emojis’ usage, occurrences, and positions in the context of sarcasm are also studied in this compiled dataset. With SarcOji we have been able to demonstrate that frequency of occurrence of an emoji and its position are strong indicators of sarcasm. SarcOji dataset is now publicly available with several derived features like sentiment scores of text and emojis, most frequent emoji, and its position in the text. Compilation of the SarcOji dataset is an initial step to enable the study of the role of emojis in communicating sarcasm. SarcOji dataset can also serve as a go-to dataset for various emoji-based sarcasm detection techniques.

Understanding the Sarcastic Nature of Emojis with SarcOji

This paper proposes EmojiCloud, an open-source Python-based emoji cloud visualization tool, to generate a quick and straightforward understanding of emojis from the perspective of frequency and importance. EmojiCloud is flexible enough to support diverse drawing shapes, such as rectangles, ellipses, and image masked canvases. We also follow inclusive and personalized design principles to cover the unique emoji designs from seven emoji vendors (e.g., Twitter, Apple, and Windows) and allow users to customize plotted emojis and background colors. We hope EmojiCloud can benefit the whole emoji community due to its flexibility, inclusiveness, and customizability.

EmojiCloud: a Tool for Emoji Cloud Visualization

This study examines the evolutionary trajectory of graphicons in a 13-year corpus of comments from BiliBili, a popular Chinese video-sharing platform. Findings show that emoticons (kaomoji) rose and fell in frequency, while emojis and stickers are both presently on the rise. Graphicon distributions differ in comments and replies to comments. There is also a strong correlation between the types of graphicons used in comments and their corresponding replies, suggesting a priming effect. Finally, qualitative analysis of the 10 most-frequent kaomojis, emojis, and stickers reveals a trend for each successive graphicon type to become less about emotion expression and more integrated with platform-specific culture and the Chinese language. These findings lend partial support to claims in the literature about graphicon evolution.

Graphicon Evolution on the Chinese Social Media Platform BiliBili

Research to tackle hate speech plaguing online media has made strides in providing solutions, analyzing bias and curating data. A challeng- ing problem is ambiguity between hate speech and offensive language, causing low perfor- mance both overall and specifically for the hate speech class. It can be argued that misclas- sifying actual hate speech content as merely offensive can lead to further harm against tar- geted groups. In our work, we mitigate this potentially harmful phenomenon by proposing an adversarial debiasing method to separate the two classes. We show that our method works for English, Arabic German and Hindi, plus in a multilingual setting, improving performance over baselines.

Separating Hate Speech and Offensive Language Classes via Adversarial Debiasing

With the widespread use of social media, online hate is increasing, and microaggressions, unintentional offensive remarks in everyday life, are receiving
attention. We explore the possibility of using pre-trained language models to
automatically generate messages that combat the associated offensive texts.
Specifically, we focus on using prompting to steer model generation as it requires fewer data and computation than fine-tuning and shows the potential for using prompting in the proposed generation task. We also propose a human evaluation perspective; offensiveness, stance, and informativeness. After obtaining 306 counterspeech and 42 micro intervention messages generated by GPT-2, GPT-Neo, and GPT-3, we conducted a human evaluation using Amazon Mechanical Turk and found that GPT-3 produces messages of the highest quality among the three systems. Also, We discuss the pros and cons of using our evaluation perspectives. We release a corpus of Countering HAte Speech and Microaggressions (CHASM), annotated machine-generated counternarratives along with the annotation to promote further research on automatic counternarrative generation and its evaluation.

Towards Automatic Generation of Messages Countering Online Hate Speech andMicroaggressions

This paper presents a comprehensive corpus for the study of socially unacceptable language in Dutch. The corpus extends and revise an existing resource with more data and intro- duces a new annotation dimension for offen- sive language, making it a unique resource in the Dutch language panorama. Each language phenomenon (abusive and offensive language) in the corpus has been annotated with a multi- layer annotation scheme modelling the explic- itness and the target(s) of the abuse/offence in the message. We have conducted a new set of experiments with different classification algo- rithms on all annotation dimensions. Monolin- gual Pre-Trained Language Models prove as the best systems, obtaining a macro-average F1 of 0.828 for binary classification of offensive language, and 0.579 for the targets of offensive messages. Furthermore, the best system obtains a macro-average F1 of 0.637 for distinguishing between abusive and offensive messages.

“Zo Grof !”: A Comprehensive Corpus for Offensive and Abusive Language in Dutch

Toxic language can take many forms, from explicit hate speech to more subtle microaggressions. Within this space, models identifying transphobic language have largely focused on overt forms. However, a more pernicious and subtle source of transphobic comments comes in the form of statements made by Trans-exclusionary Radical Feminists (TERFs); these statements often appear seemingly-positive and promote women's causes and issues, while simultaneously denying the inclusion of transgender women as women. Here, we introduce two models to mitigate this antisocial behavior. The first model identifies TERF users in social media, recognizing that these users are a main source of transphobic material that enters mainstream discussion and whom other users may not desire to engage with in good faith. The second model tackles the harder task of recognizing the masked rhetoric of TERF messages and introduces a new dataset to support this task. Finally, we discuss the ethics of deploying these models to mitigate the harm of this language, arguing for a balanced approach that allows for restorative interactions.

The subtle language of exclusion: Identifying the Toxic Speech of Trans-exclusionary Radical Feminists

In an era of increasingly large pre-trained language models, knowledge distillation is a powerful tool for
transferring information from a large model to a smaller one. In particular, distillation is of tremendous
benefit when it comes to real-world constraints such as serving latency or serving at scale. However, 
a loss of robustness in language understanding may be hidden in the process and not immediately revealed
when looking at high-level evaluation metrics. We investigate the hidden costs: what
is "lost in distillation", especially in regards to identity-based model bias using the case study
of toxicity modeling. With 
reproducible models using open source training sets, we investigate models distilled
from a BERT teacher baseline. Using both open source and proprietary
big data models, we investigate these hidden performance costs.

Cross-linguistic comparison of linguistic feature encoding in BERT models for typologically different languages

Downloads

Next from NAACL 2022

Interpreting Emoji with Emoji: 🏖️ => ☀️😎🌊

Similar lecture

Evaluating morphological typology in zero-shot cross-lingual transfer

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Cross-linguistic comparison of linguistic feature encoding in BERT models for typologically different languages

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from NAACL 2022

Interpreting Emoji with Emoji: 🏖️ => ☀️😎🌊

Similar lecture

Evaluating morphological typology in zero-shot cross-lingual transfer

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads