United States

Despite recent advances in machine learning based hate speech detection, classifiers still struggle with generalizing knowledge to out-
of-domain data samples. In this paper, we investigate the generalization capabilities of deepearning models to different target groups of hate speech under clean experimental settings. Furthermore, we assess the efficacy of three different strategies of unsupervised domain adaptation to improve these capabilities. Given the diversity of hate and its rapid dynamics in the online world (e.g. the evolution of new target groups like virologists during the COVID-19 pandemic), robustly detecting hate aimed at newly identified target groups is a highly relevant research question. We show that naively trained models suffer from a target group specific bias, which can be reduced via domain adaptation. We were able to achieve a relative improvement of the F1-score between 5.8% and 10.7% for out-of-domain target groups of hate speech compared to baseline approaches by utilizing domain adaptation.

NAACL 2022

Improving Generalization of Hate Speech Detection Systems to Novel Target Groups via Domain Adaptation

hate speech detection; domain adaptation; hate targets; bias

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

workshop paper

Because of the compositionality of natural language, syntactic structure which contains the information about the relationship between words is a key factor for semantic understanding. However, the widely adopted Transformer is hard to learn the syntactic structure effectively in dialogue generation tasks. To explicitly model the compositionaity of language in Transformer Block, we restrict the information flow between words by constructing directed dependency graph and propose Dependency Relation Attention (DRA). Experimental results demonstrate that DRA can further improve the performance of state-of-the-art models for dialogue generation.

ErAConD: Error Annotated Conversational Dialog Dataset for Grammatical Error Correction

Recent work has shown that predicate-argument label representations from semantic role labeling (SRL) can be concatenated with BERT representations to improve natural language understanding tasks such as natural language inference (NLI) and reading comprehension. Two natural questions that arise are whether infusing SRL representations with BERT 1) improves model performance and 2) increases the model’s linguistic awareness. This paper aims at answering both questions with a case study on the NLI task. We start by analyzing whether and how infusing SRL information helps BERT learn linguistic knowledge. We compare model performance on two benchmark datasets, SNLI and MNLI. We also conduct in-depth analysis on two probing datasets, Breaking NLI and HANS, which contain abundant examples where SRL information is expected to be helpful. We found that combining SRL representations with BERT representations does outperform BERT-only representations in general, with better awareness in lexical meaning and world knowledge but not in logic knowledge. We also found that infusing SRL information via predicate-wise concatenation with BERT word representations followed by an interaction layer is more effective than sentence-wise concatenation.

Learning Natural Language Generation with Truncated Reinforcement Learning

Tables are an important form of structured data for both human and machine readers alike, providing answers to questions that cannot, or cannot easily, be found in texts. Recent work has designed special models and training paradigms for table-related tasks such as table-based question answering and table retrieval. Though effective, they add complexity in both modeling and data acquisition compared to generic text solutions and obscure which elements are truly beneficial. In this work, we focus on the task of table retrieval, and ask: “is table-specific model design necessary for table retrieval, or can a simpler text-based model be effectively used to achieve a similar result?” First, we perform an analysis on a table-based portion of the Natural Questions dataset (NQtable), and find that structure plays a negligible role in more than 70% of the cases. Based on this, we experiment with a general Dense Passage Retriever (DPR) based on text and a specialized Dense Table Retriever (DTR) that uses table-specific model designs. We find that DPR performs well without any table-specific design and training, and even achieves superior results compared to DTR when fine-tuned on properly linearized tables. We then experiment with three modules to explicitly encode table structures, namely auxiliary row/column embeddings, hard attention masks, and soft relation-based attention biases. However, none of these yielded significant improvements, suggesting that table-specific model design may not be necessary for table retrieval.

Enhancing Self-Attention with Knowledge-Assisted Attention Maps

Recent methods based on pre-trained language models have exhibited superior performance over tabular tasks (e.g. tabular NLI), despite showing inherent problems with reasoning over the tabular data (Gupta et al., 2021). In this work, we utilize Pattern-Exploiting Training (PET) to strengthen pre-existing knowledge
and reasoning abilities of these tabular reasoning models’. Compared to current baselines, our upgraded model exhibits a superior understanding of knowledge facts and tabular reasoning. Additionally, we demonstrate that such models are more effective for underlying downstream tasks of tabular inference on InfoTabS. Furthermore, we also show our model’s robustness against adversarial sets generated through various character and word level perturbations on InfoTabS

Masked Measurement Prediction: Learning to Jointly Predict Quantities and Units from Textual Context

Within the emerging research efforts to combine structured and unstructured knowledge, many approaches incorporate factual knowledge, e.g., available in form of structured knowledge graphs (KGs), into pre-trained language models (PLMs) and then apply the knowledge-enhanced PLMs to downstream NLP tasks. However, (1) they typically only consider \textit{static} factual knowledge, whereas, e.g., knowledge graphs (KGs) also contain \textit{temporal facts} or \textit{events} indicating evolutionary relationships among entities at different timestamps. (2) PLMs cannot be directly applied to many KG tasks, such as temporal KG completion. In this paper, we focus on \textbf{e}nhancing temporal knowledge embeddings with \textbf{co}ntextualized \textbf{la}nguage representations (ECOLA). We align structured knowledge, contained in temporal knowledge graphs, with their textual descriptions extracted from news articles, and propose a novel knowledge-text prediction task to inject the abundant information from descriptions into temporal knowledge embeddings. ECOLA jointly optimizes the knowledge-text prediction objective and the temporal knowledge embeddings, which can simultaneously take full advantage of textual and knowledge information. The proposed fusion method is model-agnostic and can be combined with potentially any temporal KG model. For training ECOLA, we introduce three temporal KG datasets with aligned textual descriptions. Experimental results on the temporal knowledge graph completion task show that ECOLA outperforms state-of-the-art temporal KG models by a large margin. The proposed datasets can serve as new temporal KG benchmarks and facilitate future research on structured and unstructured knowledge integration.

Multi-Hop Open-Domain Question Answering over Structured and Unstructured Knowledge

Hate Speech has been around in Ethiopia before social media, but with very limited reach. With the coming of social media companies that have no or little business interest to lose in low-resourced languages such as those in Ethiopia, diaspora activists that have nothing or little to lose from engaging in online hate speech, and several technical and institutional challanges, hate speech on social media slowly became mainstream in Ethiopia, tearing societies apart and eventually serving as an animating force for a genocidal war on Tigrayans. In this speech, I will briefly assess the normalization of hate speech in Ethiopia, the factors that led to this, and the role hate speech and social media played during the Tigray war, social media hate speech detection and monitoring, and what should be done going forward. 


Social Media and Hate Speech in time of War: The case of Tigray War

Large pre-trained models are usually fine-tuned on downstream task data, and tested on unseen data.
When the train and test data come from different domains, the model is likely to struggle, as it is not adapted to the test domain. We propose a new approach for domain adaptation (DA), using neuron-level interventions: 
We modify the representation of each test example in specific neurons, resulting in a counterfactual example from the source domain, which the model is more familiar with. The modified example is then fed back into the model.
While most other DA methods are applied during training time, ours is applied during inference only, making it more efficient and applicable. 
Our experiments show that our method improves performance on unseen domains.

IDANI: Inference-time Domain Adaptation via Neuron-level Interventions

Deep learning methods have enabled task oriented
semantic parsing of increasingly complex utterances.
However, a single model is still typically trained and
deployed for each task separately, requiring labeled
training data for each, which makes it challenging to
support new tasks, even within a single business
vertical (e.g., food-ordering or travel booking). In this
paper we describe Cross-TOP (Cross-Schema Task-
Oriented Parsing), a zero-shot method for complex
semantic parsing in a given vertical. By leveraging the
fact that user requests from the same vertical share
lexical and semantic similarities, a single crossschema
parser is trained to service an arbitrary
number of tasks, seen or unseen, within a vertical.
We show that Cross-TOP can achieve high accuracy
on a previously unseen task without requiring any
additional training data, thereby providing a scalable
way to bootstrap semantic parsers for new tasks. As
part of this work we release the FoodOrdering
dataset, a task-oriented parsing dataset in the foodordering
vertical, with utterances and annotations
derived from five schemas, each from a different
restaurant menu.

Cross-TOP: Zero-Shot Cross-Schema Task-Oriented Parsing

Pretrained language models represent the state of the art in NLP, but the successful construction of such models often requires large amounts of data and computational resources. Thus, the paucity of data for low-resource languages impedes the development of robust NLP capabilities for these languages. There has been some recent success in pretraining encoder-only models solely on a combination of low-resource African languages, exemplified by AfriBERTa. In this work, we extend the approach of “small data” pretraining to encoder–decoder models. We introduce AfriTeVa, a family of sequence-to-sequence models derived from T5 that are pretrained on 10 African languages from scratch. With a pretraining corpus of only around 1GB, we show that it is possible to achieve competitive downstream effectiveness for machine translation and text classification, compared to larger models trained on much more data. All the code and model checkpoints described in this work are publicly available at https://github.com/castorini/
afriteva

AfriTeVa: Extending “Small Data” Pretraining Approaches to Sequence-to-Sequence Models

Knowledge Graphs (KGs) are directed labeled graphs representing entities and the relationships between them. Most prior work focuses on supervised or semi-supervised approaches which require large amounts of annotated data. While unsupervised approaches do not need labeled training data, most existing methods either generate too many redundant relations or require manual mapping of the extracted relations to a known schema. To address these limitations, we propose an unsupervised method for KG generation that requires neither labeled data nor manual mapping to the predefined relation schema. Instead, our method leverages sentence-level semantic similarity for automatically generating relations between pairs of entities. Our proposed method outperforms two baseline systems when evaluated over four datasets.

Improving Generalization of Hate Speech Detection Systems to Novel Target Groups via Domain Adaptation

Downloads

Next from NAACL 2022

ErAConD: Error Annotated Conversational Dialog Dataset for Grammatical Error Correction

Similar lecture

Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Language

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Improving Generalization of Hate Speech Detection Systems to Novel Target Groups via Domain Adaptation

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from NAACL 2022

ErAConD: Error Annotated Conversational Dialog Dataset for Grammatical Error Correction

Similar lecture

Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Language

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads