United States

2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics.

**Whova App** 
Stay in touch with your fellow conference attendees via the [Whova App](https://whova.com/portal/webapp/nacon_202106/)

**Conference Structure**
https://2021.naacl.org/blog/conference-structure/

**Walkthrough video of how to NAACL 2021** 

Please take a moment to view this video explaining how to navigate the platform, attend sessions network with other attendees. 


<figure class="video_container">
  <iframe src="https://screencast-o-matic.com/watch/crhwbGVh3vx?v=6&ff=1&title=0&controls=1" width=640  height=350  frameborder="0" allowfullscreen="true"> </iframe>
</figure>

<br>
<br>

NAACL 2021

Robustness and counterfactual bias are usually evaluated on a test dataset. However, are these evaluations robust? If the test dataset is perturbed slightly, will the evaluation results keep the same? In this paper, we propose a "double perturbation" framework to uncover model weaknesses beyond the test dataset. The framework first perturbs the test dataset to construct abundant natural sentences similar to the test data, and then diagnoses the prediction change regarding a single-word substitution. We apply this framework to study two perturbation-based approaches that are used to analyze models' robustness and counterfactual bias in English. (1) For robustness, we focus on synonym substitutions and identify vulnerable examples where prediction can be altered. Our proposed attack attains high success rates (96.0%-99.8%) in finding vulnerable examples on both original and robustly trained CNNs and Transformers. (2) For counterfactual bias, we focus on substituting demographic tokens (e.g., gender, race) and measure the shift of the expected prediction among constructed sentences. Our method is able to reveal the hidden model biases not directly shown in the test dataset. Our code is available at https://github.com/chong-z/nlp-second-order-attack.


Double Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluation

technical paper

Universal Adversarial Attacks with Natural Triggers for Text Classification

On the Impact of Random Seeds on the Fairness of Clinical Classifiers

An Empirical Investigation of Bias in the Multimodal Analysis of Financial Earnings Calls

Case Study: Deontological Ethics in NLP

On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning

Privacy Regularization: Joint Privacy-Utility Optimization in LanguageModels

Topic Model or Topic Twaddle? Re-evaluating Semantic Interpretability Measures

Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks

Discourse Probing of Pretrained Language Models

Learning to Learn to be Right for the Right Reasons

https://d3smihljt9218e.cloudfront.net/transcript/13451/subtitle/6a7bb751067d23501a793fde6723c3de.vtt

English (automatic)

Double Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluation

Downloads

Next from NAACL 2021

Universal Adversarial Attacks with Natural Triggers for Text Classification

Similar lecture

Interactive Plot Manipulation using Natural Language