Thailand

A growing body of work makes use of probing in order to investigate the working of neural models, often considered black boxes. Recently, an ongoing debate emerged surrounding the limitations of the probing paradigm. In this work, we point out the inability to infer behavioral conclusions from probing results, and offer an alternative method that focuses on how the information is being used, rather than on what information is encoded. Our method, Amnesic Probing, follows the intuition that the utility of a property for a given task can be assessed by measuring the influence of a causal intervention that removes it from the representation. Equipped with this new analysis tool, we can ask questions that were not possible before, for example, is part-of-speech information important for word prediction? We perform a series of analyses on BERT to answer these types of questions. Our findings demonstrate that conventional probing performance is not correlated to task importance, and we call for increased scrutiny of claims that draw behavioral or causal conclusions from probing results.

ACL 2021

Amnesic Probing: Behavioral Explanation With Amnesic Counterfactuals

amnesic-probing

intervention

causality

interpretation

probing

analysis

**Welcome to ACL-IJCNLP 2021!**

The great event is jointly organized by the Association for Computational Linguistics (ACL) and Asian Federation of Natural Language Processing (AFNLP). 

This Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021) was to be held in Bangkok, Thailand, during August 1-6, 2021 and is now virtual on the Underline platform.

As in previous years, the program of the conference includes a poster session, tutorials, workshops and demonstrations in addition to the main conference.


We were able to keep the registration fees similar to those charged for the virtual ACL 2020. The one fee allows attendance at the main conference and any/all tutorials and workshops. These fees would be $125 Regular Early and $175 Regular Late; $50 Student Early and $75 Student Late. Early registration closes at midnight July 11, 2021 (Eastern Daylight Time).

**Reminder:** It is ACL’s policy that at least one author of each accepted paper (including ACL Finding papers) must register for the conference.

**Reminder2:** Underline site will open closer to the event. If you already registered you will receive access detail

Registration is now open

poster

Language Model Evaluation Beyond Perplexity

Learning to Explain: Generating Stable Explanations Fast

Alignment Rationale for Natural Language Inference

Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operators

On Sample Based Explanation Methods for NLP: Faithfulness, Efficiency and Semantic Evaluation

CausaLM: Causal Model Explanation Through Counterfactual Language Models

Apple Core-dination: Linguistic Feedback and Learning in a Speech-to-Action Shared World Game

The Reading Machine: a Versatile Framework for Studying Incremental Parsing Strategies

Levi Graph AMR Parser using Heterogeneous Attention

Translate, then Parse! A strong baseline for Cross-Lingual AMR Parsing

workshop paper

https://d3smihljt9218e.cloudfront.net/transcript/17531/subtitle/c2de0198df4a6954c9817d61fec5aa6e.vtt

English (automatic)

Amnesic Probing: Behavioral Explanation With Amnesic Counterfactuals

Downloads

Next from ACL 2021

Language Model Evaluation Beyond Perplexity

Similar lecture

Infusing Finetuning with Semantic Dependencies

VIDEO LIBRARY

EVENTS

COMPANY

RESOURCES