Previous studies have argued that pre-trained language models encode commonsense relational knowledge (e.g. that apples are edible). However, simultaneous work has revealed that such models are often insensitive to context, even ignoring overt contextual cues such as negations. In this paper, we investigate whether masked language models (the BERT family), can move beyond naive associative biases (e.g., apple -> edible) when the context warrants (e.g. ranking inedible higher when presented with the information that the apple is rotten). We introduce the WinoVenti procedure, which adversarially exploits associative biases in masked language models to create model-specific Winograd-style entailment schemas. Using our constructed WinoVenti challenges set of over 2000 schemas, we discover that language models in the BERT family experience a steep drop in performance on prompts that require them to pick answers which require reasoning about context (e.g., from 89.8% to 18.4% for BERT Large). We present evidence to show that language models exhibit different associative biases, suggesting a need for future work in developing and analyzing frameworks similar to WinoVenti that are tuned to model-specific weaknesses.

Are Rotten Apples Edible? Challenging Commonsense Inference Ability with Exceptions

Findings Papers - Interpretability and Analysis of Models for NLP

technical paper

**Welcome to ACL-IJCNLP 2021!**

The great event is jointly organized by the Association for Computational Linguistics (ACL) and Asian Federation of Natural Language Processing (AFNLP). 

As in previous years, the program of the conference includes a poster session, tutorials, workshops and demonstrations in addition to the main conference.


We were able to keep the registration fees similar to those charged for the virtual ACL 2020. The one fee allows attendance at the main conference and any/all tutorials and workshops. These fees would be $125 Regular Early and $175 Regular Late; $50 Student Early and $75 Student Late. Early registration closes at midnight July 11, 2021 (Eastern Daylight Time).

**Reminder:** It is ACL’s policy that at least one author of each accepted paper (including ACL Finding papers) must register for the conference.

**Reminder2:** Underline site will open closer to the event. If you already registered you will receive access detail

Registration is now open

IJCNLP-AACL 2021

The great event is jointly organized by the Association for Computational Linguistics (ACL) and Asian Federation of Natural Language Processing (AFNLP).

Nam Do

1

2

SHORT BIO

Presentations

Are Rotten Apples Edible? Challenging Commonsense Inference Ability with Exceptions

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES