Recent studies have shown that transformer models like BERT rely on number information encoded in their representations of sentences' subjects and head verbs when performing subject-verb agreement. However, probing experiments suggest that subject number is also encoded in the representations of all words in such sentences. In this paper, we use causal interventions to show that BERT only uses the subject plurality information encoded in its representations of the subject and words that agree with it in number. We also demonstrate that current probing metrics are unable to determine which words' representations contain functionally relevant information. This both provides a revised view of subject-verb agreement in language models, and suggests potential pitfalls for current probe usage and evaluation.

The Functional Relevance of Probed Information: A Case Study

We report the results of the SemEval 2022 Task
3, PreTENS, on evaluation the acceptability
of simple sentences containing constructions
whose two arguments are presupposed to be
or not to be in an ordered taxonomic relation.
The task featured two sub-tasks articulated as:
(i) binary prediction task and (ii) regression
task, predicting the acceptability in a continu-
ous scale. The sentences were artificially gen-
erated in three languages (English, Italian and
French). 21 systems, with 8 system papers
were submitted for the task, all based on var-
ious types of fine-tuned transformer systems,
often with ensemble methods and various data
augmentation techniques. The best systems
reached an F1-macro score of 94.49 (sub-task1)
and a Spearman correlation coefficient of 0.80
(sub-task2), with interesting variations in spe-
cific constructions and/or languages.

PreTENS — EvaluatingNeural Networks on Presuppositional Semantic Knowledge

Poster Session 4

poster

### Welcome to the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL) 
Continuing its mission of expanding and involving the science community of all European countries, EACL had selected the Ukrainian community for the 16th EACL, which took place online due to the COVID pandemic. Unfortunately, the ongoing war made the organisation in Kyiv impossible. Considering the importance of physical interaction among researchers, especially after the restrictions imposed by the COVID pandemic, in addition to an online mode, the conference will be held in Dubrovnik, Croatia, from 2 to 6 of May, 2023. The original aim of strengthening the connection with the Ukrainian community will not change as our program will feature a dedicated session and a workshop to highlight work on Ukrainian language technologies. As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

You need to register for the conference in order to access this site. Please visit https://2023.eacl.org/registration for more information.

EACL 2023

As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

**[Please visit our website.](https://semeval.github.io/SemEval2022/)**

W25: SemEval-2022: 15th International Workshop on Semantic Evaluation

workshop paper

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

NAACL 2022

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

Roberto Zamparelli

2

SHORT BIO

Presentations

The Functional Relevance of Probed Information: A Case Study

PreTENS — EvaluatingNeural Networks on Presuppositional Semantic Knowledge

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES