Singapore

We train a transformer network on the task of performing additions of two integers, achieving near perfect test-set accuracy with just 8000 positive training examples. We compute both learning curves (accuracy as a function of training set size) and model selection curves (accuracy as a function of embedding dimension and of number of layers). We observe that the number of dimension plays the biggest role, whereas the number of layers does not make a big difference after four layers. While the network generalises well from training to testing set, we did not observe any transfer-learning benefits when learning subtraction, having to relearn it from scratch. Current work involves probing the attention heads to understand the different roles played by positional embedding and the digit embedding (our conjecture is that query and key should be attending to position, while value should be attending to the actual digit). This distinction would not be possible when analysing words.

EMNLP 2023

Learning to Add as a way to Understand Transformers

maths

transformers

workshop paper

**Welcome to EMNLP 2023!**

On behalf of the EMNLP 2023 Organizing Committee, I extend a warm and heartfelt welcome to all of you to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). It is with immense pleasure and excitement that we gather here in Singapore, a vibrant hub of innovation and technological advancement.

The conference program is packed with insightful presentations, thought-provoking workshops, and engaging networking opportunities. In addition to the technical sessions, we have also planned several social events that will provide you with the opportunity to connect with your colleagues.

The conference is held in person at the Resorts World Convention Centre in Singapore, and available on-line with the help of Underline.

For those in Singapore, I hope that you will find time to explore the exciting Sentosa Island, Gardens by the Bay, Singapore Botanic Gardens and the many other attractions unique to Singapore.

Of course, we are grateful to our sponsors and partners for their generous support of EMNLP 2023, whose contributions make it possible for us to host this world-class event.

Yuji Matsumoto (RIKEN AIP) 
EMNLP 2023 General Chair

To access the **EMNLP 2023** event page on Underline, you need to register for the Conference. 
Please follow **[this link](https://2023.emnlp.org/registration/)** for more details.

EMNLP 2023 took place in Singapore from Dec 6th to Dec 10th, 2023.

Examples are a powerful tool that help us understand complex concepts and connections. In computational linguistics research, looking at example system output and example corpus entries can offer a wealth of insights that are not otherwise accessible. This paper describes the open-source software VULCAN, a visualization tool for strings, graphs, trees, alignments, attention and more. 
VULCAN's unique ability to visualize both linguistic structures and properties of neural models make it particularly relevant for neuro-symbolic models. Neuro-symbolic models, combining neural networks with often linguistically grounded structures, offer a promise of increased interpretability in an age of purely neural black-box end-to-end models. VULCAN aims to facilitate this interpretability in practice.
VULCAN is designed to be both easy to use and powerful in its capabilities.

Introducing VULCAN: A Visualization Tool for Understanding our Models and Data by Example

Flexible Model Interpretability through Natural Language Model Editing

Several methodologies have recently been proposed to evaluate the ability of Pretrained Language Models (PLMs) to interpret negation. In this article, we build on Gubelmann and Hand- schuh (2022), which studies the modification of PLMs’ predictions as a function of the polarity of inputs, in English. Crucially, this test uses “self-contained” inputs ending with a masked position: depending on the polarity of a verb in the input, a particular token is either semantically ruled out or allowed at the masked position. By replicating Gubelmann and Handschuh (2022) experiments, we have uncovered flaws that weaken the conclusions that can be drawn from this test. We thus propose an improved version, the Self-Contained Neg Test, which is more controlled, more systematic, and entirely based on examples forming minimal pairs varying only in the presence or absence of verbal negation in English.
When applying our test to the roberta and bert base and large models, we show that only roberta-large shows trends that match the expectations, while bert-base is mostly insensitive to negation. For all the tested models though, in a significant number of test instances the top-1 prediction remains the token that is semantically forbidden by the context, which shows how much room for improvement remains for a proper treatment of the negation phenomenon.

The Self-Contained Negation Test Set

Investigating the Effect of Discourse Connectives on Transformer Surprisal: Language Models Understand Connectives, Even So They Are Surprised

Collostructional analysis is a technique devised to find correlations between particular words and linguistic constructions in order to analyse meaning associations of these constructions. Contrasting collostructional analysis results with output from BERT might provide insights into the way BERT represents the meaning of linguistic constructions. This study tests to what extent English BERT's meaning representations correspond to known constructions from the linguistics literature by means of two tasks that we propose. Firstly, by predicting the words that can be used in open slots of constructions, the meaning associations of more lexicalized constructions can be observed. Secondly, by finding similar sequences using BERT's output embeddings and manually reviewing the resulting sentences, we can observe whether instances of less lexicalized constructions are clustered together in semantic space. These two methods show that BERT represents constructional meaning to a certain extent, but does not separate instances of a construction from a near-synonymous construction that has a different form.

Using Collostructional Analysis to evaluate BERT's representation of linguistic constructions

In this work-in-progress, we investigate the multi-step reasoning mechanisms of
transformer-based language models by adopting a reverse-engineering approach, inspired by work on mechanistic interpretability. To this end, we study a 6-layer transformer trained on a simplified tree traversal task. We find that the model forms backtracking heads which allow the model to backtrack a certain number of steps from the goal node. In addition, we find that the MLPs in the final layers are cru-
cial for finding paths when the goal is further away, implying that the model combines different internal mechanisms to solve even simple algorithmic tasks.

Planning in Transformers: Evidence for Backtracking Circuits in Models Trained on Tree Traversal

Probing contextualized representations typically involves comparing task-specific model predictions against ground truth linguistic labels. Although this methodology shows what information can be recovered by a classifier, it does not reveal how a classifier uses the representation to make its decision. To address the latter problem, we ask: Do task-classifiers rely on representation- and task-independent geometric patterns in the embedding space? We explore this question by developing METAPROBE, an approach that uses geometric properties of representations to predict the behavior of task-specific classifiers (i.e., their predictions as opposed to the ground truth). Our experiments reveal the existence of universal geometric patterns across representations that can predict classifier predictions. Consequently, this allows us to posit a geometric explanation for the impressive performance of contextualized representations.

METAPROBE: A Representation- and Task-Agnostic Probe

Measuring Moral Inconsistencies of Large Language Models

Contrast set consistency is a robustness measurement that evaluates the rate at which a model correctly responds to all instances in a bundle of minimally different examples relying on the same knowledge. To draw additional insights, we propose to complement consistency with relative consistency -- the probability that an equally accurate model would surpass the consistency of the proposed model, given a distribution over possible consistencies. Models with 100% relative consistency have reached a consistency peak for their accuracy. We reflect on prior work that reports consistency in contrast sets and observe that relative consistency can alter the assessment of a model's consistency compared to another. We anticipate that our proposed measurement and insights will influence future studies aiming to promote consistent behavior in models.

How Much Consistency Is Your Accuracy Worth?

Humans exhibit strong preferences in syntactic alternations based on availability effects, where more 'available' constituents are ordered earlier in a sentence. Here, I show that autoregressive LMs learn these preferences both from training data and from architectural considerations on memory. I demonstrate that constraining context window size in both LSTM RNNs and n-gram models induces a preference towards producing short constituents before long ones. In particular, this has implications for incremental models of human language production.

Premium content

Learning to Add as a way to Understand Transformers

Next from EMNLP 2023

Introducing VULCAN: A Visualization Tool for Understanding our Models and Data by Example

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES