Mathematical reasoning skills are essential for general-purpose intelligent
systems to perform tasks from grocery shopping to climate modeling.
Towards evaluating and improving AI systems in this domain, we propose
LILA, a unified mathematical reasoning benchmark consisting of 23 diverse
tasks along four dimensions:
(i) mathematical abilities e.g., arithmetic, calculus 
(ii) language format e.g., question-answering, fill-in-the-blanks 
(iii) language diversity e.g., no language, simple language 
(iv) external knowledge e.g., commonsense, physics. 
We construct our benchmark by extending 20 datasets benchmark 
by collecting task instructions and solutions in the form of Python programs,
thereby obtaining explainable solutions 
in addition to the correct answer.
We additionally introduce two evaluation datasets 
to measure out-of-distribution performance 
and robustness to language perturbation.
Finally, we introduce BHASKARA,
a general-purpose mathematical reasoning model trained on LILA. 
Importantly, we find that multi-tasking leads to significant improvements 
(average relative improvement of 21.83% F1 score vs. single-task models),
while the best performing model only obtains 60.40%,
indicating the room for improvement 
in general mathematical reasoning and understanding.

LILA: A Unified Benchmark for Mathematical Reasoning

Targeted syntactic evaluations have demonstrated the ability of language models to perform subject-verb agreement given difficult contexts. 
To elucidate the mechanisms by which the models accomplish this behavior, this study applies causal mediation analysis to pre-trained neural language models.
We investigate the magnitude of models' preferences for grammatical inflections, as well as whether neurons process subject-verb agreement similarly across sentences with different syntactic structures.
We uncover similarities and differences across architectures and model sizes—notably, that larger models do not necessarily learn stronger preferences.
We also observe two distinct mechanisms for producing subject-verb agreement depending on the syntactic structure of the input sentence.
Finally, we find that language models rely on similar sets of neurons when given sentences with similar syntactic structure.

Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models

**Topics:** Natural Language Generation 
Resources and Evaluation 
Speech, Vision, Robotics, Multimodal Grounding 
Syntax, Parsing and their Applications 
Theme Track

Poster Session 6

poster

## Welcome to EMNLP 2022!
I am delighted to welcome you to EMNLP 2022! I believe this conference has been complicated beyond any precedent. Over the past year, it’s been thrilling to see the organization team approach each new puzzle with creativity and enthusiasm. We hope that those participating in Abu Dhabi as well as those joining remotely will leave the conference feeling newly inspired by the program and newly connected to our ever-growing community. Following EMNLP 2021 and major NLP conferences since, EMNLP 2022 is “hybrid,” serving both virtual and in-person participants.

Our key innovations for EMNLP 2022 include:

* EMNLP 2022 is “hybrid” in a second sense, as well: we allowed both direct and rolling review paper submissions, building on the pilot experiment of EMNLP 2021, which considered a small number of ARR submissions. 
* Familiar from NAACL but new to EMNLP, we’ve added an industry track.
* During the conference, “portals” will link virtual poster sessions to in-person conference participants during poster sessions each day.
* The first ACL-family conference in the United Arab Emirates.

 *Message from Noah A. Smith, University of Washington and Allen Institute for AI, Seattle, Washington, USA* 
***EMNLP 2022 General Chair***
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/9eec7d4a287ee18c278b08229290aa83.png)](https://drive.google.com/file/d/1OlPv6QBeo62VVTughj2jkiLeyHd1WnUt/view)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/a3db7a768409f05192210d98601edb25.png)](https://emnlp2022.rocket.chat/)

To access this site you need to register. Please register [here](https://2022.emnlp.org/registration/).

Register here

EMNLP 2022

Welcome!
EMNLP 2022 will take place in Abu Dhabi from December 7th to December 11th, 2022. And it will be held in hybrid mode, both online and offline.

**Session Chair: ** Niranjan Balasubramanian 
**Session Volunteer: ** Mohadese Bastan

4D: Interpretability and Analysis of Models for NLP #3

technical paper

**Welcome to ACL-IJCNLP 2021!**

The great event is jointly organized by the Association for Computational Linguistics (ACL) and Asian Federation of Natural Language Processing (AFNLP). 

As in previous years, the program of the conference includes a poster session, tutorials, workshops and demonstrations in addition to the main conference.


We were able to keep the registration fees similar to those charged for the virtual ACL 2020. The one fee allows attendance at the main conference and any/all tutorials and workshops. These fees would be $125 Regular Early and $175 Regular Late; $50 Student Early and $75 Student Late. Early registration closes at midnight July 11, 2021 (Eastern Daylight Time).

**Reminder:** It is ACL’s policy that at least one author of each accepted paper (including ACL Finding papers) must register for the conference.

**Reminder2:** Underline site will open closer to the event. If you already registered you will receive access detail

Registration is now open

IJCNLP-AACL 2021

The great event is jointly organized by the Association for Computational Linguistics (ACL) and Asian Federation of Natural Language Processing (AFNLP).

Matthew Finlayson

2

3

SHORT BIO

Presentations

LILA: A Unified Benchmark for Mathematical Reasoning

Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES