Large language models (LLMs) with enormous pre-training tokens and parameters emerge diverse abilities, including math reasoning, code
generation, and instruction following. These abilities are further enhanced by supervised fine-tuning (SFT). While the open-source community has explored ad-hoc SFT for enhancing individual capabilities, proprietary LLMs exhibit versatility across various skills. Therefore, understanding the facilitation of multiple abilities via SFT is paramount. In this study, we specificially focuses on the interplay of data composition between mathematical reasoning, code generation, and general human-aligning abilities during SFT. We propose four intriguing research questions to explore the association between model performance and various factors including data amount, composition ratio, model size and SFT strategies. Our experiments reveal that distinct capabilities scale differently and larger models generally show superior performance with same amount of data. Mathematical reasoning and code generation consistently improve with increasing data amount, whereas general abilities plateau after roughly a thousand samples. Moreover, we observe data composition appears to enhance various abilities under limited data conditions, yet can lead to performance conflicts when data is plentiful. Our findings also suggest the amount of composition data influences performance more than the composition ratio. In analysis of SFT strategies, we find that sequentially learning multiple skills risks catastrophic forgetting. Our proposed Dual-stage Mixed Fine-tuning (DMT) strategy offers a promising solution to learn multiple abilities with different scaling patterns.

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition

In math reasoning with large language models (LLMs), fine-tuning data augmentation by query evolution and diverse reasoning paths is empirically verified effective, profoundly narrowing the gap between open-sourced LLMs and cutting-edge proprietary LLMs. 
In this paper, we conduct an investigation for such data augmentation in math reasoning and are intended to answer: 
(1) What strategies of data augmentation are more effective; 
(2) What is the scaling relationship between the amount of augmented data and model performance; and 
(3) Can data augmentation incentivize generalization to out-of-domain mathematical reasoning tasks?
To this end, we create two new dataset AugGSM8K and AugMATH, by complicating and diversifying the queries and sampling multiple reasoning paths from GSM8K and MATH.
We obtained a series of LLMs called MuggleMath by fine-tuning LLaMA models on AugGSM8K and AugMATH. MuggleMath substantially achieves new state-of-the-art on GSM8K and MATH.
A log-linear relationship and a segmented log-linear are presented between MuggleMath's performance and the amount of augmented data on GSM8K and MATH, respectively.
We also find that it is weak in out-of-domain math reasoning generalization from AugGSM8K to MATH and from AugMATH to GSM8K, which suggests that augmenting queries that cover a broader range of subjects is more beneficial for generalization.

MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning

Entities lie in the heart of biomedical natural language understanding, and the biomedical entity linking (EL) task remains challenging due to the fine-grained and diversiform concept names.
Generative methods achieve remarkable performances in general domain EL with less memory usage while requiring expensive pre-training.
Previous biomedical EL methods leverage synonyms from knowledge bases (KB) which is not trivial to inject into a generative method.
In this work, we use a generative approach to model biomedical EL and propose to inject synonyms knowledge in it.
We propose KB-guided pre-training by constructing synthetic samples with synonyms and definitions from KB and require the model to recover concept names.
We also propose synonyms-aware fine-tuning to select concept names for training, and propose decoder prompt and multi-synonyms constrained prefix tree for inference.
Our method achieves state-of-the-art results on several biomedical EL tasks without candidate selection which displays the effectiveness of proposed pre-training and fine-tuning strategies. The source code is available at \urlhttps://github.com/Yuanhy1997/GenBioEL.

Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning

In-domain language model pretraining has been shown to benefit various domain-specific down-stream tasks. In the biomedical domain, natural language generation (NLG) tasks such as medical dialogue system are of critical importance, while understudied. We emphasize the lack of in-domain generative language model and the unsystematic generative downstream benchmarks. We introduce the generative language model BioBART that adapts BART to the biomedical domain and collate various biomedical language generation tasks for model evaluation.

BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model

This poster session includes Main Conference posters, TACL and Demos.

In-Person Poster Session 2

poster

### Welcome!
The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. Our Virtual Poster Sessions will take place online Thursday, August 22, 2024.

You are required to register for this event. **Please register [here](https://2024.aclweb.org/registration). **

If you have already registered, please check your inbox for an email from Underline granting you access to ACL 2024 content.

Please register!

ACL 2024

The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. More information will be announced soon.

Virtual PS1 - 3G: Group 7

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

NAACL 2022

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

**Organizers:** Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, Jun-ichi Tsujii **Description:**
The BioNLP workshop associated with the ACL SIGBIOMED special interest group has established itself as the primary
venue for presenting foundational research in language processing for the biological and medical domains. Despite,
or maybe due to reaching maturity, the field of Biomedical NLP continues getting stronger. BioNLP welcomes and
encourages inclusion and diversity. BioNLP truly encompasses the breadth of the domain and brings together researchers
in bio- and clinical NLP from all over the world. The workshop will continue presenting work on a broad and interesting
range of topics in NLP. 
**Please visit our [website](https://aclweb.org/aclwiki/BioNLP_Workshop)**

(BioNLP) BioNLP 2022

workshop paper

# Welcome everyone to ACL 2022!

The 60th Annual Meeting of the Association for Computational Linguistics is taking place May 22-27, 2022 as a hybrid event, in Dublin and online. We are happy to welcome all of you to this anniversary edition with an almost 50-50 in-person and virtual participation. 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, three amazing keynote events and two new initiatives of invited talks: Spotlight Talks for Young Rising Stars (STIRS) and The Next Big Idea Talks. Posters (including Findings of ACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop will have an oral session and a poster session as part of Poster Session 1. The program also features eight Tutorials and 28 Workshops. 

 
We wish you a wonderful conference! 
[**The ACL 2022 Organizing Committee**](https://www.2022.aclweb.org/organisers)
 
[**Conference Handbook**](https://drive.google.com/file/d/1_BUCMfhMVrjG9E2e71aHdHeE28KSje0l/view?usp=sharing) 
[**Mini Handbook**](https://drive.google.com/file/d/1qlBKl0wzmlVF1oCeMQl3BahLd9nLP5Ce/view?usp=sharing) 
[**Posters and Demo guides**](https://drive.google.com/file/d/1UucMAoCNncIOaH1rMMDa0owuG9qgvJTG/view?usp=sharing)

ACL 2022

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing (NLP). 

Hongyi Yuan

4

6

1

SHORT BIO

Presentations

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition

MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning

Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning

BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES