Large Language Models (LLMs) undergo extensive evaluation against various benchmarks collected in established leaderboards to assess their performance across multiple tasks. However, to the best of our knowledge, there is a lack of comprehensive studies evaluating these models' linguistic abilities independent of specific tasks. In this paper, we introduce a novel evaluation methodology designed to test LLMs' sentence generation abilities under specific linguistic constraints. Drawing on the `linguistic profiling' approach, we rigorously investigate the extent to which five LLMs of varying sizes, tested in both zero- and few-shot scenarios, effectively adhere to (morpho)syntactic constraints. Our findings shed light on the linguistic proficiency of LLMs, revealing both their capabilities and limitations in generating linguistically-constrained sentences.

Evaluating Large Language Models via Linguistic Profiling

Several studies in the literature on the interpretation of Neural Language Models (NLM) focus on the linguistic generalization abilities of pre-trained models. However, little attention is paid to how the linguistic knowledge of the models changes during the fine-tuning steps. In this paper, we contribute to this line of research by showing to what extent a wide range of linguistic phenomena are forgotten across 50 epochs of fine-tuning, and how the preserved linguistic knowledge is correlated with the resolution of the fine-tuning task. To this end, we considered a quite understudied task where linguistic information plays the main role, i.e.\ the prediction of the evolution of written language competence of native language learners. In addition, we investigate whether it is possible to predict the fine-tuned NLM accuracy across the 50 epochs solely relying on the assessed linguistic competence. Our results are encouraging and show a high relationship between the model's linguistic competence and its ability to solve a linguistically-based downstream task.

On the Nature of BERT: Correlating Fine-Tuning and Linguistic Competence

SESSIONE POSTER

Read abstract **[here](https://s3.amazonaws.com/pf-upload-01/u-59356/0/2021-08-06/4r13qhs/CfP--SLI2020--v21.pdf)**




Esplorazioni di treebank multilingue per studi tipologici

CItA e TIscrivo: corpora di testi scritti a scuola per osservare l'evoluzione delle competenze di scrittura in italiano L1

Read abstract **[here](https://s3.amazonaws.com/pf-upload-01/u-59356/0/2021-08-07/qp13q6j/SLI2020--MORI-VENTURI--BOOK-of-ABSTRACTS.pdf)**



Norme italiane di derivazione europea tra direttive dell’UE e leggi nazionali: un’analisi linguistico-computazionale della variazione intralinguistica

In this paper we investigate the linguistic knowledge learned by a Neural Language Model (NLM) before and after a fine-tuning process and how this knowledge affects its predictions during several classification problems. We use a wide set of probing tasks, each of which corresponds to a distinct sentence-level feature extracted from different levels of linguistic annotation. We show that BERT is able to encode a wide range of linguistic characteristics, but it tends to lose this information when trained on specific downstream tasks. We also find that BERT's capacity to encode different kind of linguistic properties has a positive influence on its predictions: the more it stores readable linguistic information of a sentence, the higher will be its capacity of predicting the expected label assigned to that sentence.

Linguistic Profiling of a Neural Language Model

This poster session includes Main Conference posters and Findings from the following areas:

Question Answering  • Computational Social Science and Cultural Analytics • Machine Translation • Resources and Evaluation

In-Person Poster Session C (Riverfront Hall)

poster

## Welcome to EMNLP 2024! 
We are excited to welcome you to one of the most prominent conferences in the field of Natural Language Processing. This year, EMNLP 2024 is being held in a hybrid format,
offering both virtual and in-person participation in beautiful Miami. Due to a record-breaking number of submissions, we've expanded the total number of accepted papers to accommodate more cutting-edge research from around the globe.
### [Conference Handbook](https://drive.google.com/file/d/1WPROgxjLAC96AJL7Ugy0tEnYm7dkrbHt/view?usp=sharing)

You are required to register for this event. **Please register [here](https://2024.emnlp.org/registration/).** The EMNLP 2024 event page on Underline will be open to public one week prior to the event.

Please register!

EMNLP 2024

EMNLP 2024 will take place in Miami, Florida from Nov 12th to Nov 16th, 2024, at the Hyatt Regency Miami Hote and on Underline for remote participants.

If you are a virtual attendee please click on the button above to join the live session **Chairs:** Santiago Egea, Simon Mille and Aswarth Abhilash Dara

VPS1 - Language Modeling

[![](https://assets.underline.io/uploads/markdown_image/1/image/81a3c0317d24f663b49b996875024d45.png)](https://aclanthology.org/volumes/2022.coling-1/)



### THE CONFERENCE WE KNOW AND WE WANT.
**COLING**, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes but also in industrial research departments including tech-startups. COLING provides opportunities for all these communities to showcase their exciting discovery.

In fall of 2022, COLING will be held in Gyeongju in a hybrid format. All participants can either present at the venue site or join virtually. As more people get vaccinated, we are happy to provide safer environments for our colleagues. We believe that COLING 2022 will be one of the conferences, free from the pandemic. The hybrid format gives presenters and sponsors a valuable opportunity to promote their companies in both an online and in-person venue. For the first time in a long time, customers can interact with their sponsor's products first-hand. The online venue, too, gives sponsors the chance to network with those unable to attend the in-person session.

The hybrid format ultimately lends itself to greater exposure for our sponsors: COLING2022 will let you reach more potential partners and customers than ever before!

To gain access to this event page you are required to register. Find more information and pay the registration fee on event the organizer’s website **[https://coling2022.org/reg](https://coling2022.org/reg)**

COLING 2022

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

**Chair:** Massimo Moneglia (Università di Firenze)

SESSIONE POSTER 

### Benvenuti 
al *LIV Congresso internazionale della Società di Linguistica Italiana ''Corpora e Studi Linguistici''* che si tiene qui in rete nella piattaforma fornita da Underline. Il Congresso si sviluppa su 3 giornate divise in:

* cinque Sessioni plenarie, 
* due Sessioni Demo dedicate ai corpora italiani e internazionali, 
* una Sessione Poster e 
* una Sessione workshop con 5 workshop in parallelo. 

Gli interventi orali saranno sia in diretta che preregistrati, secondo la volontà degli autori, ma per tutti sarà possibile l’interazione dal vivo con domande e risposte. A tutte le sessioni si accede attraverso le icone sulla sinistra di questa pagina. 
 Nelle sessioni Poster e Demo ci si muove virtualmente tra le proposte attraverso Gather-Town, che permette ai congressisti anche di incontrarsi in qualsiasi momento nella Lounge virtuale.
 Ringraziamo tutti per l’impegno e la pazienza e ci auguriamo una buona riuscita del Congresso. Un ringraziamento particolare ai relatori invitati Bas Aarts (University College London), Gu Yueguo (The Chinese Academy of Social Sciences) e Mirko Tavoni (Università di Pisa) 
 

**Welcome** to the *54th Congress of the International Society of Italian Linguistics ''Corpora and Linguistic Studies''*
which is held here through the Underline Platform. The Congress develops in 3 days which host:

* five Plenary sections, 
* two Demo sessions, 
* dedicated to Italian and international corpus building initiatives, 
* one Poster session and 
* one Workshops session hosting 5 workshops in parallel. 

Oral presentations are given in synchronous or are pre registered according to the will of each author, however Q/A will be live for all. All sessions can be accessed through the icons on the left side of this page. 
 In the Poster and Demo sessions you will be able to move among the various items though Gather-town. Congress participants can always meet in the Virtual lounge.
 We wish to thank all participants for their work and patience. A special thanks to our invited speakers Bas Aarts (University College London), Gu Yueguo (The Chinese Academy of Social Sciences) e Mirko Tavoni (Università di Pisa)

![](https://assets.underline.io/uploads/markdown_image/1/image/dacffc533f05582ca945abd01bb616ea.png)

If you have already registered, please **login** to Underline.io to be able to see full conference content. 
 In case you do not have an Underline account, please **create** one. Thank you!

LIV Congresso SLI 2021

**Chair:** Fracnesca Dovetto (Università di Napoli Federico II) **Title:** EvaLatin 1.0: un Corpus per la Valutazione delle Tecnologie del Linguaggio Applicate al Latino 
**Author:** Rachele Sprugnoli, Matteo Pellegrini, Marco Passarotti, Flavio Massimo Cecchini
(Università Cattolica del Sacro Cuore, Milano; Università di Bergamo) 


 **Title:** Esplorazioni di treebank multilingue per studi tipologici 
**Authors:** Chiara Alzetta, Felice Dell'Orletta, Simonetta Montemagni, Giulia Venturi
(ILC-CNR; Università di Genova) 

**Title:** Intonation of telephone conversations in a Customer Care Service 
**Author:** Philippe Martin (Université Paris Diderot)

SESSIONE PLENARIA 2

keynote

Workshop 1 Apprendere e insegnare: il ruolo dei corpora (GISCEL)

workshop paper

LONG6: Language Modelling 2

technical paper

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes, but also in industrial research departments and in technological startups. COLING conferences provide opportunities for all these communities to showcase their exciting developments.

COLING 2020

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

Giulia Venturi

7

36

1

SHORT BIO

Presentations

Evaluating Large Language Models via Linguistic Profiling

On the Nature of BERT: Correlating Fine-Tuning and Linguistic Competence

SESSIONE POSTER

Esplorazioni di treebank multilingue per studi tipologici

CItA e TIscrivo: corpora di testi scritti a scuola per osservare l'evoluzione delle competenze di scrittura in italiano L1

Norme italiane di derivazione europea tra direttive dell’UE e leggi nazionali: un’analisi linguistico-computazionale della variazione intralinguistica

Linguistic Profiling of a Neural Language Model

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES