Biomedical question-answering systems remain popular for biomedical experts interacting with the literature to answer their medical questions. However, these systems are difficult to evaluate in the absence of costly human experts. Therefore, automatic evaluation metrics are often used in this space.
Traditional automatic metrics such as ROUGE or BLEU, which rely on token overlap, have shown a low correlation with humans. 
We present a study that uses large language models (LLMs) to automatically evaluate systems from an international challenge on biomedical semantic indexing and question answering, called BioASQ. We measure the agreement of LLM-produced scores against human judgements. We show that LLMs correlate similarly to lexical methods when using basic prompting techniques. However, by aggregating evaluators with LLMs or by fine-tuning, we find that our methods outperform the baselines by a large margin, achieving a Spearman correlation of 0.501 and 0.511, respectively.

Using Large Language Models to Evaluate Biomedical Query-Focused Summarisation

Monitoring and predicting the expression of suicidal risk in individuals' social media posts is a central focus in clinical NLP. Yet, existing approaches frequently lack a crucial explainability component necessary for extracting evidence related to an individual's mental health state. We describe the CSIRO Data61 team's evidence extraction system submitted to the CLPsych 2024 shared task. The task aims to investigate the zero-shot capabilities of open-source LLM in extracting evidence regarding an individual's assigned suicide risk level from social media discourse. The results are assessed against ground truth evidence annotated by psychological experts, with an achieved recall-oriented BERTScore of 0.919. Our findings suggest that LLMs showcase strong feasibility in the extraction of information supporting the evaluation of suicidal risk in social media discourse. Opportunities for refinement exist, notably in crafting concise and effective instructions to guide the extraction process.

Exploring Instructive Prompts for Large Language Models in the Extraction of Evidence for Supporting Assigned Suicidal Risk Levels

A simple machine learning model of pluralisation as a linear regression problem minimising a p-adic metric substantially outperforms even the most robust of Euclidean-space regressors on languages in the Indo-European, Austronesian, Trans New-Guinea, Sino-Tibetan, Nilo-Saharan, Oto-Meanguean and Atlantic-Congo language families. There is insufficient evidence to support modelling distinct noun declensions as a p-adic neighbourhood even in Indo-European languages.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.4/)

Number Theory Meets Linguistics: Modelling Noun Morphology Across 1497 Languages Using 2-adic Metrics

*Coffee Breaks will be provided between 10:30 and 11:00 and 15:30 and 16:00.* To view the workshop program in detail, click [here](https://aclweb.org/aclwiki/BioNLP_Workshop) [![](https://assets.underline.io/markdown_image/1/image/12613369ab2f6dd75393ffa8794ccdd3.png)](https://app.gather.town/app/7Rf2IGR6LODvPyOx/ACL2024Workshops?spawnToken=slkmoOLzTrOqxSgtMXfG)

W20: BioNLP 2024 and Shared Tasks (BioNLP)

workshop paper

### Welcome!
The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. Our Virtual Poster Sessions will take place online Thursday, August 22, 2024.

You are required to register for this event. **Please register [here](https://2024.aclweb.org/registration). **

If you have already registered, please check your inbox for an email from Underline granting you access to ACL 2024 content.

Please register!

ACL 2024

The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. More information will be announced soon.

Since 2014, CLPsych has brought together researchers in computational linguistics and NLP, who use computational methods to better understand human language, infer meaning and intention, and predict individuals’ characteristics and potential behavior, with mental health practitioners and researchers, who are focused on psychopathology and neurological health and engage directly with the needs of providers and their patients. This workshop’s distinctly interdisciplinary nature has improved the exchange of knowledge, fostered collaboration, and increased the visibility of mental health as a problem domain in NLP.

Together, we hope to be able to advance the common goal of using human language as a tool to better understand emotional and mental state, and to reduce emotional suffering and the potential for self-harm. Now in its tenth year, the theme of this anniversary edition will be reflection: on the history of CLPsych, the progress in the field, and where new developments, both technological and in society, may lead us in the future. [**Read more**](https://clpsych.org/) [**Invited Speakers**](https://clpsych.org/program/) **Program Overview** 
08:45 - 09:00 Workshop Intro 
09:00 - 09:45 Keynote 1 
09:45 - 10:30 Shared Task Session 
10:30 - 11:00 Break 
11:00 - 11:45 Poster Session 
11:45 - 12:45 Paper Session 1 
12:45 - 14:00 Lunch 
14:00 - 15:00 Panel 
15:00 - 15:45 Keynote 2 
15:45 - 16:15 Break 
16:15 - 17:15 Paper Session 2 
17:15 - 17:30 Closing Remarks [**Detailed Program**](https://clpsych.org/program/)

W2: Ninth Workshop on Computational Linguistics and Clinical Psychology (CLPsych)

Welcome to the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL). Continuing its mission of expanding and involving the science community of all European countries, EACL has selected the Malta community for the 18th EACL. Considering the importance of physical interaction among researchers, the conference will be held at the Hotel Radisson Blu, St. Julians, in Malta, from 17 to 22 of March, 2024. The conference will also feature the possibility to participate virtually. As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language. 
#### [Conference Handbook](https://drive.google.com/file/d/13Wn38q0yev6U3RTgnf4Tn5T9GuAuZ-xW/view?usp=sharing) *downloadable*

To access the EACL 2024 event page you need to register [**here**](https://acl.swoogo.com/EACL2024). 

EACL 2024

As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

Please select the button above to join the live Poster Session in Gather.town.

Poster Session 2

poster

The AACL, Asia-Pacific Chapter of the Association for Computational Linguistics, and the AFNLP, Asian Federation of Natural Language Processing, invite you to participate in the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL) and the 10th International Joint Conference on Natural Language Processing (IJCNLP).

A registration is required to access this event. Access to the event site will be granted to all registered attendees on **November 16th**. If you have not already done so, please register using the link below:
 
 https://conference.iis.sinica.edu.tw/servlet/Register?ConferenceID=411

Registration Is Required

IJCNLP-AACL 2022

The AACL, Asia-Pacific Chapter of the Association for Computational Linguistics, and the AFNLP, Asian Federation of Natural Language Processing, invite you to participate in the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL) and the 10th International Joint Conference on Natural Language Processing (IJCNLP). 

Diego Molla

3

1

SHORT BIO

Presentations

Using Large Language Models to Evaluate Biomedical Query-Focused Summarisation

Exploring Instructive Prompts for Large Language Models in the Extraction of Evidence for Supporting Assigned Suicidal Risk Levels

Number Theory Meets Linguistics: Modelling Noun Morphology Across 1497 Languages Using 2-adic Metrics

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES