Canada

Automatically rating the quality of published
research is a critical step in medical evidence
synthesis. While several methods have been
proposed, their algorithmic fairness has been
overlooked, even though significant risks may
result when such systems are deployed in
biomedical contexts. In this work, we study
the fairness of two systems with respect to two
sensitive attributes: participant sex and med-
ical area. In some cases, we find important
inequalities, leading us to apply various debi-
asing methods. Upon examining the interplay
of predictive performance and fairness, as well
as medically-critical selective classification ca-
pabilities and calibration performance, we find
that it is possible to improve fairness through
debiasing, but often at a cost to other perfor-
mance measures.

ACL 2023

Promoting Fairness in Classification of Quality of Medical Evidence

biomedical research

evidence synthesis

debiasing

fairness

workshop paper

### Welcome to ACL 2023, the 61st Annual Meeting of the Association for Computational Linguistics! 
 The conference will be held in Toronto, Canada, July 9-14, 2023. 
Following the succession of the recent conferences in our field, ACL 2023 will adopt a hybrid format.
While the impact of Covid has considerably diminished in terms of traveling, obtaining visas to Canada
entails a very long process. Moreover, the global economic conditions pose challenges for many individuals to travel to conferences. Recognizing these circumstances, we know many participants may not be
able to attend the conference in person. Therefore, we are committed to providing a great virtual platform
so everyone has the opportunity to interact with other participants and enjoy the conference. Based on the
current registered participants, approxiately 30% have chosen to attend the conference virtually. Whether
you join us in person or virtually, we sincerely hope everyone has a remarkable conference experience. 
This General Chair’s message is where I express my gratitude to the many individuals who have made
enormous contributions to the conference over the past year.

Read [**ACL 2023 General Chair's message**](https://docs.google.com/document/d/1WobYM7norbG4dI48s75HfJoD89qgX5a_F-6U8AteLSA/edit?usp=sharing/) in full.

##### **[Conference Handbook](https://2023.aclweb.org/downloads/acl2023-handbook.pdf)**

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing.

We consider the task of automatically extracting various overlapping frames, i.e, structured entities composed of multiple labels and mentions, from long clinical breast radiology documents. 
While many methods exist for related topics such as event extraction, slot filling, or discontinuous entity recognition, a challenge in our study resides in the fact that clinical reports typically contain overlapping frames that span multiple sentences or paragraphs.
We propose a new method that addresses these difficulties and evaluate it on a new annotated corpus.
Despite the small number of documents, we show that the hybridization between knowledge injection and a learning-based system allows us to quickly obtain proper results.
We will also introduce the concept of scope relations and show that it both improves the performance of our system, and provides a visual explanation of the predictions.


An end-to-end neural model based on cliques and scopes for frame extraction in long breast radiology reports

In this presentation, we will talk about optimal approaches to pre-train biomedical language models and evaluate their performance on the Named Entity Recognition task. Moreover, we will present our proposed method of contextualized weight distillation used to introduce domain-specific vocabulary in the continued pre-training of language models. We will also discuss how using Curriculum learning when pre-training the BIOptimus model helped us to guide the model's learning process more smoothly and introduce the biomedical language model to a broader scope of language experience.

BIOptimus: Pre-training an Optimal Biomedical Language Model with Curriculum Learning for Named Entity Recognition

Fine-tuning biomedical pre-trained language models (BioPLMs) such as BioBERT has become a common practice dominating leaderboards across various natural language processing tasks. Despite their success and wide adoption, prevailing fine-tuning approaches for named entity recognition (NER) naively train BioPLMs on targeted datasets without considering class distributions. This is problematic especially when dealing with imbalanced biomedical gold-standard datasets for NER in which most biomedical entities are underrepresented. In this talk, we discuss our work on addressing the class imbalance problem and propose a weighted loss trainer (WeLT), as a cost-sensitive fine-tuning approach based on new re-scaled class weights for the task of biomedical NER.


WeLT: Improving Biomedical Fine-tuned Pre-trained Language Models with Cost-sensitive Learning

This paper presents the results of the shared task on Lay Summarisation of Biomedical Research Articles (BioLaySumm), hosted at the BioNLP Workshop at ACL 2023. The goal of this shared task is to develop abstractive summarisation models capable of generating ``lay summaries'' (i.e., summaries that are comprehensible to non-technical audiences) in both a controllable and non-controllable setting. There are two subtasks: 1) Lay Summarisation, where the goal is for participants to build models for lay summary generation only, given the full article text and the corresponding abstract as input; and 2) Readability-controlled Summarisation, where the goal is for participants to train models to generate both the technical abstract and the lay summary, given an article’s main text as input. In addition to overall results, we report on the setup and insights from the BioLaySumm shared task, which attracted a total of 20 participating teams across both subtasks.

BioLaySumm 2023 Shared Task: Lay Summarisation of Biomedical Research Articles

This paper proposes a new document classification method that incorporates the representations of a literature graph created from bibliographic and entity information.
Recently, document classification performance has been significantly improved with large pre-trained language models; however, there still remain documents that are difficult to classify. External information, such as bibliographic information, citation links, descriptions of entities, and medical taxonomies, has been considered one of the keys to dealing with such documents in document classification. 
Although several document classification methods using external information have been proposed, they only consider limited relationships, e.g., word co-occurrence and citation relationships. However, there are multiple types of external information.
To overcome the limitation of the conventional use of external information, we propose a document classification model that simultaneously considers bibliographic and entity information to deeply model the relationships among documents using the representations of the literature graph.
The experimental results show that our proposed method outperforms existing methods on two document classification datasets in the biomedical domain with the help of the literature graph. Our source code is publicly available at https://github.com/tticoin/BDCL-LitGraph.

Biomedical Document Classification with Literature Graph Representations of Bibliographies and Entities

Radiology report summarization is a growing area of research. Given the Findings and/or Background sections of a radiology report, the goal is to generate a summary (called an Im- pression section) that highlights the key ob- servations and conclusions of the radiology study. Recent efforts have released systems that achieve promising performance as mea- sured by widely used summarization metrics such as BLEU and ROUGE. However, the research area of radiology report summariza- tion currently faces two important limitations. First, most of the results are reported on private datasets. This limitation prevents the ability to reproduce results and fairly compare different systems and solutions. Secondly, to the best of our knowledge, most research is carried out on chest X-rays. To palliate these two limitations, we propose a radiology report summarization (RadSum) challenge on i) a new dataset of eleven different modalities and anatomies pairs based on the MIMIC-III database ii) a multimodal report summarization dataset based on MIMIC-CXR enhanced with a brand-new test-set from Stanford Hospital. In total, we received 112 submissions across 11 teams.

Overview of the RadSum23 Shared Task on Multi-modal and Multi-anatomical Radiology Report Summarization

We describe the participation of team e-Health CSIRO in the BioNLP RadSum task of 2023. This task aims to develop automatic summarisa- tion methods for radiology. The subtask that we participated in was multimodal; the impression section of a report was to be summarised from a given findings section and set of Chest X-rays (CXRs) of a subject’s study. For our method, we adapted an encoder-to-decoder model for CXR report generation to the subtask. e-Health CSIRO placed seventh amongst the participat- ing teams with a RadGraph ER F1 score of 23.9.

Sentiment-guided Transformer with Severity-aware Contrastive Learning for Depression Detection on Social Media

Social media (SM) can provide valuable information about patients' experiences with multiple drugs during treatments. Although information extraction from SM has been well-studied, drug switches detection and reasons behind these switches from SM have not been studied yet. Therefore, in this paper, we present a new SM listening approach for analyzing online patient conversations that contain information about drug switching, drug effectiveness, side effects, and adverse drug reactions. We describe a deep learning-based approach for identifying instances of drug switching in SM posts, as well as a method for extracting the reasons behind these switches. To train and test our models, we used annotated SM data from internal dataset which is automatically created using a rule-based method. We evaluated our models using Text-to-Text Transfer Transformer (T5) and found that our SM listening approach can extract medication change information and reasons with high accuracy, achieving an F1-score of 98% and a ROUGE-1 score of 93%, respectively. Overall, our results suggest that our SM listening approach has the potential to provide valuable insights into patients' experiences with drug treatments, which can be used to improve patient outcomes and the effectiveness of drug treatments.

Exploring Drug Switching in Patients: A Deep Learning-based Approach to Extract Drug Changes and Reasons from Social Media


The use of seed articles in information retrieval provides many advantages, such as a longer context and more details about the topic being searched for. Given a seed article (i.e., a PMID), PubMed provides a pre-compiled list of similar articles to support the user in finding equivalent papers in the biomedical literature. We aimed at performing a quantitative evaluation of the PubMed Similar Articles based on three existing biomedical text similarity datasets, namely, RELISH, TREC-COVID, and SMAFIRA-c. Further, we carried out a survey and an evaluation of various text similarity methods on these three datasets. Our experiments considered the original title and abstract from PubMed as well as automatically detected sections and manually annotated relevant sentences. We provide an overview about which methods better perform for each dataset and compare them to the ranking in PubMed similar articles. While results varied considerably among the datasets, we were able to obtain a better performance than PubMed for all of them. Datasets and source codes are available at: https://github.com/mariananeves/reranking

Is the ranking of PubMed similar articles good enough? An evaluation of text similarity methods for three datasets

We propose a novel distantly supervised document-level biomedical relation extraction model that uses partial knowledge graphs that include the graph neighborhood of the entities appearing in each input document. 
Most conventional distantly supervised relation extraction methods use only the entity relations automatically annotated by using knowledge base entries. They do not fully utilize the rich information in the knowledge base, such as entities other than the target entities and the network of heterogeneous entities defined in the knowledge base.
To address this issue, our model integrates the representations of the entities acquired from the neighborhood knowledge graphs with the representations of the input document. 
We conducted experiments on the ChemDisGene dataset using Comparative Toxicogenomics Database (CTD) for document-level relation extraction with respect to interactions between drugs, diseases, and genes. 
Experimental results confirmed the performance improvement by integrating entities and their neighborhood biochemical information from the knowledge base.

Premium content

Promoting Fairness in Classification of Quality of Medical Evidence

Downloads

Next from ACL 2023

An end-to-end neural model based on cliques and scopes for frame extraction in long breast radiology reports

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES