Croatia

Citation screening is an essential yet time-consuming step of the systematic review process. Screening automation using machine learning techniques can speed up constructing a systematic review. Current efforts in automation are limited in their capabilities as they either only suggest the decisions or require large annotated datasets before making the predictions. In this PhD thesis, we propose several approaches to improving the screening automation techniques to reduce reviewers&#39; work. First, we want to propose a unified benchmarking method for the citation screening task. We will then explore different machine learning algorithms for conducting screening, focusing on using fewer annotated examples to maintain stable results across multiple reviews. Next, we intend to experiment with using external information coming from eligibility criteria sheets as a substitute for manual annotations. Finally, we would like to assess our approaches to literature reviews outside of the medical domain.

EACL 2023

Rapid Systematic Reviews: Zero-shot Citation Screening with the Usage of Eligibility Criteria

citation screening

systematic reviews

prompt-based learning

bionlp

evaluation

Citation screening is an essential yet time-consuming step of the systematic review process. Screening automation using machine learning techniques can speed up constructing a systematic review. Current efforts in automation are limited in their capabilities as they either only suggest the decisions or require large annotated datasets before making the predictions. In this PhD thesis, we propose several approaches to improving the screening automation techniques to reduce reviewers' work. First, we want to propose a unified benchmarking method for the citation screening task. We will then explore different machine learning algorithms for conducting screening, focusing on using fewer annotated examples to maintain stable results across multiple reviews. Next, we intend to experiment with using external information coming from eligibility criteria sheets as a substitute for manual annotations. Finally, we would like to assess our approaches to literature reviews outside of the medical domain.

poster

### Welcome to the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL) 
Continuing its mission of expanding and involving the science community of all European countries, EACL had selected the Ukrainian community for the 16th EACL, which took place online due to the COVID pandemic. Unfortunately, the ongoing war made the organisation in Kyiv impossible. Considering the importance of physical interaction among researchers, especially after the restrictions imposed by the COVID pandemic, in addition to an online mode, the conference will be held in Dubrovnik, Croatia, from 2 to 6 of May, 2023. The original aim of strengthening the connection with the Ukrainian community will not change as our program will feature a dedicated session and a workshop to highlight work on Ukrainian language technologies. As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

You need to register for the conference in order to access this site. Please visit https://2023.eacl.org/registration for more information.

As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

Examining a sample of Instant Messaging (IM) dialogues by conventional, manual methods, often used in social sciences, is difficult and inefficient for two reasons: a sample of significant size comprises a large quantity of unstructured text, and additionally, the occurrence of researched phenomena tends to be low. Therefore, automatic detection of particular psychological phenomena would enable the research of such data sources on a much larger scale than before. It will be the primary goal of the dissertation thesis to explore the possible Natural Language Processing (NLP) approaches to the problem and report their effectiveness. This proposal provides the theoretical basis by researching state-of-the-art solutions for related tasks, formulates hypotheses for the possible solutions, and presents the already achieved results. The proposed hypotheses will be verified in the dissertation thesis by designing, training, and evaluating predictive models. The resulting work will provide a comprehensive overview of different methods and their effectiveness. To the best of the author's knowledge, this thesis will be the first work to do so in this area on real IM dialogues provided by adolescents. The results could be used, for example, in parental applications, mental health applications that supervise users' well-being, risky online behavior detection, intent detection for chatbots, and many others. This proposal also describes a research plan for conducting the research and publishing the results.

Detecting Psychological Phenomena in Dialogues

Clinical natural language processing aims to tackle language and prediction tasks using text from medical practice, such as clinical notes, prescriptions, and discharge summaries. Several approaches have been tried to deal with these tasks. Since 2017, pre-trained language models (PLMs) have achieved state-of-the-art performance in many tasks. However, most works have been developed in English. This PhD research proposal addresses the development of PLMs for clinical NLP in Spanish. To carry out this study, we will build a clinical corpus big enough to implement a functional PLM. We will test several PLM architectures and evaluate them with language and prediction tasks. The novelty of this work lies in the use of only clinical text, while previous clinical PLMs have used a mix of general, biomedical, and clinical text.

Development of pre-trained language models for clinical NLP in Spanish

Generating polite responses is essential to build intelligent and engaging dialogue systems. However, this task is far from well-explored due to the difficulties of rendering a particular style in coherent responses, especially when parallel datasets for regular-to-polite pairs are usually unavailable. This paper proposes a polite chatbot that can produce responses that are polite and coherent to the given context. In this study, a politeness transfer model is first used to generate polite synthetic dialogue pairs of contexts and polite utterances. Then, these synthetic pairs are employed to train a dialogue model. Automatic and human evaluations demonstrate that our method outperforms baselines in producing polite dialogue responses while staying competitive in terms of coherent to the given context.

Polite Chatbot: A Text Style Transfer Application

Writing is an important element of language learning, and an increasing amount of learner writing is taking place in online environments. Teachers can provide valuable feedback by commenting on learner text. However, providing relevant feedback for every issue for every student can be time-consuming. To address this, we turn to the NLP subfield of feedback comment generation, the task of automatically generating explanatory notes for learner text with the goal of enhancing learning outcomes. 
However, freely-generated comments may mix multiple topics seen in the training data or even give misleading advice. In this thesis proposal, we seek to address these issues by categorizing comments and constraining the outputs of noisy classes. We describe an annotation scheme for feedback comment corpora using comment topics with a broader scope than existing typologies focused on error correction. We outline plans for experiments in grouping and clustering, replacing particularly diverse categories with modular templates, and comparing the generation results of using different linguistic features and model architectures with the original dataset versus the newly annotated one. This paper presents the first two years (the master's component) of a research project for a five-year combined master's and Ph.D program.

Template-guided Grammatical Error Feedback Comment Generation

While text-based medical applications have become increasingly prominent, access to clinical data remains a major concern. To resolve this issue, further de-identification and anonymization of the data are required. This might, however, alter the contextual information within the clinical texts and therefore influence the learning and performance of possible language models. 
This paper systematically analyses the potential effects of various anonymization techniques on the performance of state-of-the-art machine learning models based on several datasets corresponding to five different NLP tasks.
On this basis, we derive insightful findings and recommendations concerning text anonymization with regard to the performance of machine learning models. In addition, we present a simple re-identification attack applied to the anonymized text data, which can break the anonymization.

Clinical Text Anonymization, its Influence on Downstream NLP Tasks and the Risk of Re-Identification

Today, human assistants are often replaced by chatbots, designed to communicate via natural language, however, some disadvantages are notorious with this replacement. This PhD thesis project consists of researching, implementing,
and testing a solution for guiding the action of a human in a contact center. It will start with the discovery and creation of datasets in Portuguese. Next, it will go through three main components: Extraction for processing dialogs and using the information to describe interactions; Representation for discovering the most
frequent dialog flows represented by graphs; Guidance for helping the agent during a new dialog. These will be integrated in a single framework. In order to avoid service degradation resulting from the adoption of chatbots, this work aims to explore technologies in order to increase the efficiency of the human’s job without losing human contact.

Automatic Dialog Flow Extraction and Guidance

With the rise in larger language models, researchers started exploiting them by pivoting the downstream tasks as language modeling tasks using prompts. In this work, we convert the Named Entity Recognition task into a seq2seq task by generating the synthetic sentences using templates. Our main contribution is the conversion framework which provides faster inference. In addition, we test our method’s performance in resource-rich, low resource and domain transfer settings. Results show that our method achieves comparable results in the resource-rich setting and outperforms the current seq2seq paradigm state-of-the-art approach in few-shot settings. Through the experiments, we observed that the negative examples play an important role in model's performance. We applied our approach over BART and T5-base models, and we notice that the T5 architecture aligns better with our task. The work is performed on the datasets in English language.

Improving and Simplifying Template-Based Named Entity Recognition

Access to natural language processing resources is essential for their continuous improvement. This can be especially challenging in educational institutions where the software development effort required to package and release research outcomes may be overwhelming and under-recognized. Access to well-prepared and reliable research outcomes is important both for their developers as well as the greater research community. This presentation presents an approach to address this concern with two main goals: (1) to create an open-source easily deployable platform where resources can be easily shared and explored, and (2) to use this platform to publish open-source Turkish NLP resources (datasets and tools) created by a research lab. The Turkish Natural Language Processing (TULAP) was designed and developed as an easy-to-use platform to share dataset and tool resources which supports interactive tool demos. Numerous open access Turkish NLP resources have been shared on TULAP. All tools are containerized to support portability for custom use. This presentation describes the requirements, design, implementation, and deployment of TULAP with several use cases (available at https://tulap.cmpe.boun.edu.tr/). A short video demonstrating our system is available at https://figshare.com/articles/media/TULAP_Demo/22179047.

TULAP - An Accessible and Sustainable Platform for Turkish Natural Language Processing Resources Authors:

The methods used to create many of the wellknown Question-Answering (QA) datasets are hard to replicate for low-resource languages. A commonality amongst these methods is hiring annotators to source answers from the internet by querying a single answer source, such as Wikipedia. Applying these methods for lowresource languages can be problematic since there is no single large answer source for these languages. Consequently, this can result in a high ratio of unanswered questions, since the amount of information in any single source is limited. To address this problem, we developed a novel crowd-sourcing platform to gather multiple-domain QA data for low-resource languages. Our platform, which consists of a mobile app and a web API, gamifies the data collection process. We successfully released the app for Icelandic (a low-resource language with about 350,000 native speakers) to build a dataset which rivals large QA datasets for highresource languages both in terms of size and ratio of answered questions. We have made the platform open source with instructions on how to localize and deploy it to gather data for other low-resource languages.

GameQA: Gamified Mobile App Platform for Building Multiple-Domain Question-Answering Datasets

The paper addresses TermoUD - a language-independent terminology extraction tool. Its previous version, i.e. TermoPL, uses language dependent shallow grammar which selects candidate terms. The goal behind the development of TermoUD is to make the procedure as universal as possible, while taking care of the linguistic correctness of selected phrases. The tool is suitable for languages for which the Universal Dependencies (UD) parser exists. We describe a method of candidate term extraction based on UD POS tags and UD relations. The candidate ranking is performed by the C-value metric (contexts counting is adapted to the UD formalism), which doesn't need any additional language resources. The performance of the tool has been tested on texts in English, French, Dutch, and Slovenian. 
The results are evaluated on the manually annotated datasets: ACTER, RD-TEC 2.0, GENIA and RSDO5, and compared to those obtained by other tools.

Rapid Systematic Reviews: Zero-shot Citation Screening with the Usage of Eligibility Criteria

Downloads

Next from EACL 2023

Detecting Psychological Phenomena in Dialogues

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES