Croatia

Automatic error type classification is an important process in both learner corpora creation and evaluation of large-scale grammatical error correction systems. 
Rule-based classifier approaches such as ERRANT have been widely used to classify edits between correct-erroneous sentence pairs into predefined error categories.
However, the used error categories are far from being universal yielding many language specific variants of ERRANT.
In this paper, we discuss the applicability of the previously introduced grammatical error types to an agglutinative language, Turkish. We suggest changes on current error categories and discuss a hierarchical structure to better suit the inflectional and derivational properties of this morphologically highly rich language. We also introduce ERRANT-TR, the first automatic error type classification toolkit for Turkish. ERRANT-TR currently uses a rule-based error type classification pipeline which relies on word level morphological information. Due to unavailability of learner corpora in Turkish, the proposed system is evaluated on a small set of 106 annotated sentences and its performance is measured as 77.04\% $F_{0.5}$ score. 
The next step is to use ERRANT-TR for the development of a Turkish learner corpus. The code will be made publicly available.

EACL 2023

Towards Automatic Grammatical Error Type Classification for Turkish

automatic evaluation of gec

rule-based classifier

error type classification in turkish

poster

### Welcome to the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL) 
Continuing its mission of expanding and involving the science community of all European countries, EACL had selected the Ukrainian community for the 16th EACL, which took place online due to the COVID pandemic. Unfortunately, the ongoing war made the organisation in Kyiv impossible. Considering the importance of physical interaction among researchers, especially after the restrictions imposed by the COVID pandemic, in addition to an online mode, the conference will be held in Dubrovnik, Croatia, from 2 to 6 of May, 2023. The original aim of strengthening the connection with the Ukrainian community will not change as our program will feature a dedicated session and a workshop to highlight work on Ukrainian language technologies. As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

You need to register for the conference in order to access this site. Please visit https://2023.eacl.org/registration for more information.

As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

Automatically generated emotion arcs, which capture how an individual or a population feels over time, are widely used in industry and research. However, there is little research on evaluating the accuracy of these generated arcs. This is partly due to the difficulty of establishing the true (gold) emotion arc. Our work, for the first time, systematically and quantitatively evaluates automatically generated emotion arcs. We also compare two common ways of generating emotion arcs: Machine-Learning (ML) models and Lexicon-Only (LexO) methods. Using 36 diverse datasets, we systematically study factors that impact the quality of generated arcs. We show that despite being markedly poor at instance level, LexO methods are highly accurate at generating emotion arcs by aggregating information from hundreds of instances. These findings have wide-spread implications for both commercial development as well as research in fields such as psychology, public health, digital humanities, and social sciences, as LexO methods are simple, interpretable, have low carbon footprint, and do not require domain-specific training data or extensive programming expertise.

Frustratingly Easy Sentiment Analysis of Text Streams: Generating High-Quality Emotion Arcs Using Emotion Lexicons

Open-retrieval question answering systems are generally trained and tested on large datasets in well-established domains. However, low-resource settings such as new and emerging domains would especially benefit from reliable question answering systems. Furthermore, multilingual and cross-lingual resources in emergent domains are scarce, leading to few or no such systems.
In this paper, we demonstrate a cross-lingual open-retrieval question answering system for the emergent domain of COVID-19.
Our system adopts a corpus of scientific articles to ensure that retrieved documents are reliable. To address the scarcity of cross-lingual training data in emergent domains, we present a method utilizing automatic translation, alignment, and filtering to produce English-to-all datasets.
We show that a deep semantic retriever greatly benefits from training on our English-to-all data and significantly outperforms a BM25 baseline in the cross-lingual setting.
We illustrate the capabilities of our system with examples and release all code necessary to train and deploy such a system.

Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering Systems For Emergent Domains

In recent years, COVID-19 has impacted all aspects of human life. As a result, numerous publications relating to this disease have been issued. Due to the massive volume of publications, some retrieval systems have been developed to provide researchers with useful information. In these systems, lexical searching methods are widely used, which raises many issues related to acronyms, synonyms, and rare keywords. In this paper, we present a hybrid relation retrieval system, CovRelex-SE, based on embeddings to provide high-quality search results.

CovRelex-SE: Adding Semantic Information for Relation Search via Sequence Embedding

The development of poetry generation system mainly focuses on enhancing the capacity of generation model. However, the demands of customization and polishing are generally ignored, which highly reduces the scope of application. In this work, we present Yu Sheng, a web-based poetry generation system that is featured a human-in-loop generation framework, providing various customization options for users with different backgrounds to engage in the process of poetry composition. To this end, we propose two methods and train the models that can perform constrained generation and fine-grained polishing. The automatic and human evaluation results show that our system has a strong ability to generate and polish poetry compared to other vanilla models. Our system is publicly accessible at: https://yusheng.cis.um.edu.mo.

Yu Sheng: Human-in-Loop Classical Chinese Poetry Generation System

We develop FISH, an interactive system that extracts and highlights crucial textual signals from financial statements efficiently and precisely. The system is available online at https://cfda.csie.org/FISH/, and a short video for introduction is at https://youtu.be/ZbvZQ09i6aw.

FISH: A Financial Interactive System for Signal Highlighting

We present GATE Teamware 2: an open-source web-based platform for managing teams of annotators working on document classification tasks. GATE Teamware 2 is an entirely re-engineered successor to GATE Teamware, using contemporary web frameworks. The software allows the management of teams of multiple annotators, project managers and administrators - including the management of annotators - across multiple projects. Projects can be configured to control and monitor the annotation statistics and have a highly flexible JSON-configurable annotation display which can include arbitrary HTML. Optionally, documents can be uploaded with pre-existing annotations and documents are served to annotators in a random order by default to reduce bias. Crucially, annotators can be trained on applying the annotation guidelines correctly and then screened for quality assurance purposes, prior to being cleared for independent annotation. GATE Teamware 2 can be self-deployed, including in container orchestration environments, or provided as private, hosted cloud instances. GATE Teamware 2 is an open-source software and can be downloaded from Github.

GATE Teamware 2: An open-source tool for collaborative document classification annotation

Chain-of-thought (CoT) prompting enables large language models (LLMs) to solve complex reasoning tasks by generating an explanation before the final prediction. Despite it’s promising ability, a critical downside of CoT prompting is that the performance is greatly affected by the factuality of the generated explanation. To improve the correctness of the explanations, fine-tuning language models with explanation data is needed. However, there exists only a few datasets that can be used for such approaches, and no data collection tool for building them. Thus, we introduce CoTEVer, a tool-kit for annotating the factual correctness of generated explanations and collecting revision data of wrong explanations. Furthermore, we suggest several use cases where the data collected with CoTEVer can be utilized for enhancing the faithfulness of explanations. Our toolkit is publicly available at https://github.com/SeungoneKim/CoTEVer

CoTEVer: Chain of Thought Prompting Annotation Toolkit for Explanation Verification

We present TrialsSummarizer, a system that aims to automatically summarize
evidence presented in the set of randomized controlled trials most relevant to a given query.
Building on prior work, the system retrieves trial publications matching a query
specifying a combination of condition, intervention(s), and outcome(s), and ranks
these according to sample size and estimated study quality.The top-k such studies are passed through a neural multi-document summarization system, yielding a synopsis of these trials. We consider two architectures: A standard sequence-to-sequence model based on BART, and a multi-headed architecture intended to provide greater transparency to end-users.Both models produce fluent and relevant summaries of evidence retrieved for queries, but their tendency to introduce unsupported statements render them inappropriate for use in this domain at present. The proposed architecture may help users verify outputs allowing users the ability to interact with outputs and to trace generated tokens back to inputs.

Automatically Summarizing Evidence from Clinical Trials: A Prototype Highlighting Current Challenges

State-of-the-art models for identifying offensive language often fail to generalize over more nuanced or implicit cases of offensive and hateful language. Understanding model performance on complex cases is key for building robust models that are effective in real-world settings. To help researchers efficiently evaluate their models, we introduce OLEA, a diagnostic, open-source, extensible Python library that provides easy-to-use tools for error analysis in the context of detecting offensive language in English. OLEA packages analyses and datasets proposed by prior scholarship, empowering researchers to build effective, explainable and generalizable offensive language classifiers.

OLEA: Tool and Infrastructure for Offensive Language Error Analysis in English

Electronic Medical Records are integral parts of modern healthcare. Part of the records are clinical notes that healthcare providers take during encounters with patients. Notes are key to differential analysis which is the reasoning process leading to diagnosis and treatment. This paper presents DAVE, a differential analysis automation and visualization to assist healthcare professionals through the differential analysis process. DAVE takes as input clinical notes as they are being written by professionals and suggests candidate diagnostic algorithms. We digitized textbook diagnostic algorithms into directed acyclic graphs. We trained a distributional semantics model using an annotated corpora of electronic medical records and text from diagnostic algorithm descriptions. The model, boosted with PUBMed-based semantic similarity metrics, ranks the diagnostic algorithm graphs, and suggests the top three. The model achieved 74.3% success rate and was highly accepted by multiple medical professionals for usability

Premium content

Towards Automatic Grammatical Error Type Classification for Turkish

Downloads

Next from EACL 2023

Frustratingly Easy Sentiment Analysis of Text Streams: Generating High-Quality Emotion Arcs Using Emotion Lexicons

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES