Brazil

Named entity recognition (NER) tasks imply token-level labels. Annotating documents can be time-consuming, costly, and prone
to human error. In many real-life scenarios, the lack of labeled data has
become the biggest bottleneck preventing NER being effectively used in
some domains and with some natural languages, with negative impacts
on the quality of some tasks. To overcome the barrier of the lack of annotated data for new application domains in some natural languages,
we propose a method that uses the output of an ensemble of NER’s to
automatically annotate the data needed to train a Bidirectional Encoder
Representations from Transformers (BERT) based NER for Portuguese.
The performance was assessed using MiniHAREM dataset with promising results. For domain relevant classes such as LOCAL, F1, Precision
and Recall above 50% were obtained when training only with automatically annotated data.

PROPOR 2022

Named Entity Extractors for New Domains by Transfer Learning with Automatically Annotated Data

transfer learning

named entity recognition(ner)

automatic annotation

The International Conference on Computational Processing of Portuguese (PROPOR) is the main event in the area of human language processing that is focused on theoretical and technological issues of written and spoken Portuguese and Galician. The meeting has been a very rich forum for the exchange of ideas and partnerships for the research and industry communities dedicated to the automated processing of Portuguese, promoting the development of methodologies, resources and projects that can be shared among researchers and practitioners in the field.

Find more information and pay the registration fee on [event organizer’s website](https://sites.universidadedefortaleza.com/propor2022/?page_id=401)

The International Conference on Computational Processing of Portuguese (PROPOR) is the main event in the area of human language processing that is focused on theoretical and technological issues of written and spoken Portuguese and Galician.

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible. The experience for on-site participants would closely approximate a normal pre-COVID *ACL conference, with 5-6 thematically organized parallel sessions and live Q/A and interactive discussion immediately after the talks. Presentations by virtual participants will be equitably interleaved with those of on-site participants, projected on the auditorium screens as if on-site, and also followed immediately by live Q/A and interactive discussion at a time during reasonable waking hours for the virtual presenter. For all participants, on-site and virtual, who are unable to attend a session due to either time-zone issues or because they are participating in another session live, talk recordings and slides will be available online at a minimum after the live presentation (and in many cases before as well), and questions may be submitted in advance on session-specific discussion boards and answered live in session with the usual visual aids if desired.

<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/ceby/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>

Please Note: The EMNLP registration system is not currently connected to the underline site as we are still in the process of building out EMNLP 2021. You will receive access instructions from underline the week of November 1st. 

Access is given only to EMNLP upon registration, if you have not registered please do so [here](https://2021.emnlp.org/registration).

Registered attendees will receive access the week of November 1st.

EMNLP 2021

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible.

poster

We introduce the problem of identifying characters in literary text, and mention some specific issues that are special for Portuguese, in the context of presenting DIP, a shared task to foster work in the area and produce resources for computational literature studies in Portuguese. We describe how the task is organized, the resources that will be created, and how we plan to evaluate the results produced by the participant systems.

Identifying literary characters in Portuguese: Challenges of an international shared task

The lack of a pattern widely accepted in Corpus Linguistics for the construction of text corpora has resulted in multiple possible paths to compilation, resulting in research products that are difficult to interface and have low continuity. The varie-ty of existing corpora aggravates this problem since individual studies use differ-ent forms of expression to describe and solve similar problems. Thus, a gap is identified in the production initiatives to create corpora regarding the “explorato-ry” way of conducting such initiatives. Based on the Lapelinc Workflow for the construction of historical corpora, the Lapelinc Framework presents a working pattern for construction and research activities involving Portuguese language corpora, thereby establishing a set of steps and by-products common to corpora-based linguistic research. Providing a set of tools in the Import Stage with a re-duced learning curve and allowing the convergence of efforts during develop-ment, the Lapelinc Framework tools add functionalities for multiple purposes in the construction of corpora, simplifying the construction of multi-type corpora, facilitated by the resources it makes available.

The systematic construction of multiple types of corpora through the Lapelinc Framework

Deep learning techniques have been shown to be efficient in various tasks, especially in the development of speech recognition systems, that is, systems that aim to transcribe an audio sentence in a sequence of written words. Despite the progress in the area, speech recognition can still be considered difficult, especially for languages lacking available data, such as Brazilian Portuguese (BP). In this sense, this work presents the development of an public Automatic Speech Recognition (ASR) system using only open available audio data, from the fine-tuning of the Wav2vec 2.0 XLSR-53 model pre-trained in many languages, over BP data. The final model presents an average word error rate of 12.4% over 7 different datasets (10.5% when applying a language model). According to our knowledge, the obtained error is the lowest among open end-to-end (E2E) ASR models for BP.

Brazilian Portuguese Speech Recognition Using Wav2vec 2.0

BERT has revolutionized the NLP field by enabling transfer learning with large language models that can capture complex textual patterns, reaching the state-of-the-art for an expressive number of NLP applications. For text classification tasks, BERT has already been extensively explored. However, aspects like how to better cope with the different embeddings provided by the BERT output layer and the usage of language-specific instead of multilingual models are not well studied in the literature, especially for the Brazilian Portuguese language. The purpose of this article is to conduct an extensive experimental study regarding different strategies for aggregating the features produced in the BERT output layer, with a focus on the sentiment analysis task. The experiments include BERT models trained with Brazilian Portuguese corpora and the multilingual version, contemplating multiple aggregation strategies and open-source datasets with predefined training, validation, and test partitions to facilitate the reproducibility of the results. BERT achieved the highest ROC-AUC values for the majority of cases as compared to TF-IDF. Nonetheless, TF-IDF represents a good trade-off between the predictive performance and computational cost.

BERT for Sentiment Analysis: Pre-trained and Fine-Tuned Alternatives

This paper proposes an interface between linguistic concepts and computational interventions to support hate speech detection.
We applied a Natural Language Processing pipeline involving topic modeling and semantic role labeling, allowing a semi-automatic identification of hate speech. We also discuss how such speech qualifies as a type of verbal violence widespread on social networks to reinforce a sexist stereotype. Finally, we use Twitter data to analyze information that resulted in virtual attacks against a specific person.
As an achievement, this work validates the use of linguistic features to annotate data either as hate speech or not. It also proposes using fallacies as a potential additional feature to identify potential intolerant discourses.

Sexist Hate Speech: Identifying Potential Online Verbal Violence Instances

Artificial Intelligence applied to Law is getting attention in the community, both from the Judiciary and the lawyers, due to possible gains in procedural celerity and the automation of repetitive tasks, among other use cases. Many of its bene-fits can be derived from Natural Language Processing since legal proceedings are textual document-based. Named Entity Recognition is the NLP task of identifying and classifying named entities in unstructured text to help achieve these goals. In recent years, transformers emerged as the new state-of-the-art architecture for some tasks in NLP systems. NLP resources in Portuguese, such as models and datasets, are needed to allow Portuguese speaking countries to benefit from this new NLP level. This paper presents the first fine-tuned BERT models trained ex-clusively on Brazilian Portuguese for Legal NER. The models achieved new state-of-the-art on LeNER-Br dataset, a Portuguese NER corpus for the legal domain. We also built a prototype application for Judiciary users to evaluate how the model performed with authentic law documents. The results showed that the models were able to extract information with good quality. Both the models and the prototype application are publicly available for the community.

Fostering Judiciary applications with new fine-tuned models for Legal Named Entity Recognition in Portuguese

Most approaches to role-filler entity extraction (REE) rely on large labeled training corpora in which entity mentions are directly annotated in the input document. In this work, we leverage an existing knowledge base (KB) of entities to perform document-level REE from drug seizure petitions. We propose a system that learns to extract entities from petitions to fill 29 roles of a drug seizure event. Although we have access to a KB covering more than 170 thousand entities and six thousand petitions, such that each entity in the KB is linked to a specific petition, the mentions to an entity within a petition's text are not annotated. The lack of these annotations brings challenges related to mismatches between entity values in the KB and entity mentions in the documents. Additionally, there are entities with same type or same value. Thus, we propose a distant annotation method to overcome these challenges and automatically label petition documents using the available KB. This annotation method includes a parameter that controls the balance between precision and recall. We also propose a strategy to effectively tune this parameter in order to optimize a given metric. We then train a BERT-based sequence labeling model that learns to identify entity mentions and label them. Our system achieves an F1-score of 78.59 with precision over 82%. We also report ablation studies regarding the distant annotation method.

Entity Extraction from Portuguese Legal Documents Using Distant Supervision

We present some results of applying natural language processing (NLP) techniques in the domain of History, having as object of investigation the Brazilian Historical-Biographical Dictionary (Dicionário Histórico-Biográfico Brasileiro, DHBB). After improving or adding annotation of specific fields, information extraction techniques based on manually derived patterns were applied to three relevant problems: the age of entrance in Brazilian politics, the academic background of Brazilian politicians, and family ties among the political elites.

Automatic information extraction: a distant reading of the Brazilian Historical-Biographical Dictionary

Products, services, among many other things in life have a quality standard, are inclusive, or do not harm customers. Regulations required from their manufacturers or providers make it possible. This type of requirement also exists in the finance sector. Governments, international agencies, or civil institutions are responsible for creating, applying, and inspecting these regulations. Regulators from all spheres (federal, state, and municipal) constantly demand changes in the finance sector to meet current needs adequately. This paper presents the constant evolution of a banking compliance application in Brazil. It aims to classify the relevance or irrelevance of regulatory documents published by more than 100 Brazilian regulators, affecting the businesses of more than 40 departments of Banco do Brasil. The application uses a hybrid strategy, combining machine learning and rules for a binary classification challenge involving each company department. This work also presents a particular type of corpus imbalance called The Imbalance Within Class.

Banking Regulation Classification in Portuguese

Deep learning transformers have drastically improved systems that automatically answer questions in natural language. However, different questions demand different answering techniques; here we propose, build and validate an architecture that integrates different modules to answer two distinct kinds of queries. Our architecture takes a free-form natural language text and classifies it to send it either to a Neural Question Answering Reasoner or a Natural Language parser to SQL. We implemented a complete system for the Portuguese language, using some of the main tools available for the language and translating training and testing datasets. Experiments show that our system selects the appropriate answering method with high accuracy (over 99%), thus validating a modular question answering strategy.


Named Entity Extractors for New Domains by Transfer Learning with Automatically Annotated Data

Downloads

Next from PROPOR 2022

Identifying literary characters in Portuguese: Challenges of an international shared task

Similar lecture

SciBERT-based Multitasking Deep Neural Architecture to identify Contribution Statements from Scientific articles

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES