Brazil

This paper describes the development of the first test suite for the language direction Portuguese-English. Designed for fine-grained linguistic analysis, the test suite comprises 330 test sentences for 66 linguistic phenomena and 14 linguistic categories. Eight different MT systems were compared using quantitative and qualitative methods via the test suite: DeepL, Google Sheets, Google Translator, Microsoft Translator, Reverso, Systran, Yandex and an internally built NMT system
trained over 30 hours on 2,5M sentences. It was found that ambiguity, named entity &amp; terminology and verb valency are the categories where MT systems struggle most. Negation, pronouns, subordination, verb tense/aspect/mood and false friends are the categories where MT systems perform best.

PROPOR 2022

A test-suite for the evaluation of Portuguese-English machine translations

test suite

portuguese

evaluation

machine translation

# Welcome everyone to ACL 2022!

The 60th Annual Meeting of the Association for Computational Linguistics is taking place May 22-27, 2022 as a hybrid event, in Dublin and online. We are happy to welcome all of you to this anniversary edition with an almost 50-50 in-person and virtual participation. 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, three amazing keynote events and two new initiatives of invited talks: Spotlight Talks for Young Rising Stars (STIRS) and The Next Big Idea Talks. Posters (including Findings of ACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop will have an oral session and a poster session as part of Poster Session 1. The program also features eight Tutorials and 28 Workshops. 

 
We wish you a wonderful conference! 
[**The ACL 2022 Organizing Committee**](https://www.2022.aclweb.org/organisers)
 
[**Conference Handbook**](https://drive.google.com/file/d/1_BUCMfhMVrjG9E2e71aHdHeE28KSje0l/view?usp=sharing) 
[**Mini Handbook**](https://drive.google.com/file/d/1qlBKl0wzmlVF1oCeMQl3BahLd9nLP5Ce/view?usp=sharing) 
[**Posters and Demo guides**](https://drive.google.com/file/d/1UucMAoCNncIOaH1rMMDa0owuG9qgvJTG/view?usp=sharing)

ACL 2022

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing (NLP). 

This paper describes the development of the first test suite for the language direction Portuguese-English. Designed for fine-grained linguistic analysis, the test suite comprises 330 test sentences for 66 linguistic phenomena and 14 linguistic categories. Eight different MT systems were compared using quantitative and qualitative methods via the test suite: DeepL, Google Sheets, Google Translator, Microsoft Translator, Reverso, Systran, Yandex and an internally built NMT system
trained over 30 hours on 2,5M sentences. It was found that ambiguity, named entity & terminology and verb valency are the categories where MT systems struggle most. Negation, pronouns, subordination, verb tense/aspect/mood and false friends are the categories where MT systems perform best.

technical paper

The International Conference on Computational Processing of Portuguese (PROPOR) is the main event in the area of human language processing that is focused on theoretical and technological issues of written and spoken Portuguese and Galician. The meeting has been a very rich forum for the exchange of ideas and partnerships for the research and industry communities dedicated to the automated processing of Portuguese, promoting the development of methodologies, resources and projects that can be shared among researchers and practitioners in the field.

Find more information and pay the registration fee on [event organizer’s website](https://sites.universidadedefortaleza.com/propor2022/?page_id=401)

The International Conference on Computational Processing of Portuguese (PROPOR) is the main event in the area of human language processing that is focused on theoretical and technological issues of written and spoken Portuguese and Galician.

Paraphrasing is a fundamental technique for many text applications. Typically, this task is performed through models that perform
lexical and translation operations, which tend to present a trade-off between meaning preservation and diversity. In this paper, we present a
transformer-based approach called PTT5-Paraphraser, a PPT5 model
fine-tuned on TaPaCo, a large corpus of paraphrases. PTT5-Paraphraser
achieves good results according to a number of metrics, showing a good
compromise between diversity and fidelity to the original meaning. Two
human evaluations are made to explore the paraphrases produced by our
model: the first analyzes their quality in terms of preserving the meaning
and diversity of sentences, while the second compares automatically generated paraphrases with human-made ones. Finally, we perform a classification task, which shows that datasets augmented with paraphrases
can substantially increase the performance of classifiers.

PTT5-Paraphraser: Diversity and Meaning Fidelity in Automatic Portuguese Paraphrasing

This work presents an analysis of Brazilian political discourse from speeches and social media posts, focusing on the ability to transfer learned models’ knowledge between different contexts. The analysis is conducted through PoliS, a new resource containing two datasets of political discussions labeled for party and ideological leaning from congressional speeches and social media posts by political influencers. The transfer learning experiments are performed using the transcripts of the congressional speeches to train a model used to predict the political leaning of social media influencers. To evaluate the robustness of the model, the analysis includes a time-lag study of the performance degradation of the transferred model. We find that relatively little social media data (about one hour) is needed to achieve reasonable performance in classification, and that performance does not degrade significantly over time.

A transfer learning analysis of political leaning classification in cross-domain content

Legal text processing is a challenging task for modeling approaches due to the peculiarities inherent to its features, such as long texts and their technical vocabulary. Topic modeling consists of discovering a semantic structure in the text. This paper investigates the application of topic modeling and the use of information about the legislation cited in identifying the subject of legal documents and evaluating its applicability in the classification of Brazilian lawsuits. The models were trained with a Golden Collection of 16 thousand initial petitions and indictments from the Court of Justice of the State of Ceará, in Brazil, whose lawsuits were classified in the five more representative National Council of Justice (CNJ) of Brazil classes - Common Civil Procedure, Execution of Extrajudicial Title, Criminal Action - Ordinary Procedure, Special Civil Court Procedure, and Tax Enforcement. The results obtained outperform the baseline, achieving 0.89 of F1 score (macro). Our interpretation is that the representation of the document through contextual embeddings generated by BERT, as well as the architecture of the model with bidirectional contexts, makes it possible to capture the specific context of the domain of legal documents. Thus, the use of the legislation mentioned in the representation of documents can improve the accuracy of the classification task.


Using Topic Modeling in Classification of Brazilian Lawsuits

Even though most of its energy generation comes from renewable sources, Brazil is one of the largest emitters of greenhouse gases in the world, due to intense farming and deforestation of biomes such as the Amazon Rainforest, whose preservation is essential for compliance with the Paris Agreement. Still, regardless of lobbies or prevailing political orientation, all government legal actions are published daily in the Brazilian Federal Official Gazette (BFOG, or ``Diário Oficial da União" in Portuguese). However, with hundreds of decrees issued every day by the authorities, it is absolutely burdensome to manually analyze all these processes and find out which ones can pose serious environmental hazards. In this paper, we present a strategy to compose automated techniques and domain expert knowledge to process all the data from the BFOG. We also provide the Government Actions Tracker, a highly curated dataset, in Portuguese, annotated by domain experts, on federal government acts about the Brazilian environmental policies. Finally, we build and compared four different NLP models on the classfication task in this dataset. Our best model achieved a F1-score of $0.714 \pm 0.031$. In the future, this system should serve to scale up the high-quality tracking of all oficial documents with a minimum of human supervision and contribute to increasing society's awareness of government actions.

Tracking Environmental Policy Changes in the Brazilian Federal Official Gazette

The anchorage to real data is one of the main parameters that guarantees the quality and the coverage of lexical resources, especially in the context of specialized domains. Thus, lexicon extraction from corpora is a consensual method for building lexical resources. However, given that data validation by experts in specialized contexts is a necessary step, the automatic screening of data becomes fundamental to maximize the informational value of the interaction with experts. In this presentation we present and discuss a hybrid methodology, combining linguistic and statistical approaches, focusing on the extraction of specialized lexical units and salient semantic information using CQL grammars. The proposed method involves several steps, from frequency information analyses, concordances and collocations extraction to manual revision and expert validation and encompasses the construction and application of knowledge-based patterns CQL grammars. We present two CQL grammars for lexical and semantic information extraction developed for Portuguese and Italian and evaluate results from its application to specialized corpora on Public Art domain, demonstrating the value of this method for lexicon and semantic information extraction from large data.

CQL grammars for lexical and semantic information extraction for Portuguese and Italian

We compiled a set of patterns that denote lexico-semantic relations in Portuguese, created templates where one argument is fixed and the other is replaced by a mask, and then used BERT for predicting the latter. For most relations and measures, when assessed in a test of lexico-semantic analogies, BERT predictions outperformed those of earlier methods computed from static word embeddings, either with a pattern alone or with a combination of patterns. There is still a large margin of progression, but this suggests that BERT can be used as a source of lexico-semantic knowledge in Portuguese.

Drilling Lexico-Semantic Knowledge in Portuguese from BERT

Named Entity Recognition (NER) is the task of identifying textual elements and categorizing them into predefined classes, such as names of people, locations, organizations, and others. Recently, NER systems based on neural networks have obtained state-of-the-art results. One of the main components of this kind of system is the word embeddings representation adopted that can be static (traditional) or contextual trained in language models. This work analyzes the strategy of combining traditional and contextual embeddings to obtain richer representations for Portuguese NER. We adopt a Bidirectional Long ShortTerm Memory (BiLSTM) as a classification model integrated with the Conditional Random Fields (CRF) algorithm. Experiments were carried out in different corpora, and the results obtained demonstrate that the combination strategy of word embeddings models is viable and led to results comparable with state-of-the-art NER systems in Portuguese.

Combining Word Embeddings for Portuguese Named Entity Recognition

Traditional text classification approaches often require a good amount of labeled data, which is difficult to obtain, especially in restricted domains or less widespread languages. This lack of labeled data has led to the rise of low-resource methods, that assume low data availability in natural language processing. Among them, zero-shot learning stands out, which consists of learning a classifier without any previously labeled data. The best results reported with this approach use language models such as Transformers, but fall into two problems: high execution time and inability to handle long texts as input. This paper proposes a new model, ZeroBERTo, which leverages an unsupervised clustering step to obtain a compressed data representation before the classification task. We show that ZeroBERTo has better performance for long inputs and shorter execution time, outperforming XLM-R by about 12% in the F1 score in the FolhaUOL dataset.

ZeroBERTo - Leveraging Zero-Shot Text Classification by Topic Modeling

Our contribution presents the first ever stance and sentiment annotated corpus in Portuguese. Using data from Twitter, we annotated a data set with over 15,000 tweets. In building the corpus, we made both random and supervised searches to maximize balance. Using four annotators, we have classified each tweet on four categories: support, against, neutral or inconclusive. Furthermore, we have annotated each tweet for sentiment: positive, negative, neutral, sarcasm or inconclusive. Our annotators yield strong inter-annotator agreement. In addition, we test a baseline model using a pre-trained BERT model for Portuguese. Results suggest that our data set is in line with its counterparts in other languages.

SS-PT: A Stance and Sentiment data set from Portuguese quoted tweets

This paper presents a targeted syntactic evaluation of Transformer models for Galician-Portuguese. We defined three experiments that allow to explore how these models, trained with a masked language modeling objective, encode syntactic knowledge. To do so, we created a new dataset including test instances of number (subject-verb), gender (subject-predicative adjective), and person (subject-inflected infinitive) agreement. This dataset was used to evaluate monolingual and multilingual BERT models, controlling for various aspects such as the presence of attractors or the distance between the dependent elements. The results show that Transformer models perform competently in many cases, but they are generally confounded by the presence of attractors in long-distance dependencies. Moreover, the different behavior of monolingual models trained with the same corpora reinforces the need for a deep exploration of the network architectures and their learning process.

Premium content

A test-suite for the evaluation of Portuguese-English machine translations

Downloads

Next from PROPOR 2022

PTT5-Paraphraser: Diversity and Meaning Fidelity in Automatic Portuguese Paraphrasing

Similar lecture

A Natural Diet: Towards Improving Naturalness of Machine Translation Output

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES