Indonesia

Neural machine translation (NMT) for low-resource local languages in Indonesia faces
significant challenges, including the need for a representative benchmark and limited data
availability. This work addresses these challenges by comprehensively analyzing training NMT
systems for four low-resource local languages in Indonesia: Javanese, Sundanese,
Minangkabau, and Balinese. Our study encompasses various training approaches, paradigms,
data sizes, and a preliminary study into using large language models for synthetic low-resource
languages parallel data generation. We reveal specific trends and insights into practical
strategies for low-resource language translation. Our research demonstrates that despite
limited computational resources and textual data, several of our NMT systems achieve
competitive performances, rivaling the translation quality of zero-shot gpt-3.5-turbo. These
findings significantly advance NMT for low-resource languages, offering valuable guidance for
researchers in similar contexts.

IJCNLP-AACL 2023

Replicable Benchmarking of Neural Machine Translation (NMT) on Low-Resource Local Languages in Indonesia

workshop paper

### Welcome to IJCNLP-AACL 2023!
I am honored and happy to welcome you to IJCNLP-AACL 2023! The conference hosts both the 13th
International Joint Conference on Natural Language Processing (IJCNLP) and the 3rd Conference of
the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL). 
The joint conferences of IJCNLP and AACL have been organized by alternating leadership in the
Asia-Pacific region, in odd years by Asian Federation of Natural Language Processing (AFNLP) and
in even years by AACL. The conference is organized by AFNLP this year. 

Considering the current situation that we are not yet fully over the coronavirus pandemic, the
conference is offered in a hybrid mode, off-line in Nusa Dua, Bali, Indonesia, and on-line with the help
of Underline, during November 1-4, 2023. For those attending off-line, I hope that you will find time to
explore the exciting local area, which remains quite warm and pleasant at this time of the year. For
those attending on-line, I hope that the attendance is not much hampered, sometimes even enhanced
and comfortable, though the on-line arrangements have been made without ample time. 

Message from Jong C. Park, Korea Advanced Institute of Science and Technology (KAIST) 
IJCNLP-AACL 2023 General Chair 
###### [Conference Handbook](https://drive.google.com/file/d/1S2JSXa105xJxIkNBfhYmGH20Om6wARBx/view?usp=sharing)

In order to acces this event page, you need to register [here](http://www.ijcnlp-aacl2023.org/)

The 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics

We present the development of a Named Entity Recognition (NER) dataset for
Tagalog. This corpus helps fill the resource gap present in Philippine languages today, where
NER resources are scarce. The texts were obtained from a pretraining corpora containing news
reports, and were labeled by native speakers in an iterative fashion. The resulting dataset
contains ~7.8k documents across three entity types: Person, Organization, and Location. The
inter-annotator agreement, as measured by Cohen&#39;s Kappa, is 0.81.We also conducted
extensive empirical evaluation of state-of-the-art methods across supervised and transfer
learning settings. Finally, we released the data and processing code publicly to inspire future
work on Tagalog NLP.

Developing a Named Entity Recognition Dataset for Tagalog

Wordnets are indispensable tools for various natural language processing
applications. Unfortunately, wordnets get outdated, and producing or updating wordnets can
be slow and costly in terms of time and resources. This problem intensifies for low-resource
languages. This study proposes a method for word sense induction and synset induction using
only two linguistic resources, namely, an unlabeled corpus and a sentence embeddings-based
language model. The resulting sense inventory and synonym sets can be used in automatically
creating a wordnet. We applied this method on a corpus of Filipino text. The sense inventory
and synsets were evaluated by matching them with the sense inventory of the machine
translated Princeton WordNet, as well as comparing the synsets to the Filipino WordNet. This
study empirically shows that the 30% of the induced senses are valid and 40% of induced
synsets are valid in which 20% are novel synsets.

Towards Automatic Construction of Filipino WordNet: Word Sense Induction and Synset Induction Using Sentence Embeddings

Clickbait spoiling aims to generate a short text to satisfy the curiosity induced by a
clickbait post. As it is a newly introduced task, the dataset is only available in English so far. Our
contributions include the construction of manually labeled clickbait spoiling corpus in
Indonesian and an evaluation on using cross-lingual zero-shot question answering-based
models to tackle clickbait spoiling for low-resource language like Indonesian. We utilize
selection of multilingual language models. The experimental results suggest that XLM-RoBERTa
(large) model outperforms other models for phrase and passage spoilers, meanwhile,
mDeBERTa (base) model outperforms other models for multipart spoilers.

Low-Resource Clickbait Spoiling for Indonesian via Question Answering

While there are many grammar checkers available for various languages, especially
the English language, those that exist for the low resource Filipino language can only effectively
correct lexical errors. There is yet to be a publicly available Filipino grammar checker that can
also address semantic errors, which are more complex. As such, this study found an
opportunity to introduce Balarila, a deep learning-based Filipino GEC model inspired by the
GECToR approach. To address the absence of a training and test dataset, an automated error
generation pipeline was devised, creating synthetic datasets of error-free and error-filled
Filipino sentences sourced from various online news sources. Tagalog BERT and RoBERTa
models were fine-tuned in two stages using this generated corpus. Evaluation metrics included
precision, recall, and F0.5 scores for GEC, and a multi-class confusion matrix for GED. The top-
performing model, RoBERTa Tagalog Large, achieved an F0.5 score of 70.75, while the RoBERTa
Tagalog Base, with a F0.5 score of 69.00, demonstrated cost-effectiveness in training. The
created datasets can also be used as a benchmark for Filipino grammar checker models

Balarila: Deep Learning for Semantic Grammar Error Correction in Low-Resource Settings

Fluency is a crucial goal of all Natural Language Generation (NLG) systems. Widely
used automatic evaluation metrics fall short in capturing the fluency of machine-generated
text. Assessing the fluency of NLG systems poses a challenge since these models are not limited
to simply reusing words from the input but may also generate abstractions. Existing reference-
based fluency evaluations, such as word overlap measures, often exhibit weak correlations with
human judgments. This paper presents an unsupervised technique for measuring text fluency
without the need for any reference. Our approach leverages various word embeddings and
trains language models using Recurrent Neural Network (RNN) architectures. We also

experiment with other available multilingual Language Models (LMs). To assess the
performance of the models, we conduct a comparative analysis across 10 Indic languages,
correlating the obtained fluency scores with human judgments. We also release a human-
annotated benchmark test set for fluency.

Unsupervised approach to evaluate sentence-level fluency: Do we really need reference? 

Rumors have emerged as significant global challenges in recent years. Once praised as means of information, unity, and global connectivity, social media platforms and online communication tools have now become breeding grounds for the spread of false, misleading, and manipulative information on an unprecedented scale. Rumors, composed of untrue information and deceptive messages, are designed in a way that manipulates emotions. Therefore, emotions play a vital role in determining the veracity of rumors. In this paper, we present a novel emotion-enriched and psycholinguistics features-based approach for rumor detection on online social media. The proposed approach encompasses detecting rumors using lexicons and various linguistics features-based learning approaches, mainly extracting the psychological association of words and their emotions. The emotion-related and psycholinguistic features are extracted for both posts and comments to augment the approach and facilitate detecting rumors more effectively. The semantic relations of a post and its comments with their underlying emotions are preserved using word-level GloVe embedding. The proposed approach is evaluated over a publicly available PHEME dataset and compared with various baselines and SOTA approaches, which show significantly better performance for rumor detection on online social media.

An Emotion-Enriched and Psycholinguistics Features-Based Approach for Rumor Detection on Online Social Media

Biomedical entity linking, also known as biomedical concept normalization, has recently witnessed the rise to prominence of zero-shot contrastive models. However, the pretraining material used for these models has, until now, largely consisted of specialist biomedical content such as MIMIC-III clinical notes and PubMed papers. While the resulting in-domain models have shown promising results for many biomedical tasks, adverse drug event normalization on social media texts has so far remained challenging for them.

In this paper, we propose a new approach for adverse drug event normalization on social media relying on general-purpose model initialization via BioLORD and a semantic-text-similarity finetuning named STS.

Our experimental results on several social media datasets demonstrate the effectiveness of our proposed approach, by achieving state-of-the-art performance. Based on its strong performance across all the tested datasets, we believe this work could emerge as a turning point for the task of adverse drug event normalization on social media and has the potential to serve as a benchmark for future research in the field.

Boosting Adverse Drug Event Normalization on Social Media: General-Purpose Model Initialization and Biomedical Semantic Text Similarity Benefit Zero-Shot Linking in Informal Contexts

Identifying mental health conditions using user-generated text on social media paved the way for automated computational methods for mental health surveillance on social media. Inferring the accurate state of mental health markers requires understanding the user's emotions and history of mental health conditions for identifying the mental health landscape of at-risk users. Existing works have exploited social media data by using users' most recent posts; however, previous works ignored users' emotional historical activity on social media, indicating their mental state over time. In this work, we address this gap and present a novel emotion- and time-aware architecture that jointly learns social media users' emotional historical context and temporal posting irregularities for mental health surveillance on social media. We conduct classification experiments to identify social media users with mental conditions, i.e., (i) healthy users and users affected with depression and (ii) healthy users and users prone to self-harm. Experimental results demonstrate proposed method outperforms recent competitive methods, demonstrating the importance of capturing the user's temporal emotional spectrum and time-aware emotional context through historical posts for social media users' mental health surveillance.


Temporal Tides of Emotional Resonance: A Novel Approach to Identify Mental Health on Social Media

Huge amounts of sentiment-rich data are produced on social networks, the rate of which is growing year by year. As such, a global view of sentiment on social media platforms is only possible at scale through automation. The huge data sizes also mean that sentiment analysis systems must balance both accuracy and computing resource requirements. Typical NLP research maximizes accuracy, while other metrics are overlooked and thus older models are forgotten. Prior models are easily forgotten despite their possible suitability in settings where large computing resources are unavailable or relatively more costly. In this paper, we perform a broad comparative evaluation of document-level sentiment analysis models with a focus on resource costs that are important for the feasibility of model deployment and general climate consciousness. Our experiments consider different feature extraction techniques, the effect of ensembling, task-specific deep learning modeling, and domain-independent large language models (LLMs). We find that while a fine-tuned LLM achieves the best accuracy, some alternate configurations provide huge (up to 24, 283×) resource savings for a marginal (<1%) loss in accuracy. Furthermore, we find that for smaller datasets, the differences in accuracy shrink while the difference in resource consumption grows further.

Efficient Sentiment Analysis: A Resource-Aware Evaluation of Feature Extraction Techniques, Ensembling, and Deep Learning Models

Code-mixing is a common linguistic phenomenon when words or syllables from two or more languages are used within a single sentence. Several works have been conducted on building such datasets and performing downstream tasks on them. Although it is very common to observe code-mixing with multiple languages, most datasets in this domain contain code-mixed data from only two languages. In this paper, we introduce OffMix-3L, a novel dataset for the task of offensive language identification which contains code-mixed data from 3 different languages. We also provide a comprehensive experimental analysis with several models and observe that BanglishBERT outperforms all the transformer-based models and also the generative model GPT-3.5 on OffMix-3L.

Premium content

Replicable Benchmarking of Neural Machine Translation (NMT) on Low-Resource Local Languages in Indonesia

Downloads

Next from IJCNLP-AACL 2023

Developing a Named Entity Recognition Dataset for Tagalog

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES