Malta

Detection of Homophobia and Transphobia in social media comments serves as an important step in the overall development of Equality, Diversity and Inclusion (EDI). In this research, we describe the system we formulated while participating in the shared task of Homophobia/Transphobia detection as a part of the Fourth Workshop On Language Technology For Equality, Diversity, Inclusion (LT-EDI-2024) at EACL 2024. We used an ensemble of three state-of-the-art multilingual transformer models, namely Multilingual BERT (mBERT), Multilingual Representations for Indic Languages (MURIL) and XLM-RoBERTa to detect the presence of Homophobia or Transphobia in YouTube comments. The task comprised of datasets in ten languages - Hindi, English, Telugu, Tamil, Malayalam, Kannada, Gujarati, Marathi, Spanish and Tulu. Our system achieved rank 1 for the Spanish and Tulu tasks, 2 for Telugu, 3 for Marathi and Gujarati, 4 for Tamil, 5 for Hindi and Kannada, 6 for English and 8 for Malayalam. These results speak for the efficacy of our ensemble model as well as the data augmentation strategy we adopted for the detection of anti-LGBT+ language in social media data.

EACL 2024

MEnTr@LT-EDI-2024: Multilingual Ensemble of Transformer Models for Homophobia/Transphobia Detection

homophobia/transphobia detection

multilingual classification

transformer models

workshop paper

Welcome to the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL). Continuing its mission of expanding and involving the science community of all European countries, EACL has selected the Malta community for the 18th EACL. Considering the importance of physical interaction among researchers, the conference will be held at the Hotel Radisson Blu, St. Julians, in Malta, from 17 to 22 of March, 2024. The conference will also feature the possibility to participate virtually. As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language. 
#### [Conference Handbook](https://drive.google.com/file/d/13Wn38q0yev6U3RTgnf4Tn5T9GuAuZ-xW/view?usp=sharing) *downloadable*

To access the EACL 2024 event page you need to register [**here**](https://acl.swoogo.com/EACL2024). 

Please register!

As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

Machine learning and deep learning models have shown great potential in detecting hate speech from social media posts. This study focuses on the homophobia and transphobia detection task of LT-EDI-2024 in English. Several machine learning models, a Deep Neural Network (DNN), and the Bidirectional Encoder Representations from Transformers (BERT) model have been trained on the provided dataset using different feature vectorization techniques. We secured top rank with the best macro-f1 score of 0.4963, which was achieved by fine-tuning the BERT model on the English test set.

dkit@LT-EDI-2024: Detecting Homophobia and Transphobia in English Social Media Comments

The pervasive impact of stress on individuals necessitates proactive identification and intervention measures, especially in social media interaction. This research paper addresses the imperative need for proactive identification and intervention concerning the widespread influence of stress on individuals. This study focuses on the shared task, "Stress Identification in Dravidian Languages," specifically emphasizing Tamil and Telugu code-mixed languages. The primary objective of the task is to classify social media messages into two categories: stressed and non stressed. We employed various methodologies, from traditional machine-learning techniques to state-of-the-art transformer-based models. Notably, the Tamil-BERT and Telugu-BERT models exhibited exceptional performance, achieving a noteworthy macro F1-score of \textbf{0.71} and \textbf{0.72}, respectively, and securing the position in Tamil code-mixed language and the position in the Telugu code-mixed language. These findings underscore the effectiveness of these models in recognizing stress signals within social media content composed in Tamil and Telugu.

CUET_DUO@StressIdent_LT-EDI@EACL2024: Stress Identification Using Tamil-Telugu BERT

Identifying an individual where he/she is stressed or not stressed is our shared task topic. we have used several machine learning models for identifying the stress. This paper presents our system submission for the task 1 and 2 for both Tamil and Telugu dataset, focusing on us- ing supervised approaches. For Tamil dataset, we got highest accuracy for the Support Vector Machine model with f1-score of 0.98 and for Telugu dataset, we got highest accuracy for Random Forest algorithm with f1-score of 0.99. By using this model, Stress Identification System will be helpful for an individual to improve their mental health in optimistic manner.

KEC-AI-MIRACLE-MAKERS@LT-EDI-2024: Stress Identification in Dravidian Languages using Machine Learning Techniques

Homophobic/Transphobic (H/T) content includes hate speeches, discriminatory texts directed at LGBT individuals. Furthermore, social media text data is often code-mixed, which is difficult to analyze, detect H/T content. To address this issue effectively, in this paper, we - team MUCS, describe the ML and TL based models submitted to "Homophobia/Transphobia Detection in social me- dia comments:LT-EDI@EACL 2024" shared task at European Chapter of the Association for Computational Linguistics (EACL) 2024. Three distinct models: i) Homo_Ensemble- a Machine Learning (ML) model trained with syllable TF-IDF of n= (1, 3), ii) Homo_TL - model makes use of oversampling technique used to fine-tune the Transfer Learning (TL) based pretrained Multilingual Bidirectional Encoder Representations from Transformers (mBERT) models, and iii) Homo_mvlearn - a multiview (mvlearn) model which ultizes the probability scores of two models. Among the proposed models Homo_Ensemble obtained macro F1 score of 0.958 securing 4th rank for Telugu, Homo_TL obtained macro F1 scores of 0.493, 0.537, 0.458, 0.948, 0.958 securing 2nd , 2nd , 1st , 1st , and 4th ranks for English, Marathi, Hindi, Kannada and Gujarathi languages respectively. The proposed Homo_mvlearn model obtained macro F1 scores of 0.860, 0.870, and 0.532 se- curing 2nd , 6th , and 2nd ranks for Tamil, Malayalam, and Spanish languages respectively. Tulu Language task consists of Few-Shot Learning (FSL) problem. For the same, proposed two FSL models comprised of Sentence Transformer (ST) model with BERT pre-trained variant models namely, indic_sbert, kannada_sbert. Among these ST model with kannada_sbert ob- tained macro F1 score of 0.620 securing 2nd rank.

MUCS@LT-EDI-2024: Learning Approaches to Empower Homophobic/Transphobic Comment Identification

The results of the Shared Task on Speech Recognition for Vulnerable Individuals in Tamil (LT-EDI-2024) are discussed in this paper. The goal is to create an automated system for Tamil voice recognition. The older population that speaks Tamil is the source of the dataset used in this task. The proposed ASR system is designed with pre-trained model akashsivanandan/wav2vec2-large-xls-r300m-tamil-colab-final. The Tamil common speech dataset is utilized to fine-tune the pretrained model that powers our system. The suggested system receives the test data that was released from the task; transcriptions are then created for the test samples and delivered to the task. Word Error Rate (WER) is the evaluation statistic used to assess the provided result based on the task. Our Proposed system attained a WER of 29.297%.

ASR TAMIL SSN@ LT-EDI-2024: Automatic Speech Recognition system for Elderly People

Misogynistic memes are a category of memes which contain disrespectful language targeting women on social media platforms. Hence, detecting such memes is necessary in order to maintain a healthy social media environment. To address the challenges of detecting misogynistic memes, "Multitask Meme classification - Unraveling Misogynistic and Trolls in Online Memes: LT-EDI@EACL 2024" shared task organized at European Chapter of the Association for Computational Linguistics (EACL) 2024, invites researchers to develop models to detect misogynistic memes in Tamil and Malayalam. The shared task has two subtasks and in this paper, we - team MUCS, describe the learning models submitted to Task 1 - Identification of Misogynistic Memes in Tamil and Malayalam. As memes represent multi-modal data of image and text, three models: i)~Bidirectional Encoder Representations from Transformers (BERT)+Residual Network (ResNet)-50, ii) Multilingual Representations for Indian Languages (MuRIL)+ResNet-50, and iii) multilingual BERT (mBERT)+ResNet-50, are proposed based on joint representation of text and image, for detecting misogynistic memes in Tamil and Malayalam. Among the proposed models, mBERT+ResNet-50 and MuRIL+ ResNet-50 models obtained macro F1 scores of 0.73 and 0.87 for Tamil and Malayalam datasets respectively securing 1st rank for both the datasets in the shared task.

MUCS@LT-EDI-2024: Exploring Joint Representation for Memes Classification

This paper provides a comprehensive summary of the "Homophobia and Transphobia Detection in Social Media Comments" shared task, which was held at the LT-EDI@EACL 2024. The objective of this task was to develop systems capable of identifying instances of homophobia and transphobia within social media comments. This challenge was extended across ten languages: English, Tamil, Malayalam, Telugu, Kannada, Gujarati, Hindi, Marathi, Spanish, and Tulu. Each comment in the dataset was annotated into three categories. The shared task attracted significant interest, with over 60 teams participating through the CodaLab platform. The submission of prediction from the participants was evaluated with the macro F1 score.

Overview of Third Shared Task on Homophobia and Transphobia Detection in Social Media Comments

The overview of the shared task on speech recognition for vulnerable individuals in Tamil (LT-EDI-2024) is described in this paper. The work comes with a Tamil dataset that was gath- ered from elderly individuals who identify as male, female, or transgender. The audio sam- ples were taken in public places such as mar- ketplaces, vegetable shops, hospitals, etc. The training phase and the testing phase are when the dataset is made available. The task required of the participants was to handle audio signals using various models and techniques, and then turn in their results as transcriptions of the pro- vided test samples. The participant’s results were assessed using WER (Word Error Rate). The transformer-based approach was employed by the participants to achieve automatic voice recognition. This overview paper discusses the findings and various pre-trained transformer- based models that the participants employed

Overview of the Third Shared Task on Speech Recognition for Vulnerable Individuals in Tamil

This paper offers a detailed overview of the first shared task on "Multitask Meme Classification - Unraveling Misogynistic and Trolls in Online Memes," organized as part of the LT-EDI@EACL 2024 conference. The task was set to discern and classify misogynistic content and troll memes within online platforms, focusing specifically on memes in Tamil and Malayalam languages. A total of 52 teams registered for the competition, with four submitting systems for the Tamil meme classification task and three for the Malayalam task. The outcomes of this shared task are significant, providing insights into the current state of misogynistic content in digital memes and highlighting the effectiveness of various computational approaches in identifying such detrimental content. The top-performing model got a macro F1 score of 0.73 in Tamil and 0.87 in Malayalam.

Overview of Shared Task on Multitask Meme classification - Unraveling Misogynistic and Trolls in Online Memes

This paper presents an overview of the shared task on "Caste/Immigration Hate Speech Detection." The shared task is organized as part of LT-EDI@EACL 2024. The system must classify whether the text is caste/immigration hate speech or not. This shared task is conducted in the Tamil language. Fifty-one teams participated in the task in total. Out of that, 15 teams submitted their findings to the task. As far as we know, this is the first shared task that happened on caste/immigration hate speech detection in texts. In this paper, we analyzed and presented all the submissions submitted by the participants. This article includes an in-depth investigation and presentation of the submissions made by all participants.

Premium content

MEnTr@LT-EDI-2024: Multilingual Ensemble of Transformer Models for Homophobia/Transphobia Detection

Downloads

Next from EACL 2024

dkit@LT-EDI-2024: Detecting Homophobia and Transphobia in English Social Media Comments

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES