Austria

Despite growing interest in automated hate speech detection, most existing approaches overlook the linguistic diversity of online content. Multilingual instruction-tuned large language models such as LLaMA, Aya, Qwen, and BloomZ offer promising capabilities across languages, but their effectiveness in identifying hate speech through zero-shot and few-shot prompting remains underexplored. This work evaluates LLM prompting-based detection across eight non-English languages, utilizing several prompting techniques and comparing them to fine-tuned encoder models. We show that while zero-shot and few-shot prompting lag behind fine-tuned encoder models on most of the real-world evaluation sets, they achieve better generalization on functional tests for hate speech detection. Our study also reveals that prompt design plays a critical role, with each language often requiring customized prompting techniques to maximize performance.

ACL 2025

Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Study

workshop paper

### Welcome to The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

Message from the General Chair: 
*It is my great pleasure and honor to welcome you to the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025), held in beautiful Vienna, Austria, from July 27 to August 1, 2025. ACL2025continues our field’s tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.* 
[Read more](https://drive.google.com/file/d/1GI_hvOpjswAuYdUTromfeDiPpCcqidwg/view?usp=sharing)

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2025". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you need to log in with the **email address you registered with**. 

Welcome to The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

Conspiracist narratives posit an omnipotent, evil group causing harm throughout domains. However, modern-day online conspiracism is often more erratic, consisting of loosely connected posts displaying a general anti-establishment attitude pervaded by negative emotions. We gather a dataset of 300 conspiracist and mainstream, Telegram channels in Italian and English and use the automatic extraction of entities and emotion detection to compare structural characteristics of both types of channels. We create a co-occurrence network of entities to analyze how the different types of channels introduce and use them across posts and topics. We find that conspiracist channels are characterized by anger. Moreover, co-occurrence networks of entities appearing in conspiracist channels are more dense. We theorize that this reflects a narrative structure where all actants are pushed into a single domain. Conspiracist channels disproportionately associate the most central group of entities with anger and fear. We do not find evidence that entities in conspiracist narratives occur across more topics. This could indicate an erratic type of online conspiracism where everything can be connected to everything and that is characterized by a high number of entities and high levels of anger.

Multilingual Analysis of narrative characteristics in conspiracist vs mainstream Telegram channels

Hate speech detection is vital for creating safe online environments, as harmful content can drive social polarization. This study explores the impact of enriching text with intent and group tags on machine performance and human moderation workflows. For machine performance, we enriched text with intent and group tags to train hate speech classifiers. Intent tags were the most effective, achieving state-of-the-art F1-score improvements on the IHC, SBIC, and DH datasets, respectively. Cross-dataset evaluations further demonstrated the superior generalization of intent-tagged models compared to other pre-trained approaches. Then, through a user study (N=100), we evaluated seven moderation settings, including intent tags, group tags, model probabilities, and randomized counterparts. Intent annotations significantly improved the accuracy of the moderators, allowing them to outperform machine classifiers by 12.9%. Moderators also rated intent tags as the most useful explanation tool, with a 41% increase in perceived helpfulness over the control group. Our findings demonstrate that intent-based annotations enhance both machine classification performance and human moderation workflows.

Hate Explained: Evaluating NER-Enriched Text in Human and Machine Moderation of Hate Speech

Technology-facilitated gender-based violence has become a global threat to women's political representation and democracy. Understanding how online hate affects its targets is thus paramount. This study analyzes 10 million tweets directed at female candidates in the Brazilian election in 2022 and examines their reactions to online misogyny. Using a self-trained machine learning classifier to detect Portuguese misogynistic tweets and a quantitative analysis of the candidates' tweeting behavior, the study investigates how the number of misogynistic attacks received alters the online activity of the female candidates. The analysis finds that young and left-wing candidates and candidates with higher visibility online received significantly more attacks. Furthermore, the analysis finds that an increase in misogynistic attacks in the previous week is associated with a decrease in female candidates' tweets in the following week. This potentially threatens their equal participation in public opinion formation and silences women's voices in political discourse.

Online Misogyny Against Female Candidates in the 2022 Brazilian Elections: A Threat to Women's Political Representation?

Technology-facilitated violence constitutes a growing threat to inclusive democratic participation, particularly affecting women and marginalized groups. Drawing on a mixed-methods study from Brazil, this research examines how online harassment impacts politically active individuals, with a focus on gendered experiences and differential outcomes. A quantitative survey of 597 politically active individuals and seven in-depth interviews with female politicians were analyzed to assess the psychological and political consequences of technology-facilitated violence. More than 70% of survey respondents reported harassment within the past four weeks, with women disproportionately affected by gendered types of abuse. Regression results demonstrate that years of activism, education, and occupation significantly shape both exposure to harassment and coping responses. While some interviewees reported emotional exhaustion and disengagement, others described increased motivation and political resolve. These findings offer critical insights into how technology-facilitated violence shapes political behavior and underscore the need for platform accountability and institutional support structures.

Technology-Facilitated Violence and Political Participation: Evidence from a Mixed-Methods Study in Brazil

This work in progress presents an early-stage, conceptual exploration of how civil society actors can be meaningfully involved in the development of an AI-assisted tool for monitoring anti-democratic movements on Telegram. Using conspiracy theory classification as a use case, we outline and compare two concrete approaches to enabling such collaboration.

Civil Society in the Loop: Feedback-Driven Adaptation of (L)LM-Assisted Classification in an Open-Source Telegram Monitoring Tool

**(Civil Society Reports)**


Online hate speech propagation is a complex issue, deeply influenced by both the perpetrator and the target's cultural, historical, and societal contexts. Consequently, developing a universally robust hate speech classifier for diverse social media texts remains a challenging and unsolved task. The lack of mechanisms to track the spread and severity of hate speech further complicates the formulation of effective solutions. In response to this, to monitor hate speech in Indonesia during the recent 2024 presidential election, we have employed advanced
Natural Language Processing (NLP) technologies to create an improved hate speech classifier
tailored for a narrower subset of texts; specifically, texts that target vulnerable groups that
have historically been the targets of hate speech in Indonesia.

(Civil Society Reports) - Monitoring Hate Speech in Indonesia: An NLP-based Classification of Social Media Texts

**(Non-Archival Submission)**


Toxicity and polarization are often treated as distinct problems in NLP, despite their increasing overlap in online discourse. Indonesia, as the third-largest democracy in the world, has recently seen a sharp increase in both toxicity and polarizing messages online. We present a novel multi-labeled Indonesian dataset annotated for both toxicity and polarization, and with information on annotator demographics. Our benchmark results show that polarization cues help toxicity detection and vice versa. Including demographic features further improves toxicity and polarization classification, suggesting that identity and opinion dynamics are deeply intertwined in toxic and polarizing speech.

(Non-Archival Submission) A Multi-Labeled Dataset for Indonesian Discourse: Examining Toxicity, Polarization, and Demographics Information

In recent years, researchers have investigated the socio-demographic and personal factors influencing perceptions of toxic speech, with a primary focus on English-speaking contexts. We introduce a work-in-progress annotation study conducted in a German-language setting. Preliminary findings from our pre-study suggest that annotators' perceptions of the toxicity of statements correlate more strongly with their perceptions of the toxicity of other statements than with socio-demographic factors, attitudes toward freedom of speech, or perceived (dis-)similarity to the author and target of the statement. Subsequently, we aim to utilize this novel dataset to explore perspectivist fine-tuning approaches for LLMs.

Alignment by Disagreement? Toward Investigating LLMs' Adaptation to Personal and Sociodemographic Variability in the Perception of Toxicity

We present a novel approach for enhancing diversity and control in data annotation tasks by personalizing large language models (LLMs). We investigate the impact of injecting diverse persona descriptions into LLM prompts across two studies, exploring whether personas increase annotation diversity and whether the impacts of individual personas on the resulting annotations are consistent and controllable. Our results indicate that persona-prompted LLMs generate more diverse annotations than LLMs prompted without personas, and that the effects of personas on LLM annotations align with subjective differences in human annotations. These effects are both controllable and repeatable, making our approach a valuable tool for enhancing data annotation in subjective NLP tasks such as toxicity detection.

Personas with Attitudes: Controlling LLMs for Diverse Data Annotation

As large language models (LLMs) become increasingly prevalent, ensuring their robustness against adversarial misuse is crucial. This paper introduces the GAP (Graph of Attacks with Pruning) framework, an advanced approach for generating stealthy jailbreak prompts to evaluate and enhance LLM safeguards. GAP addresses limitations in existing tree-based methods by implementing an interconnected graph structure that enables knowledge sharing across attack paths. Our experimental evaluation demonstrates GAP's superiority over existing techniques, achieving a 20.8% increase in attack success rates while reducing query costs by 62.7%. GAP consistently outperforms state-of-the-art methods across various open and closed LLMs, with attack success rates of 
96%. Additionally, we present specialized variants like GAP-Auto for automated seed generation and GAP-VLM for multimodal attacks. GAP-generated prompts prove highly effective in improving content moderation systems, increasing true positive detection rates by 108.5% and accuracy by 183.6% when used for fine-tuning.

Downloads

Next from ACL 2025

Multilingual Analysis of narrative characteristics in conspiracist vs mainstream Telegram channels

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2025

Multilingual Analysis of narrative characteristics in conspiracist vs mainstream Telegram channels

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads