Mexico

Safety classifiers are critical in mitigating toxicity on online forums such as social media and in chatbots. 
Still, they continue to be vulnerable to emergent, and often innumerable, adversarial attacks.
Traditional automated adversarial data generation methods, however, tend to produce attacks that are not diverse, but variations of previously observed harm types.
We formalize the task of automated adversarial discovery for safety classifiers - to find new attacks along previously unseen harm dimensions that expose new weaknesses in the classifier.
We measure progress on this task along two key axes (1) adversarial success: does the attack fool the classifier? and (2) dimensional diversity: does the attack represent a previously unseen harm type?
Our evaluation of existing attack generation methods on the CivilComments toxicity task reveals their limitations: Word perturbation attacks fail to fool classifiers, while prompt-based LLM attacks have more adversarial success, but lack dimensional diversity.
Even our best-performing prompt-based method finds new successful attacks on unseen harm dimensions of attacks only 5\% of the time.
Automatically finding new harmful dimensions of attack is crucial and there is substantial headroom for future research on our new task.

NAACL 2024

Automated Adversarial Discovery for Safety Classifiers

safety classifiers

adversarial discovery

ai safety

workshop paper

## Welcome to 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics!
This year, the conference is in Mexico City. NAACL was actually already planned for Mexico City in 2021, but due to the pandemic the entire conference was moved online. This year, finally, we get to go! So it is my sincere pleasure to welcome you to Mexico City, whether in person or virtually. Having the conference in Mexico City is a good opportunity to emphasize that NAACL is our flagship conference for ACL members not only in North America but also in Central and South America, even though NAACL has been bearing “North” in its name. At this year’s conference, we have a theme to match, with a theme track on the Languages of Latin America to showcase the linguistic diversity of the region.
 
The opportunity to present at NAACL should not depend on a researcher’s travel budget, or their family status. This is why it is so important to make virtual participation at NAACL as good an experience as possible – but we want to also provide a good experience for in-person participants. As a community, we are still working out the best way to do that. This year at NAACL, we are trying out a big virtual poster session ahead of the conference, with the hope that this will make make for a lively and interactive experience. At the same time, we are reducing virtual oral presentations, which seem to be particularly tricky to make to work well. A big thanks to the NAACL program chairs and to Luciana Benotti for all their ideas and work to improve the virtual experience. And participants, virtual as well as in-person: Please let us know what worked for you and what didn’t, so we can continue to improve hybrid conferences. 
 
I have been lucky to work with many amazing people. Without their insight, dedication and patience, and without the many hours of work they put in, NAACL would not have been possible. A huge thank you to the program chairs Helena Gomez, Kevin Duh, and Steve Bethard – you are the best! 
 
Finally, I would like to thank all authors, invited speakers and panelists, area chairs and reviewers, the volunteers organizing and chairing sessions, and all attendees, in-person and virtual. Thank you for helping us make NAACL 2024 come to life. 
 
Welcome and hope you all enjoy the conference! 
Katrin Erk 
The University of Texas at Austin 
NAACL-2024 General Chair 
*You can read the full Welcome message in the [Conference Handbook (downloadable)](https://drive.google.com/file/d/1H1NvW0VASQjkSCYgw3mr-3l4yYDSxz6B/view?usp=sharing)*

To access the event page you need to register [**here.**](http://acl.swoogo.com/naacl2024) 
Your access to the event page is limited based on your registration type. If you registered for workshops only, you will gain access to full workshops content on the day of the workshops program.

Please register!

2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics

Current natural language processing (NLP) research tends to focus on only one or, less frequently, two dimensions -- e.g., performance, interpretability, or efficiency -- at a time, which may lead to suboptimal conclusions. Work on adapter modules
focuses on improving performance and efficiency, with no investigation of unintended consequences on other aspects such as fairness. To address this gap, we conduct experiments on three text classification datasets by either (1) finetuning all parameters or (2) using adapter modules. Regarding performance and efficiency, we confirm prior findings that the accuracy of adapter-enhanced models is roughly on par with that of fully finetuned models, while training time is substantially reduced. Regarding fairness, we show that adapter modules result in mixed fairness across sensitive groups. Further investigation reveals that, when the standard finetuned model exhibits limited biases, adapter modules typically do not introduce extra bias. On the other hand, when the finetuned model exhibits increased bias, the use of adapter modules poses the potential danger of amplifying these biases to a significant extent. Our findings highlight the need for a case-by-case evaluation rather than a one-size-fits-all judgment.

The Trade-off between Performance, Efficiency, and Fairness in Adapter Modules for Text Classification

The immense attraction towards text generation garnered by ChatGPT has spurred the need for discriminating machine-text from human text. In this work, we provide preliminary evidence that the scores computed by existing zero-shot and supervised machine-generated text detection methods are not solely determined by the generated texts, but are affected by prompts and real texts as well. Using techniques from causal inference, we show the existence of backdoor paths that confounds the relationships between text and its detection score and how the confounding bias can be partially mitigated. We open up new research directions in identifying other factors that may be interwoven in the detection of machine text. Our study calls for a deeper investigation into which kinds of prompts make the detection of machine text more difficult or easier

Exploring Causal Mechanisms for Machine Text Detection Methods

Gender biases can manifest in language agency: men are often described as "agentic" and achievement-oriented, whereas women are frequently described as "communal" and more social-oriented or service-oriented. Previous studies identify agency level by string matching for words in corresponding lexicons, and investigate agency in a limited scope. We adopt a model-based method to calculate "language agency" at sentence level in a broader context of natural language, and propose it as a common theme for identifying and analyzing stylistic gender biases in texts. Specifically, we contribute a novel ChatGPT-generated and human-reannotated Language Agency Classification dataset for training reliable agency classifiers. We then apply a classifier trained on our dataset to a reference letter dataset, a biography dataset, and a RateMyProfessor dataset to reveal significant language agency gender biases. We observe that on the biography dataset, the most significant language agency biases lie in biographies for individuals with occupations such as pastor, architect, and software engineer. On the RateMyProfessor dataset, severe language agency biases lie in reviews for professors in Accounting, Sociology, and Chemistry departments. Our findings reveal complex patterns of gender biases through the lens of language agency, proving its efficacy in providing deeper insights into biases in natural language texts.

Men Who Lead And Women Who Help: The Manifestation of Gender Bias In Language Agency

Recent studies reveal that Large Language Models (LLMs) face challenges in balancing safety with utility, particularly when processing long texts for NLP tasks like summarization and translation. Despite defenses against malicious short questions, the ability of LLMs to safely handle dangerous long content, such as manuals teaching illicit activities, remains unclear. Our work aims to develop robust defenses for LLMs in processing malicious documents alongside benign NLP task queries. We introduce a defense dataset comprised of safety-related examples and propose single-task and mixed-task losses for instruction tuning. Our empirical results demonstrate that LLMs can significantly enhance their capacity to safely manage dangerous content with appropriate instruction tuning. Additionally, strengthening the defenses of tasks most susceptible to misuse is effective in protecting LLMs against processing harmful information. We also observe that trade-offs between utility and safety exist in defense strategies, where Llama2, utilizing our proposed approach, displays a significantly better balance compared to Llama1.

Cross-Task Defense: Instruction-Tuning LLMs for Content Safety

The emergence of several contemporary text-to-image generation models such as DALL-E and Stable Diffusion has demonstrated remarkable proficiency in producing high-quality images. While these generated images have been used to improve text quality in natural language generation (NLG) tasks via visual augmentation, parallel research endeavors have found biases within these generated images. Conversely, image-to-text models, grounded in large language models (LLMs), excel in crafting vivid descriptions of images using high-quality language, albeit inheriting the biases inherent in LLMs. This research explores how these biases are amplified when generated images are used as input for image-to-text generation models. Through empirical analysis, we show that by feeding biased images into image-to-text models, the generated response becomes even more biased.

Beyond Visual Augmentation: Investigating Bias in Multi-Modal Text Generation

Large Language Models (LLMs) have revolutionized the field of AI by their ability to understand and generate human-like text, and ensuring their reliability is crucial. This paper aims to evaluate the ability of LLMs to estimate
their uncertainty in natural language processing (NLP) tasks. Furthermore, this paper extends the topic by evaluating the newly released Vision Language Models (VLMs) and their ability to estimate verbalized uncertainty in an image recognition task. To investigate this, four LLMs are tested on three different NLP tasks. For all
tasks, the models are prompted to express their confidence level for each answer. Additionally, two VLMs are similarly tested on a novel image recognition dataset. The results show that both the LLMs and VLMs have a high calibration error and are overconfident most of the time, indicating a poor capability for uncertainty es-
timation. The findings of this study provide a foundational basis for future research in enhancing uncertainty estimation methods within LLMs and VLMs.

Overconfidence is Key: Verbalized Uncertainty Evaluation in Large Language and Vision-Language Models

Large language models (LLMs), trained on vast datasets, can carry biases that manifest in various forms, from overt discrimination to implicit stereotypes. One facet of bias is performance disparities in LLMs, often harming underprivileged groups, such as racial minorities. A common approach to quantifying bias is to use template-based bias probes, which explicitly state group membership (e.g. White) and evaluate if the outcome of a task, sentiment analysis for instance, is invariant to the change of group membership (e.g. change White race to Black). This approach is widely used in bias quantification. However, in this work, we find evidence of an unexpectedly overlooked consequence of using template-based probes for LLM bias quantification. We find that in doing so, text examples associated with White ethnicities appear to be classified as exhibiting negative sentiment at elevated rates. We hypothesize that the scenario arises artificially through a mismatch between the pre-training text of LLMs and the templates used to measure bias through reporting bias, unstated norms that imply group membership without explicit statement. Our finding highlights the potential misleading impact of varying group membership through explicit mention in bias quantification.

Reevaluating Bias Detection in Language Models: The Role of Implicit Norms

Modern large language models (LLMs) have a significant amount of world knowledge, which enables strong performance in commonsense reasoning and knowledge-intensive tasks when harnessed properly. The language model can also learn social biases, which has a significant potential for societal harm. There have been many mitigation strategies proposed for LLM safety, but it is unclear how effective they are for eliminating social biases. In this work, we propose a new methodology for attacking language models with knowledge graph-augmented generation. We refactor natural language stereotypes into a knowledge graph, and use adversarial attacking strategies to induce biased responses from several open- and closed-source language models. We find our method increases bias in all models, even those trained with safety guardrails. This demonstrates the need for further research in AI safety, and we encourage further work in this new adversarial space.

BiasKG: Adversarial Knowledge Graphs to Induce Bias in Large Language Models

Amidst growing popularity of language models (LMs), the appropriate communication of uncertainty is crucial to reliable human-AI interactions. Existing work has examined models' abilities to communicate their internal uncertainty but does not consider how LMs interpret communicated uncertainty. In this paper, we investigate the ability of LMs to understand the degree of confidence expressed in language. To this end, we systematically analyse how models' behaviors change with the use of different probability phrases (e.g., likely, doubtful, unlikely) and compare LMs' perceptions to previously reported human perceptions. We find that, while popular LMs are able to rank probability phrases, models struggle to assign accurate estimates for each probability phrase, failing to distinguish phrases, such as doubtful, uncertain, probable. Furthermore, ChatGPT tends to mirror human perceptions, whilst perceptions of Llama-2 7B and Gemma 7B reveal disparate behaviors across different experiments.

Can Language Models Interpret Verbalized Uncertainty?

As the utilization of large language models (LLMs) has proliferated worldwide, it is crucial for them to have adequate knowledge and fair representation for diverse global cultures. In this work, we uncover culture perceptions of three SOTA models on 110 countries and regions on 8 culture-related topics through culture-conditioned generations, and extract symbols from these generations that are associated to each culture by the LLM. We discover that culture-conditioned generation consist of linguistic "markers" that distinguish marginalized cultures apart from default cultures. We also discover that LLMs have an uneven degree of diversity in the culture symbols, and that cultures from different geographic regions have different presence in LLMs' culture-agnostic generation. Our findings promote further research in studying the knowledge and fairness of global culture perception in LLMs. Code and Data can be found in: https://github.com/huihanlhh/Culture-Gen/

Premium content

Automated Adversarial Discovery for Safety Classifiers

Next from NAACL 2024

The Trade-off between Performance, Efficiency, and Fairness in Adapter Modules for Text Classification

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES