Large pre-trained language models achieve impressive results across many tasks. However, recent works point out that pre-trained language models may memorize a considerable fraction of their training data, leading to the privacy risk of information leakage. In this paper, we propose a method named Ethicist for targeted training data extraction through loss smoothed soft prompting and calibrated confidence estimation, investigating how to recover the suffix in the training data when given a prefix. To elicit memorization in the attacked model, we tune soft prompt embeddings while keeping the model fixed. We further propose a smoothing loss that smooths the loss distribution of the suffix tokens to make it easier to sample the correct suffix. In order to select the most probable suffix from a collection of sampled suffixes and estimate the prediction confidence, we propose a calibrated confidence estimation method, which normalizes the confidence of the generated suffixes with a local estimation. We show that Ethicist significantly improves the extraction performance on a recently proposed public benchmark. We also investigate several factors influencing the data extraction performance, including decoding strategy, model scale, prefix length, and suffix length. Our code is availabel at https://github.com/thu-coai/Targeted-Data-Extraction.

<iframe src="https://app.sli.do/event/1kgCXjyZMgdV89PsksaJsj/embed/polls/10813b4b-fed2-4c7e-8e27-d454ac5d465b" width="300" height="400"></iframe>

ETHICIST: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimation

Recent studies have shown the impressive efficacy of counterfactually augmented data (CAD) for reducing NLU models' reliance on spurious features and improving their generalizability. However, current methods still heavily rely on human efforts or task-specific designs to generate counterfactuals, thereby impeding CAD's applicability to a broad range of NLU tasks. In this paper, we present AutoCAD, a fully automatic and task-agnostic CAD generation framework. AutoCAD first leverages a classifier to unsupervisedly identify rationales as spans to be intervened, which disentangles spurious and causal features. Then, AutoCAD performs controllable generation enhanced by unlikelihood training to produce diverse counterfactuals. Extensive evaluations on multiple out-of-domain and challenge benchmarks demonstrate that AutoCAD consistently and significantly boosts the out-of-distribution performance of powerful pre-trained models across different NLU tasks, which is comparable or even better than previous state-of-the-art human-in-the-loop or task-specific CAD methods.

AutoCAD: Automatically Generate Counterfactuals for Mitigating Shortcut Learning

Ethics and NLP

technical paper

### Welcome to ACL 2023, the 61st Annual Meeting of the Association for Computational Linguistics! 
 The conference will be held in Toronto, Canada, July 9-14, 2023. 
Following the succession of the recent conferences in our field, ACL 2023 will adopt a hybrid format.
While the impact of Covid has considerably diminished in terms of traveling, obtaining visas to Canada
entails a very long process. Moreover, the global economic conditions pose challenges for many individuals to travel to conferences. Recognizing these circumstances, we know many participants may not be
able to attend the conference in person. Therefore, we are committed to providing a great virtual platform
so everyone has the opportunity to interact with other participants and enjoy the conference. Based on the
current registered participants, approxiately 30% have chosen to attend the conference virtually. Whether
you join us in person or virtually, we sincerely hope everyone has a remarkable conference experience. 
This General Chair’s message is where I express my gratitude to the many individuals who have made
enormous contributions to the conference over the past year.

Read [**ACL 2023 General Chair's message**](https://docs.google.com/document/d/1WobYM7norbG4dI48s75HfJoD89qgX5a_F-6U8AteLSA/edit?usp=sharing/) in full.

##### **[Conference Handbook](https://2023.aclweb.org/downloads/acl2023-handbook.pdf)**

ACL 2023

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing.

[![](https://assets.underline.io/uploads/markdown_image/1/image/26c36e70b5f083b72ae632cbdcc2b5b1.png)](https://us06web.zoom.us/rec/play/A2VNLbC1xoObGgGAXdJADePjdbrI4GfpM-olGlTMO5xp0BM1iN8lgRdm-37i8MQXEvDQHfEvE0hBkjRf.pIMa7Ex7beVmMK-E?autoplay=true&startTime=1670387560000) 

[![](https://assets.underline.io/uploads/markdown_image/1/image/7e41b6370eefd82b0bf5eb67e35a574b.png)](https://us06web.zoom.us/rec/play/CIr3QA3uTAXC4k83Z3LrY1PShqTc_tZI0yVgmzY4ofVG4e2lfSZAOLDzlKcs3qbDceTe8xcfriX6f91P.AdSAW4gIQjDk-2zh?autoplay=true&startTime=1670428311000) 

**Chairs:** Gyuwan KIm, Roy Schwartz, Iryna Gurevych, Roy Schwartz, Andreas Ruckle, Zornitsa Kozareva, Sujith Ravi, Sasha Luccioni and Angela Fan **Organizers:** Nafise Sadat Moosavi, Iryna Gurevych, Angela Fan, Yufang Hou, Zornitsa Kozareva, Sujith Ravi, Sasha Luccioni, Gyuwan Kim, Roy Schwartz, and Andreas Rücklé 
**Invited speakers**: [Hinrich Schutze](https://underline.io/speakers/40293-hinrich-schutze), [Percy Liang](https://underline.io/speakers/99010-percy-liang), [Kurt Keutzer](https://underline.io/speakers/214081-kurt-keutzer) and [Song Han](https://underline.io/speakers/61119-song-han) 
 Please visit our [**website**](https://sites.google.com/view/sustainlp2022/home) for more information and detailed schedule.

**RocketChat channel for this workshop is available here:** https://emnlp2022.rocket.chat/channel/workshop-11-sustainlp

W11 (SustaiNLP) The Third Workshop on Simple and Efficient Natural Language Processing

workshop paper

## Welcome to EMNLP 2022!
I am delighted to welcome you to EMNLP 2022! I believe this conference has been complicated beyond any precedent. Over the past year, it’s been thrilling to see the organization team approach each new puzzle with creativity and enthusiasm. We hope that those participating in Abu Dhabi as well as those joining remotely will leave the conference feeling newly inspired by the program and newly connected to our ever-growing community. Following EMNLP 2021 and major NLP conferences since, EMNLP 2022 is “hybrid,” serving both virtual and in-person participants.

Our key innovations for EMNLP 2022 include:

* EMNLP 2022 is “hybrid” in a second sense, as well: we allowed both direct and rolling review paper submissions, building on the pilot experiment of EMNLP 2021, which considered a small number of ARR submissions. 
* Familiar from NAACL but new to EMNLP, we’ve added an industry track.
* During the conference, “portals” will link virtual poster sessions to in-person conference participants during poster sessions each day.
* The first ACL-family conference in the United Arab Emirates.

 *Message from Noah A. Smith, University of Washington and Allen Institute for AI, Seattle, Washington, USA* 
***EMNLP 2022 General Chair***
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/9eec7d4a287ee18c278b08229290aa83.png)](https://drive.google.com/file/d/1OlPv6QBeo62VVTughj2jkiLeyHd1WnUt/view)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/a3db7a768409f05192210d98601edb25.png)](https://emnlp2022.rocket.chat/)

To access this site you need to register. Please register [here](https://2022.emnlp.org/registration/).

Register here

EMNLP 2022

Welcome!
EMNLP 2022 will take place in Abu Dhabi from December 7th to December 11th, 2022. And it will be held in hybrid mode, both online and offline.

Jiaxin Wen

3

3

SHORT BIO

Presentations

ETHICIST: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimation

AutoCAD: Automatically Generate Counterfactuals for Mitigating Shortcut Learning

AutoCAD: Automatically Generate Counterfactuals for Mitigating Shortcut Learning

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES