Taiwan

Universal Sentence Encoder (USE) has gained much popularity recently as a general-purpose sentence encoding technique. As the name suggests, USE is designed to be fairly general and has indeed been shown to achieve superior performances for many downstream NLP tasks. In this paper, we present an interesting &quot;negative&#39;&#39; result on USE in the context of zero-shot text classification, a challenging task, which has recently gained much attraction. More specifically, we found some interesting cases of zero-shot classification, where topic based inference outperformed USE-based inference in terms of F1 score. Further investigation revealed that USE struggles to perform well on datasets with a large number of labels with high semantic overlaps, while topic-based classification works well for the same.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.18/)

IJCNLP-AACL 2022

Exploring Universal Sentence Encoders for Zero-shot Text Classification

universal sentene encoder

sentene encoder

zero-shot

annotation

stages 

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes, but also in industrial research departments and in technological startups. COLING conferences provide opportunities for all these communities to showcase their exciting developments.

COLING 2020

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

Universal Sentence Encoder (USE) has gained much popularity recently as a general-purpose sentence encoding technique. As the name suggests, USE is designed to be fairly general and has indeed been shown to achieve superior performances for many downstream NLP tasks. In this paper, we present an interesting "negative'' result on USE in the context of zero-shot text classification, a challenging task, which has recently gained much attraction. More specifically, we found some interesting cases of zero-shot classification, where topic based inference outperformed USE-based inference in terms of F1 score. Further investigation revealed that USE struggles to perform well on datasets with a large number of labels with high semantic overlaps, while topic-based classification works well for the same.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.18/)

technical paper

The AACL, Asia-Pacific Chapter of the Association for Computational Linguistics, and the AFNLP, Asian Federation of Natural Language Processing, invite you to participate in the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL) and the 10th International Joint Conference on Natural Language Processing (IJCNLP). 
<br>
<br>

A registration is required to access this event. Access to the event site will be granted to all registered attendees on **November 16th**. If you have not already done so, please register using the link below:
<br>
<br>https://conference.iis.sinica.edu.tw/servlet/Register?ConferenceID=411

Registration Is Required

The AACL, Asia-Pacific Chapter of the Association for Computational Linguistics, and the AFNLP, Asian Federation of Natural Language Processing, invite you to participate in the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL) and the 10th International Joint Conference on Natural Language Processing (IJCNLP). 

BERT-like language models (LMs), when exposed to large unstructured datasets, are known to learn and sometimes even amplify the biases present in such data. These biases generally reflect social stereotypes with respect to gender, race, age, and others. In this paper, we analyze the variations in gender and racial biases in BERT, a large pre-trained LM, when exposed to different demographic groups. Specifically, we investigate the effect of fine-tuning BERT on text authored by historically disadvantaged demographic groups in comparison to that by advantaged groups. We show that simply by fine-tuning BERT-like LMs on text authored by certain demographic groups can result in the mitigation of social biases in these LMs against various target groups.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.38/)

Demographic-Aware Language Model Fine-tuning as a Bias Mitigation Technique

Conversational AI systems can engage in unsafe behaviour when handling users' medical queries that may have severe consequences and could even lead to deaths.
Systems therefore need to be capable of both recognising the seriousness of medical inputs and producing responses with appropriate levels of risk. 
We create a corpus of human written English language medical queries and the responses of different types of systems. We label these with both crowdsourced and expert annotations.
While individual crowdworkers may be unreliable at grading the seriousness of the prompts, their aggregated labels tend to agree with professional opinion to a greater extent on identifying the medical queries and recognising the risk types posed by the responses.Results of classification experiments suggest that, while these tasks can be automated, caution should be exercised, as errors can potentially be very serious.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.30/)

Risk-graded Safety for Handling Medical Queries in Conversational AI

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.44/)

Modeling Document-level Temporal Structures for Building Temporal Dependency Graph

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.23/)

Improving Chinese Story Generation via Awareness of Syntactic Dependencies and Semantics

To improve the performance of long text generation, recent studies have leveraged automatically planned event structures (i.e. storylines) to guide story generation. Such prior works mostly employ end-to-end neural generation models to predict event sequences for a story. However, such generation models struggle to guarantee the narrative coherence of separate events due to the hallucination problem, and additionally the generated event sequences are often hard to control due to the end-to-end nature of the models. To address these challenges, we propose NGEP, an novel event planning framework which generates an event sequence by performing inference on an automatically constructed event graph and enhances generalisation ability through a neural event advisor. We conduct a range of experiments on multiple criteria, and the results demonstrate that our graph-based neural framework outperforms the state-of-the-art (SOTA) event planning approaches, considering both the performance of event sequence generation and the effectiveness on the downstream task of story generation.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.24/)

NGEP: A Graph-based Event Planning Framework for Story Generation

Pre-trained language models such as KoBART often fail to generate perfect interrogative sentences when they are applied to Korean question generation. 
This is mainly due to the fact that the language models are trained with declarative sentences, but not with interrogative sentences. 
Therefore, this paper proposes a novel post-training of KoBART to enhance it for Korean question generation. 
The enhancement of KoBART is accomplished in three ways: 
(i) introduction of question infilling objective to KoBART to enforce it to focus more on the structure of interrogative sentences, 
(ii) augmentation of training data for question generation with another MRC data from AI-Hub to cope with the lack of training instances for post-training, 
(iii) introduction of Korean spacing objective to make KoBART understand the linguistic features of Korean. 
Since there is no standard data set for Korean question generation, this paper also proposes KorQuAD-QG, 
a new data set for this task, to verify the performance of the proposed post-training. 
Our code are publicly available at https://github.com/gminipark/post_training_qg.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.26/)

Post-Training with Interrogative Sentences for Enhancing BART-based Korean Question Generator

A bottleneck to developing Semantic Parsing (SP) models is the need for a large volume of human-labeled training data. Given the complexity and cost of human annotation for SP, labeled data is often scarce, particularly in multilingual settings. Large Language Models (LLMs) excel at SP given only a few examples, however LLMs are unsuitable for runtime systems which require low latency. In this work, we propose CLASP, a simple method to improve low-resource SP for moderate-sized models: we generate synthetic data from AlexaTM 20B to augment the training set for a model 40x smaller (500M parameters). We evaluate on two datasets in low-resource settings: English PIZZA, containing either 348 or 16 real examples, and mTOP cross-lingual zero-shot, where training data is available only in English, and the model must generalize to four new languages. On both datasets, we show significant improvements over strong baseline methods.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.56/)

CLASP: Few-Shot Cross-Lingual Data Augmentation for Semantic Parsing

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.55/)

An Empirical Study of Pipeline vs. Joint approaches to Entity and Relation Extraction

We address the challenging task of Localization via Embodied Dialog (LED). Given a dialog from two agents, an Observer navigating through an unknown environment and a Locator who is attempting to identify the Observer's location, the goal is to predict the Observer's final location in a map of the environment. We present the novel LED-Bert architecture and demonstrate an effective pre-training strategy from disembodied web data which allows our method to achieve state of the art performance on the task. Additionally during experimentation we find that using a graph-based map representation is more effective than the top-down 2D maps used in prior works.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.36/)

Transformer-based Localization from Embodied Dialog with Large-scale Pre-training

Topic-sensitive query set expansion is an important area of research that aims to improve search results for information retrieval. It is particularly crucial for queries related to sensitive and emerging topics. In this work, we describe a method for query set expansion about emerging topics using vector space interpolation. We use a transformer model called OPTIMUS, which is suitable for vector space manipulation due to its variational autoencoder nature. One of our proposed methods -- Dirichlet interpolation shows promising results for query expansion. Our methods effectively generate new queries about the sensitive topic by incorporating set-level diversity, which is not captured by traditional sentence-level augmentation methods such as paraphrasing or back-translation.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.50/)

Exploring Universal Sentence Encoders for Zero-shot Text Classification

Downloads

Next from IJCNLP-AACL 2022

Demographic-Aware Language Model Fine-tuning as a Bias Mitigation Technique

Similar lecture

Universal Dependency Treebank for Xibe

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Exploring Universal Sentence Encoders for Zero-shot Text Classification

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from IJCNLP-AACL 2022

Demographic-Aware Language Model Fine-tuning as a Bias Mitigation Technique

Similar lecture

Universal Dependency Treebank for Xibe

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads