Thailand

In this paper we implement and compare 7 different data augmentation strategies for the task of automatic scoring of children&#39;s ability to understand others&#39; thoughts, feelings, and desires (or ``mindreading&#39;&#39;). 

We recruit in-domain experts to re-annotate augmented samples and determine to what extent each strategy preserves the original rating. We also carry out multiple experiments to measure how much each augmentation strategy improves the performance of automatic scoring systems. To determine the capabilities of automatic systems to generalize to unseen data, we create UK-MIND-20 - a new corpus of children&#39;s performance on tests of mindreading, consisting of 10,320 question-answer pairs.

We obtain a new state-of-the-art performance on the MIND-CA corpus, improving macro-F1-score by 6 points. Results indicate that both the number of training examples and the quality of the augmentation strategies affect the performance of the systems. The task-specific augmentations generally outperform task-agnostic augmentations. Automatic augmentations based on vectors (GloVe, FastText) perform the worst. 

We find that systems trained on MIND-CA generalize well to UK-MIND-20. We demonstrate that data augmentation strategies also improve the performance on unseen data.

ACL 2021

Can vectors read minds better than experts? Comparing data augmentation strategies for the automated scoring of children’s mindreading ability

mindreading

theory of mind

automatic scoring

psycholinguistics

data augmentation

**Welcome to ACL-IJCNLP 2021!**

The great event is jointly organized by the Association for Computational Linguistics (ACL) and Asian Federation of Natural Language Processing (AFNLP). 

This Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021) was to be held in Bangkok, Thailand, during August 1-6, 2021 and is now virtual on the Underline platform.

As in previous years, the program of the conference includes a poster session, tutorials, workshops and demonstrations in addition to the main conference.


We were able to keep the registration fees similar to those charged for the virtual ACL 2020. The one fee allows attendance at the main conference and any/all tutorials and workshops. These fees would be $125 Regular Early and $175 Regular Late; $50 Student Early and $75 Student Late. Early registration closes at midnight July 11, 2021 (Eastern Daylight Time).

**Reminder:** It is ACL’s policy that at least one author of each accepted paper (including ACL Finding papers) must register for the conference.

**Reminder2:** Underline site will open closer to the event. If you already registered you will receive access detail

Registration is now open

In this paper we implement and compare 7 different data augmentation strategies for the task of automatic scoring of children's ability to understand others' thoughts, feelings, and desires (or ``mindreading''). 

We recruit in-domain experts to re-annotate augmented samples and determine to what extent each strategy preserves the original rating. We also carry out multiple experiments to measure how much each augmentation strategy improves the performance of automatic scoring systems. To determine the capabilities of automatic systems to generalize to unseen data, we create UK-MIND-20 - a new corpus of children's performance on tests of mindreading, consisting of 10,320 question-answer pairs.

We obtain a new state-of-the-art performance on the MIND-CA corpus, improving macro-F1-score by 6 points. Results indicate that both the number of training examples and the quality of the augmentation strategies affect the performance of the systems. The task-specific augmentations generally outperform task-agnostic augmentations. Automatic augmentations based on vectors (GloVe, FastText) perform the worst. 

We find that systems trained on MIND-CA generalize well to UK-MIND-20. We demonstrate that data augmentation strategies also improve the performance on unseen data.

poster

Uncovering Constraint-Based Behavior in Neural Models via Targeted Fine-Tuning

More Identifiable yet Equally Performant Transformers for Text Classification

AugNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation

A Span-based Dynamic Local Attention Model for Sequential Sentence Classification

How effective is BERT without word ordering? Implications for language understanding and data privacy

Predicting pragmatic discourse features in the language of adults with autism spectrum disorder

Adversarial Datasets for NLI Tasks: the Case of the Chinese Causative-Passive Homonymy

SUMPUBMED: Summarization Dataset of PubMed Scientific Articles

Topicalization in Language Models: A Case Study on Japanese

A Case Study of Analysis of Construals in Language on Social Media Surrounding a Crisis Event

workshop paper

https://d3smihljt9218e.cloudfront.net/transcript/17864/subtitle/b28452e6a90cb92d3ed6411d089106e0.vtt

English (automatic)

Can vectors read minds better than experts? Comparing data augmentation strategies for the automated scoring of children’s mindreading ability

Downloads

Next from ACL 2021

Uncovering Constraint-Based Behavior in Neural Models via Targeted Fine-Tuning

Similar lecture

Learning to Perturb Word Embeddings for Out-of-distribution QA