This paper presents interdisciplinary and participatory research conducted with laryngectomees with an aim to widen research paradigms for cognitive sciences. Studying daily communication in an interactive context of people living without larynx, we put special emphasis on methodology combining engaged epistemology with ethnographic fieldwork, novel in the domain of this specific perturbation. Results valuable for designers, therapists and research participants themselves demonstrate that the greater awareness is needed of i) influences of anatomical and physiological adaptations on communication and social relations, ii) multimodal and context-dependent alternative strategies of conversation, iii) the crucial role of participants’ agency. We claim that some of our unexpected observations were possible, because of choosing the research approach that extended our epistemological capacities, namely by remaining open to novel interpretive categories and perspectives. We describe how dialogue between epistemologically engaged cognitive science and ethnographic fieldwork facilitated our understanding of the problem in a specific case.

Researching Communication in Context: Engaged Epistemology and Ethnographic Fieldwork Transforms Understanding of Interactions after Laryngectomy

Elman Prize Symp: Introduction

The meanings of iconic signs are usually not easily accessible to sign-naive people. However, most previous studies asked participants to guess the meaning of iconic signs in isolation and without any contextual cues. We ask whether signs whose form is based on more cross-linguistically common underlying motivations are easier to interpret than signs based on less common underlying motivations. Since recent research suggests that iconicity is a relationship of resemblance between the signifier and the signified that is instantiated contextually, we also provide participants with a contextual cue (in the form of a word). We find that interpretability of iconic signs does correlate with cross-linguistic frequency of the underlying motivation of the sign.

Transparency in Sign Language Animal Signs

The purpose of this study is to examine Theory of Mind (ToM) in processing the Chinese implicit causality verbs. The Experiment identified the Chinese causal verbs of high tendency with a bias decision task and used the ToM paradigms: Reading the Mind in the Eyes test revised and YONI task, to explore the relationship between English proficiency, Theory of Mind, and causal verb processing.
The main results are: Proficiency in both English and Chinese may affect one’s interpretation of the Chinese implicit causality/consequentiality, while on the other hand, the affective ability of ToM is enhanced as the English proficiency increases due to the understanding of the implicit causes/effects of the same concept under the context of the different languages. The study infers that the process of Chinese and English are different and Bilingualism can uses two mechanism at the same time compared to 
monolingual.

Implicit Causality in Chinese Verbs: The Correlations between Bilingualism and Theory of Mind

In this paper, we present a dataset of subjective views (beliefs and attitudes) held by individuals or groups. We analyze the usefulness of the dataset by training a neural classifier that identifies belief-containing sentences that are relevant for our broader project of interest---scientific modeling of complex systems. We also explore and discuss difficulties related to annotation of subjective views and propose ways of addressing them.

Annotating and Training for Population Subjective Views

Despite their widespread adoption, neural conversation models have yet to exhibit natural chat capabilities with humans. In this research, we examine user utterances as causes and generated responses as effects, recognizing that changes in a cause should produce a different effect. To further explore this concept, we have compiled and expanded upon a new dataset called CausalDialogue through crowd-
sourcing. This dataset includes multiple cause-effect pairs within a directed acyclic graph (DAG) structure. Our analysis reveals that traditional loss functions struggle to effectively incorporate the DAG structure, leading us to propose a causality-enhanced method called Exponential Maximum Average Treatment Effect (ExMATE) to enhance the impact of causality at the utterance level in training neural conversation models. To evaluate the needs of considering causality in dialogue generation, we built a comprehensive benchmark on CausalDialogue dataset using different models, inference, and training methods. Through experiments, we find that a causality-inspired loss like ExMATE can improve the diversity and agility of conventional loss function and there is still room for improvement to reach human-level quality on this new dataset.

CausalDialogue: Modeling Utterance-level Causality in Conversations

In recent years, we have seen many Transformer based models being created to address Dialog Summarization problem. While there has been a lot of work on understanding how these models stack against each other in summarizing regular conversations such as the ones found in DialogSum dataset, there haven't been many analysis of these models on Clinical Dialog Summarization. In this article, we describe our solution to MEDIQA-Chat 2023 Shared Tasks as part of ACL-ClinicalNLP 2023 workshop which benchmarks some of the popular Transformer Architectures such as BioBart, Flan-T5, DialogLED, and OpenAI GPT3 on the problem of Clinical Dialog Summarization. We analyse their performance on two tasks - summarizing short conversations and long conversations. In addition to this, we also benchmark two popular summarization ensemble methods and report their performance.

HealthMavericks@MEDIQA-Chat 2023: Benchmarking different Transformer based models for Clinical Dialogue Summarization

It is common sense that one should prefer to eat a salad with a fork rather than with a chainsaw. However, for eating a bowl of rice, the choice between a fork and a pair of chopsticks is culturally relative. We introduce FORK, a small, manually-curated set of CommonsenseQA-style questions for probing cultural biases and assumptions present in commonsense reasoning systems, with a specific focus on food-related customs. We test several CommonsenseQA systems on FORK, and while we see high performance on questions about the US culture, the poor performance of these systems on questions about non-US cultures highlights systematic cultural assumptions aligned with US over non-US cultures.

FORK: A Bite-Sized Test Set for Probing Culinary Cultural Biases in Commonsense Reasoning Models

We present the Verifee dataset: a multimodal dataset of news articles with fine-grained trustworthiness annotations. We bring a diverse set of researchers from social, media, and computer sciences aboard to study this interdisciplinary problem holistically and develop a detailed methodology that assesses the texts through the lens of editorial transparency, journalist conventions, and objective reporting while penalizing manipulative techniques. We collect over 10,000 annotated articles from 60 Czech online news sources. Each item is categorized into one of the 4 proposed classes on the credibility spectrum -- ranging from entirely trustworthy articles to deceptive ones -- and annotated of its manipulative attributes. We fine-tune prominent sequence-to-sequence language models for the trustworthiness classification task on our dataset and report the best F-1 score of 0.53. We open-source the dataset, annotation methodology, and annotators' instructions in full length at https://www.verifee.ai/research/ to enable easy build-up work.

Czech-ing the News: Article Trustworthiness Dataset for Czech

Linguistic analysis is a core task in the process of documenting, analyzing, and describing endangered and less-studied languages. 
In addition to providing insight into the properties of the language being studied, having tools to automatically label words in a language for the grammatical category and morphological features can support a range of applications useful for language pedagogy and revitalization.
At the same time, most modern NLP methods for these tasks require both large amounts of data in the language and compute costs well beyond the capacity of most research groups and language communities.

Here, we present a gloss-to-gloss (g2g) model for linguistic analysis (specifically, morphological analysis and part-of-speech tagging) that is lightweight in terms of both data requirements and computational expense.
The model is designed for the interlinear glossed text (IGT) format, in which we expect the source text of a sentence in a low-resource language, a translation of that sentence into a language of wider communication, and a detailed glossing of the morphological properties of each word in the sentence.

We test the model on both low-resource and high-resource languages and find that our simple CNN-based model achieves comparable performance to a state-of-the-art transformer-based model, at a fraction of the computational cost.

Lightweight morpheme labeling in context: Using structured linguistic representations to support linguistic analysis for the language documentation context

In this paper, we highlight our approach for the "WASSA 2023 Empathy and Emotion Shared Task". We present our approach for track 3 of the shared task which aims to identify emotions from text. Each sample in the dataset has one or more labels making it a multi-label classification task. We compared multiple transformer-based models by fine-tuning them for multi-label classification. Oversampling was used to overcome the class imbalance in the dataset. Ensembling techniques were used to improve the performance of the system. We obtained a macro F1-score of 0.5649 using XLNet on the test dataset in the official phase and secured rank 6 on the official leaderboard. During the post-competition phase, a threshold-based voting mechanism was performed on three models (Longformer, BERT, BigBird) that yielded the highest overall macro F1-score of 0.6605.

Converge at WASSA 2023 Empathy, Emotion and Personality Shared Task: A Transformer-based Approach for Multi-Label Emotion Classification

Annotated clinical text corpora are essential for machine learning studies that model and predict care processes and disease progression. However, few studies describe the necessary experimental design of the annotation guideline and annotation phases. This makes replication, reuse, and adoption challenging.

Using clinical questions about sepsis, we designed a semantic annotation guideline to capture sepsis signs from clinical text. The clinical questions aid guideline design, application, and evaluation. Our method incrementally evaluates each change in the guideline by testing the resulting annotated corpus using clinical questions. Additionally, our method uses inter-annotator agreement to judge the annotator compliance and quality of the guideline. We show that the method, combined with controlled design increments, is simple and allows the development and measurable improvement of a purpose-built semantic annotation guideline. We believe that our approach is useful for incremental design of semantic annotation guidelines in general.

Underline digital video library

Researching Communication in Context: Engaged Epistemology and Ethnographic Fieldwork Transforms Understanding of Interactions after Laryngectomy

Elman Prize Symp: Introduction

Transparency in Sign Language Animal Signs

Implicit Causality in Chinese Verbs: The Correlations between Bilingualism and Theory of Mind

Annotating and Training for Population Subjective Views

CausalDialogue: Modeling Utterance-level Causality in Conversations

HealthMavericks@MEDIQA-Chat 2023: Benchmarking different Transformer based models for Clinical Dialogue Summarization

FORK: A Bite-Sized Test Set for Probing Culinary Cultural Biases in Commonsense Reasoning Models

Czech-ing the News: Article Trustworthiness Dataset for Czech

Lightweight morpheme labeling in context: Using structured linguistic representations to support linguistic analysis for the language documentation context

Converge at WASSA 2023 Empathy, Emotion and Personality Shared Task: A Transformer-based Approach for Multi-Label Emotion Classification

Method for Designing Semantic Annotation of Sepsis Signs in Clinical Text

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES