Selective avoidance of uncongenial facts to users’ preexisting false belief is a biased click tendency that detracts the utility of fact-checking websites. This study examined the strength of this avoidance tendency and whether it can be reduced with interventions. In a preregistered experiment with 1,203 participants, we compared two different types of interventions: An intervention with instruction that directly calls for reflection via text (instruction intervention); An intervention with a ranking-biased order that induces users to click on what they easily see and vice versa (ranking-biased intervention). The results showed no significant effect of the instruction intervention. However, the ranking-biased intervention showed preventive outcomes regarding the selective avoidance behaviors of users and promoted their clicking on links to uncongenial facts. The ranking-biased intervention was effective for users with high reflexiveness as well as for users with low reflexiveness. We discuss implications for designing fact-checking websites based on user understanding.

Fight Bias with Bias: Two Interventions for Mitigating the Selective Avoidance of Clicking Uncongenial Facts

Computing the semantic similarity between two texts is crucial in various NLP tasks. For more than a decade, a framework, known as Semantic Textual Similarity (STS) has been used to test computational models of semantic similarity (Agirre et al., 2012). The STS evaluation framework assumes that a model that performs well for the general STS task should also perform well for specific application-oriented tasks. However, does this assumption indeed hold? This study empirically demonstrates that the answer is not always positive. We found a considerable gap between model performance in STS and each specific task. We identified three factors that contributed to the gap, namely, (i) sentence length distribution, (ii) vocabulary coverage, and (iii) granularity of gold-standard similarity scores. We believe that these findings will be considered in future research on semantic similarity.

Chat Translation Error Detection for Assisting Cross-lingual Communications

Explainability is a crucial component of natural language processing (NLP) systems. Explanation is communication, and research on explainability is expected to address notions such as communicative goals and common grounding. Automated Writing Evaluation (AWE) is an ideal field to contribute to such research. AWE refers to NLP tasks designed to support human learners by evaluating the quality of texts produced in educational contexts, such as written answers to questions, argumentative essays, etc., and providing constructive feedback. Since explanations are vital in educational assessment, pedagogical literature is rich with insights that can facilitate research on explainability in AWE systems. This talk will overview recent research trends, explore what style of explanations are pedagogically suitable and technologically feasible at each layer of language production, ranging from content planning to surface realization, and identify open research issues to encourage researchers to enter this emerging field.

Kentaro Inui

3

27

SHORT BIO

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES