United States

While saliency-based explainable AI (XAI) methods have been well developed for image classification models, they fall short in comparison with human explanations. Here we examined human explanation strategies for image classification and their relationship with explanation quality to inform better XAI designs. We found that individuals differed in attention strategies during explanation: Participants adopting more explorative strategies used more visual information in their explanations, whereas those adopting more focused strategies included more conceptual information. In addition, visual explanations were rated higher for effectiveness in teaching learners without prior category knowledge, whereas conceptual explanations were more diagnostic for observers with prior knowledge to infer the class label. Thus, individuals differ in the use of visual and conceptual information to explain image classification, which facilitate different aspects of explanation quality and suit learners with different experiences. These findings have important implications for adaptive use of visual and conceptual information in XAI development.

CogSci 2023

Individual differences in explanation strategies for image classification and implications for explainable AI

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Since Darwin (1871), researchers have proposed that
musicality evolved in a reproductive context in which males
produce music to signal their mate quality. The extent to
which evidence supports this contention, however, remains
unclear. Related traits in many non-human animals are
sexually differentiated, and while some sex differences in
human auditory perception have been documented, the
pattern of results is murky. Here, we study melodic
discrimination, mistuning perception, and beat alignment
perception in 360,009 men and 194,291 women from 208
countries. We find that, in contrast to other non-music human
traits, and in contrast to non-human traits, there was no
overall advantage for either gender, and the observed gender
differences were minuscule (Cohen’s d: 0.009 - 0.11) and of
inconsistent direction. These results do not provide
compelling support for human music perception being a
sexually dimorphic trait, and therefore it is unlikely to have
been shaped by sexual selection.

Human music perception ability is not a sexually dimorphic trait

For cognition to evolve, environmental cues must be sufficiently reliable to allow an animal to regularly make correct decisions. For example, the sound of rustling must covary with prey for attempting to detect prey to be possible. However, the reliability of information is not solely a property of the environment, but also results from the sophistication of an animal’s sensory systems. Here, we extend signal detection theory to examine how selection acts on sensory systems, alongside an animal’s policy for using information. Our model finds cognition can still be favored if sensory systems are poor, by compensating with greater bias to avoid costly errors. Yet, investing in reliable information and bias are not always alternatives, and can coevolve in order to be doubly protected from important errors. Sophisticated sensory systems evolve when the environment is complex, metabolic costs are low, and both false alarms and missed detections are moderately costly.

The joint evolution of sensory systems and decision policy allows cognition

With the advancement of technology, audiovisual (AV) products (i.e., films, TV shows, podcasts...) are now disseminated online and thus easily accessible by internet users. This paper takes advantage of eye-tracking technology to investigate how audiences receive and process different translated swearwords in films, which is still underrepresented and an uncharted area in audiovisual translation (AVT) studies. An eye-tracking experiment was carried out to expose 150 participants to four film clips and allocating them into one of the five subtitling groups: four experimental groups with four different translation strategies of swearwords ranging from low to high profanity, and one control group with only original English captions. Questionnaires were supplemented to gauge their immersion, satisfaction, enjoyment, comprehension and sense of offensiveness. This doctoral project is intended to add more human-related element in and expand the knowledge boundary of AVT field triangulated with cognitive science, audience reception in film study, psycholinguistics in general.

An eye-tracking study on the audience reception and processing of English-Chinese swearwords in films

Previous research found that loaded language (i.e., words, phrases, and overall rhetorical strategies that have strong emotional implications and intent to sway others) is used with significantly greater frequency by users of an online anti-vaccination forum compared to users of a vaccination-neutral forum. Our present research expands on these findings by examining the presence of loaded language on forums dedicated to discussing a broader range of conspiratorial topics. We compare loaded language usage on the subreddit r/conspiracy with usage on r/science and r/wallstreetbets. Results corroborate the previous literature. We additionally examined the presence of loaded language in a dataset from the alt-tech social networking platform Parler on January 6, 2021, when the U.S. Capitol was attacked because of conspiracy theories regarding the 2020 Presidential Election. Results show that loaded language, whose usage is linked to the cognitive motivations underlying belief in conspiracy theories, is a linguistic marker of conspiracy theorizing.

Loaded Language and Conspiracy Theories on Reddit and Parler

The legal system is concerned with interpreting and applying existing laws to real world circumstances. In the U.S., the final arbiters of these laws are the nine justices of the U.S. Supreme Court. The cases presented before the court generally involve disputes between two parties, the petitioner and the respondent. However, in some cases third parties provide additional information to the court as amici curiae (‘friends of the court’). The present study uses corpus statistics and moral foundation theory to explore and compare the types of arguments brought up by the different parties during oral arguments over the past 6 decades, with a focus on amicus curiae arguments and how they affect the results of the case. Overall, the results show that both amici and respondents rely on moral arguments more frequently than petitioners do.

Supreme Court Oral Arguments by Amicus Curiae: A comparative analysis

Drawing on qualitative discourse analysis, this study conducted semi-structured interviews with seventeen sexual minorities, to investigate how the categorization mechanism generates institutional, interpersonal, and internalized stigmas and stressors against Chinese sexual minority youth, and how the categorization mechanism is represented in their contextualized lived experience. Heterosexual matrix and homonormativity hidden in Chinese traditions and ideologies are identified under the framework of queer linguistics. This study reveals that Chinese sexual minority youth are otherized as problematic out-groups on the textual level. They are negatively constructed as hypersexualized, pathologized, and dehumanized. On the micro-linguistic level, deixis, evaluations, and emotive words are used to classify in-groups and out-groups, show intimate and remote social distances, and express positive and negative attitudes. Results uncover that the categorization mechanism produces asymmetric relationships and power dynamics between heterosexual and non-heterosexual people, and between homonormative and non-homonormative sexual minorities.

The categorization mechanism in queer linguistics: A study on discourses of Chinese LGB youth

Across cultures, people use natural and supernatural explanations to explain adverse life events, such as illness or death. Yet little is known about the psychological implications of this type of causal reasoning. Here, we ask, does explanatory coexistence help or hinder coping with significant misfortune? We examined this question through structured interviews with a diverse sample of 147 Los Angelinos who had suffered from severe illness or bereavement during the COVID-19 pandemic. As predicted, mean scores on the Posttraumatic Growth Inventory were significantly higher for participants who employed coexisting explanations for their misfortune than those who employed singular explanations (natural, supernatural) alone. These findings provide evidence of an association between coexistence reasoning about misfortune and positive post-event processing.

Coexistence Reasoning about Misfortune During COVID-19 is Associated with Positive Psychological Well-Being

How responsible someone is for an outcome depends on both the causal role of their actions, and what those actions reveal about their moral character. Prior work has successfully modeled people's causal attributions and mental state inferences using planning algorithms assumed to approximate people's intuitive theory of mind about others' behavior. In this paper, we develop a unified computational framework for responsibility judgments in which the same generative planner can model both of these processes. We test our framework on a variety of animated social scenarios in two experiments. Experiment 1 features simple cases of helping and hindering. Experiment 2 features more complex interactions that require recursive reasoning, including cases where one agent affects another by merely signaling their intentions without physically acting on the world. Our model accurately captures participants' counterfactual simulations and intention inferences. Together, these two factors explain responsibility judgments.

A computational framework for responsibility judgments from counterfactual simulations and intention inferences

Causal pluralism proposes that humans can reason about causes and effects in terms of both dependency and process relations, depending on the scenario. Empirical support for this view is provided by responses to double prevention scenarios in which an actor attempts to bring about an outcome, a preventer attempts to prevent the outcome, and a double preventer intervenes to stop the preventer’s prevention attempt. In contrast to the predictions of the causal pluralism account, two pre-registered experiments (Ns = 400 and 450) indicate (a) that intentional actions are judged to be significantly more causative of an outcome than unintentional actions for both the actor and the double preventer and (b) that reasoners interpret the double preventer’s link to the outcome in terms of a process relation. These results underscore the need to revisit fundamental questions regarding how reasoners form and reason over representations of causal scenarios featuring intentional actions.

Revisiting Causal Pluralism: Intention, Process, and Dependency in Cases of Double Prevention

A causal illusion occurs when people perceive a causal relationship between two events that are not contingent on each other. How this illusion varies when people reason diagnostically (i.e., in an effect-to-cause direction) is not well known. Thus, this experiment explored how diagnostic reasoning affects causal illusions. Participants learnt about a illusory cause-effect relationship in which the probability of the cause and the probability of the effect were orthogonally manipulated to be either high or low. Participants learnt either predictively (i.e., cause-to-effect) or diagnostically, and at test had to make two causal judgements that encouraged either predictive (cause-to-effect) or diagnostic (effect-to-cause) reasoning. Diagnostic reasoning at test increased the strength of the cause density bias and decreased the strength of the effect density bias. It also decreased causal ratings, but only after predictive learning. These findings highlight the promise of diagnostic reasoning as a tool to debias causal decision making.

Premium content

Individual differences in explanation strategies for image classification and implications for explainable AI

Next from CogSci 2023

Human music perception ability is not a sexually dimorphic trait

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES