United States

Recent years have seen remarkable advances in the development and use of Artificial Intelligence (AI) in image classification, driving cars, and writing scientific articles. Although AI can outperform humans in many tasks, there remain domains where humans and AI working together can outperform either working alone. For humans and AI to work together effectively, the human must trust the AI bot to the right degree (calibrated). If the human does not trust the bot sufficiently, or conversely trusts the bot more than is warranted, the human-bot team will not perform as well as they could. We report three experiments examining trust in human-AI teaming. While existing studies typically collect binary responses (to trust, or not to trust), we present a novel paradigm that quantifies trust in a bot-recommendation in a continuous fashion. These data allow better precision, and in the future the development of more refined models of human-bot trust.

CogSci 2023

Trust in Human-bot Teaming: Applications of the Judge Advisor System

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

With AIs becoming prevalent in our daily lives, it is still controversial whether they are a reliable conversational assistant. We, therefore, developed a chatbot, a Friendly-Bot, where users can share interpersonal experiences (e.g., friendship and romantic relationships). We then manipulated chatbot appearances into two types: robot and human-looking. We found that chatbot appearance predicted users' trust. Participants preferred the robot-looking chatbot over the human-looking one. Participants also showed higher trust in robot-looking chatbots when conversing about positive interpersonal relationships. Participants showed a different pattern of trust depending on the appearance condition: positive experience led to higher trust in robot-looking condition, whereas the opposite was observed in human-looking condition. Our findings show how chatbot appearance influences rapport-building in AI-assisted interactions (e.g., counseling).

Friendly-Bot: The Impact of Chatbot Appearance and Relationship Style on User Trust

People can understand images that vary in visual abstraction---from detailed illustrations to schematic icons. 
To what degree are current vision algorithms sensitive to such variation when encoding their meaning? 
We first obtained >80K human-generated sketches produced under different time limits (4s, 8s, 16s, 32s; N=5,563 participants) and AI-generated sketches (Vinker et al., 2022) produced under different ink limits (8, 16, 32 strokes) of 2,048 real-world objects spanning 128 categories from the THINGS dataset (Hebart et al.,2019).
We then evaluated how 13 state-of-the-art vision algorithms varying in architecture and training represented the meaning of these sketches, both at the category and exemplar levels, and without additional training.
We found that these models were generally sensitive to variation in visual abstraction, although some models were substantially more sensitive than others.
Together, these findings suggest promising avenues for building better models of human visual abstraction.

Evaluating machine comprehension of sketch meaning at different levels of abstraction

Although object detection AI plays an important role in many critical systems, corresponding Explainable AI (XAI) methods remain very limited. Here we first developed FullGrad-CAM and FullGrad-CAM++ by extending traditional gradient-based methods to generate object-specific explanations with higher plausibility. Since human attention may reflect features more in-terpretable to humans, we explored the possibility to use it as guidance to learn how to combine the explanatory information in the detector model to best present as an XAI saliency map that is interpretable (plausible) to humans. Interestingly, we found that human attention maps had higher faithfulness for explaining the detector model than existing saliency-based XAI methods. By using trainable activation functions and smoothing kernels to maximize the XAI saliency map similarity to human attention maps, the generated map had higher faithfulness and plausibility than both existing XAI methods and human atten-tion maps. The learned functions were model-specific, well generalizable to other databases.

Human Attention-Guided Explainable AI for Object Detection

For the last decade, convolutional neural networks (CNNs) have vastly superseded their predecessors in nearly all vision tasks in artificial intelligence, including object recognition. However, despite abundant advancements, they continue to pale in comparison to biological vision. This chasm has prompted the development of biologically-inspired models that have attempted to mimic the human visual system, primarily at a neural level, which is evaluated using standard dataset benchmarks. However, more work is needed to understand how these models perceive the visual world. This article proposes a state-of-the-art procedure that generates a new metric, Psychophysical-Score, which is grounded in visual psychophysics and is capable of reliably estimating perceptual responses across numerous models --- representing a large range in complexity and biological inspiration. We perform the procedure on twelve models that vary in degree of biological inspiration and complexity, we compare the results against the aggregated results of 2,390 Amazon Mechanical Turk workers who together provided $\sim2.7$ million perceptual responses. Each model's Psychophysical-Score is compared against the state-of-the-art neural activity-based metric, Brain-Score. Our study indicates that models with a high correlation to human perceptual behavior also have a high correlation with the corresponding neural activity.

Psychophysical-Score: A Behavioral Measure for Assessing the Biological Plausibility of Visual Recognition Models

Relational ability—the ability to compare situations or ideas and discover common relations – is a key process in higher-order cognition that underlies transfer in learning and creative problem solving. For this reason, it has generated intense interest both among developmentalist and in cross-species comparative studies. The gold standard for evaluating relational ability is the Relational-Match-to-Sample (RMTS) task (Premack, 1983). Current work in cognitive development has produced inconsistent results as to when children are able to pass the RMTS, with Christie and Gentner (2014) finding earlier success than do Hochmann et al. (2017). In this research, we attempt to resolve this issue. We first describe two studies that bear out and extend the Christie and Gentner (2014) findings. We then discuss factors that might explain the discrepancy between the findings.

When do Children Pass the Relational-Match-To-Sample Task?

Abstract reasoning in early childhood is often described as following a "relational shift," over which children become increasingly sensitive to relations. However, recent work has challenged the generality of this account, showing that children in the US and China follow distinct trajectories in a relational match-to-sample task (Carstensen et al., 2019). This difference aligns with multiple cultural and linguistic factors implicated in relational reasoning, in which English speakers in the US and Mandarin speakers in China appear at opposite ends of a continuum spanning from a focus on objects (US) to relations (CN). We explore early relational reasoning in a context that represents a cultural middle ground with a key linguistic similarity (noun spurts) to the US: Korean-learning children in South Korea. In two experiments with 262 Korean children, we document relational reasoning in this novel cultural context, revealing similarities and differences to developmental trajectories in the US and China.

Relational abstraction in early childhood: Three cultures and three trajectories

Analogy is considered a core cognitive capacity that enables relational reasoning. It has been commonly believed that relational reasoning can be tested by the four-term analogy task of type “what is to c as b is to a?” or “a:b::c:?” in short. In this study, we challenge this common belief by investigating an alternative hypothesis that this classic task is solvable only by attributional reasoning on attributional similarities, rather than relational reasoning on the relation of the two relations. A word vector model was employed to demonstrate how these two types of reasoning were performed in the classic three-stem analogy task “a:b::c:?” and the two-stem analogy task “a:b::?:?”. The results showed that the two-stem analogy task was solvable only using relational reasoning, while the three-stem analogy task was solvable using both types of reasoning. It suggests that the two-stem analogy task is a better test for relational reasoning.

Two-stem Analogy Task is a Better Test for Relational Reasoning

Analogical reasoning is central to learning and thought with a key focus on similarity in which the same system of relations holds across different sets of elements. The current research investigated both first-order relation and second-order relation, represented in the combination pattern of symbols following the syntactic rules. The primary goal was to specify the neural substrates for abstract syntactic rule analogy.Using fMRI, the study identifies brain regions involved in this task, including left rostrolateral prefrontal cortex (rlPFC), left dorsolateral prefrontal cortex (dlPFC), and right precuneus. The results of this study reveal that although analogical reasoning of syntactic rules recruits similar brain regions as those in visuospatial and semantic analogy, their activation patterns are not the same. The role of rlPFC, the core area for analogical reasoning, needs to be reconsidered, and the complexity of precuneus is worth more attention.

Neural Correlates of Analogical Reasoning of Syntactic Rules

Detecting covariation in sequential events provides us with a powerful means of inferring the causal structure of our world. However, people often overestimate the causal relationship between unrelated events, a phenomenon referred to as illusory causation. This tendency is greatest when the putative effect occurs frequently; the widely replicated outcome density (OD) effect. Most laboratory research on illusory causation and the OD effect has focused on possible causes of a positive outcome, such as a drug that causes patient recovery. Despite its relevance, relatively few studies have examined illusory causation in cases where a cue is hypothesized to generate an unfavorable (negative) outcome, such as a drug that produces unwanted side effects. Here, we directly compared how people develop illusory beliefs about the generation of positive versus negative outcomes. We presented all participants with a drug treatment that was hypothesized to cause high readings of a fictitious cell count (but had no effect on cell count across a series of learning trials). We manipulated whether a high cell count occurred frequently or infrequently and whether a high cell count should be considered a beneficial medical outcome or an undesirable side effect. We found consistent evidence of an OD effect but no effect of the valence of the high cell count outcome. This suggests that illusory beliefs are not controlled by the desirability of the cause-effect relationship. We discuss implications for theories of applied causal reasoning.

Learning about Benefits and Side Effects of a Bogus Treatment are Similarly Influenced by the Frequency of the Outcome Occurring

Most research on causal judgment operates on an implicit premise: that the only relevant candidates for actual causation are individual variables, corresponding to the distinct nodes of a causal system. Instead, we claim that causal judgment can be about plural causes featuring more than one variable. We show that plural cause judgments are influenced by the prior probabilities of events just like other actual causation judgments. We further show that this dependence on probabilities cannot be explained away by supposing that subjects estimate the strength of a plural cause by combining their estimates for the individual variables that feature in it. Instead, plural causes appear to be apprehended holistically, as the object of an ad hoc computation, providing evidence that the notion of plural cause is a category of human judgment in its own right.

Premium content

Trust in Human-bot Teaming: Applications of the Judge Advisor System

Next from CogSci 2023

Friendly-Bot: The Impact of Chatbot Appearance and Relationship Style on User Trust

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES