United States

The inverse base-rate effect is a robust irrational bias that arises when people face ambiguity. The most prominent theories of this irrational bias depend on prediction error.
In this study, we gradually removed elements of a predictive learning design to test the extent to which error-driven processes underly this bias. In our first experiment, we removed explicit feedback by implementing the inverse base-rate effect in an observational learning procedure. In our second study, we further removed any causal relationship between stimulus features and category labels by moving towards an unsupervised learning procedure. This removed any information participants could use to identify category labels. In both experiments, the inverse base-rate effect persisted and remained robust. This outcome suggests that this irrational bias is independent of supervised learning procedures. We propose that any theories and models of the inverse base-rate effect must manage information encoding and connection updates without explicit prediction error. We end by proposing two clear paths for future investigations.


CogSci 2023

Errorless irrationality: removing error-driven components from the inverse base-rate effect paradigm

technical paper

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Recent studies suggest that errors facilitate learning in certain conditions. Despite this, reinforcement paradigms dominate learning methods, subscribing to the narrative that errorless learning is the foundation of an ideal learning environment. If we continue to view learning from this restrictive perspective, we may fail to capture and apply the benefits of errors. In this paper, we investigate two potential mechanisms of post-error learning. Participants (N = 61) learned word pairs in either a study or error trial before taking a final test. Supporting past error learning literature, errors before a study opportunity led to better performance on a final test. Differences in reaction times between conditions support the theory that errors increase learning by acting as a mediator, or secondary cue, to the correct answer on subsequent tests.


Faulty Memories, Favored Outcomes: How Errors Impact Learning Processes

Efforts to correct misconceptions have produces multiple types of complex interventions. However, past research has consistently shown that simple, factual statements may be equally effective in activating the incorrect information and prompting an integration of the correct concept. The current study tests the effectiveness of simple statements on 76 different misconceptions. Findings indicate that for the majority of misconceptions, simple statements are an effective intervention to reduce misconceptions. Limitations and future directions are discussed.


Just Tell the Truth: Correcting Misconceptions with Simple, Factual Statements

Psycholinguistic analyses provide a means of evaluating large language model (LLM) output and making systematic comparisons to human-generated text. These methods can be used to characterize the psycholinguistic properties of LLM output and illustrate areas where LLMs fall short in comparison to human-generated text. In this work, we apply psycholinguistic methods to evaluate the psycholinguistic properties within sentences from student-generated long-form analogies about biochemical concepts. We compare analogies generated by human subjects enrolled in introductory biochemistry courses to analogies generated by chatGPT. We use 78 features extracted from Coh-metrix that analyze text cohesion, language, and readability (Graesser, McNamara, Louwerse, & Cai, 2004). We perform an unsupervised cluster analysis. Results illustrate a high level of performance for clustering student-generated and chatGPT-generated analogies. To evaluate which features contribute most to model performance, we use a hierarchical cluster analysis. Results from the hierarchical cluster analysis illustrate several linguistic differences between the two groups.


Long-form analogies generated by chatGPT lack human-like psycholinguistic properties

Regular polysemes are sets of ambiguous words that all
share the same relationship between their meanings, such
as CHICKEN and LOBSTER both referring to an an-
imal or its meat. To probe how a context embedding
model, here exemplified by BERT, represents regular pol-
ysemy, we analyzed whether its embeddings support an-
swering sense analogy questions similar to “is the map-
ping between CHICKEN (as an animal) to CHICKEN (as
a meat) the same as that which maps between LOBSTER
(as an animal) and LOBSTER (as a meat)?” We found
that (1) the model was sensitive to the shared structure
within a regularity type; (2) the shared structure varies
across regularity types, potentially reflective of a “reg-
ularity continuum;” (3) some high-order latent structure
may be shared across regularity types, suggestive of a
similar latent structure across types; and (4) the is equivo-
cal evidence that the aforementioned effects are explained
by meaning overlap.


Probing the Representational Structure of Regular Polysemy in a Contextual Word Embedding Model via Sense Analogy Questions

There is a growing interest in emotions in textual data. The studies on labeling dataset or detecting emotions from text are based on some psychological theories. However, there has been little research on the appropriateness of using these theories. In this study, we reviewed the commonly used basic emotion theories. We analyze 3 major theories: Ekman's Basic Emotions, Plutchik's Wheel of Emotions, and GoEmotions. We used an emotionally fine-tuned language model to analyze the emotional relationship between words that each theory designates as basic emotions and evaluated the adequacy of the emotional structure each theory presents. As a result of the fine-tuning, clear patterns of similarity emerged based on the emotional meaning of each word, and through this we analyzed the structure of basic emotions. Ekman's and GoEmotions were almost in line with our results, while Plutchik's had some differences. We thoroughly discussed these matches and mismatches.


A Structure of basic emotions: A review of basic emotion theories using an emotionally fine-tuned language model

Online experiments with participants from around the world speaking different languages can inform research about culture, cognition, and language. To achieve this, it is necessary to objectively assess participants' language proficiency. However, available proficiency tests are small and only cover a few languages, making them unsuitable for cross-cultural online studies. Here, we present an automated pipeline to generate vocabulary tests based on Wikipedia called ``WikiVocab". Our pipeline samples rare nouns from a word-frequency distribution and creates pseudowords with the same low-level statistics. Six behavioral experiments (N=236) show that (a) our test can distinguish between native speakers of closely related languages, (b) participants' scores are consistent across test items, and (c) performance strongly correlates with existing tests (LexTale) and self-reports. We benchmark our results with GPT-3 and show that test accuracy and language distance are negatively correlated. Our test, available in eight languages, can easily be extended to other languages.


Around the world in 60 words: A generative vocabulary test for online research

Is “cow” more closely related to “grass” or to “chicken”? Speakers of different languages judge similarity in this context differently, but why? One possibility is that cultures co-varying with these languages induce differences in conceptualizations of similarity. Specifically, East Asian cultures may promote reasoning about thematic similarity, by which cow and grass are more related, whereas Western cultures may bias judgments toward taxonomic relations, like cow and chicken. We measure similarity judgments across the US, China, and Vietnam and replicate US-China differences, but do not find that responding in Vietnam patterns with China. Instead, similarity judgments in Vietnam are intermediate between the US and China. We also show that word embedding models (fastText models for each language) are related to judgments within each country, suggesting a possible alternative interpretation of cross cultural differences. Perhaps notions of similarity are similar across contexts, but the statistics of the linguistic environment vary.


Re-examining cross-cultural similarity judgments using language statistics

People often prefer simpler explanations (that posit the presence of fewer causes), judging these more probable than more complex alternatives. However, simplicity preferences are only mathematically justified under certain conditions. We examine one case where, mathematically, complexity preferences are justified, and test whether this corresponds to a boundary condition on simplicity preferences. Specifically, we focus on cases where causes occur frequently (rather than rarely), and where explanations specify the absence of additional causes (e.g., “Cause A and not B or C”) rather than remaining agnostic about their presence or absence (e.g., “Cause A”). Study 1 showed that, in these cases, simplicity preferences were attenuated, but not reversed. Study 2 suggests that simplicity preferences partly stemmed from failures to explicitly represent absent causes. We suggest that biases towards oversimplification may arise due to over-application of a cognitively simple version of Ockham’s razor, that is insensitive to the probability of absent causes.


The Edge of Ockham’s Razor: Examining Boundary Conditions on Preferences for Simpler Explanations

What shape do people's mental models take? We hypothesize that people build causal models that are suited to the task at hand. These models abstract away information to represent what matters. To test this idea empirically, we presented participants with causal learning paradigms where some features were outcome-relevant and others weren't. In Experiment 1, participants had to learn what objects of different shape and color made a machine turn on. In Experiment 2, they had to predict whether blocks sliding down ramps would cross a finish line. In both experiments, participants made systematic errors in a surprise test that asked them to recall what they had seen earlier. The errors people made suggest that they had build mental models of the task that privileged causally relevant information. Our results contribute to recent efforts trying to characterize the important role that causal abstraction plays in human learning and inference.


Learning what matters: Causal abstraction in human inference

Deciding whether an event could happen, or should happen, is critical to learning and decision-making yet takes years to develop, as young children are prone to deny the possibility and permissibility of any event that violates their expectations. Here, we explore whether this form of reasoning—modal cognition—is aided by cognitive reflection, or the disposition to privilege analysis over intuition. Ninety-nine children between the ages of 4 and 12 judged the possibility and permissibility of several hypothetical events, and their judgments were compared to their score on a developmental version of the cognitive reflection test, the CRT-D. Children’s CRT-D scores predicted their ability to differentiate possible events from impossible ones and impermissible events from permissible ones, as well as their ability to differentiate possibility from permissibility in general. Such differentiations were predicted by children’s CRT-D scores independent of age and executive function. These findings suggest that mature modal cognition may require the ability to reflect on, and override, the intuition that unexpected events cannot happen.


Premium content

Errorless irrationality: removing error-driven components from the inverse base-rate effect paradigm

Downloads

Next from CogSci 2023

Faulty Memories, Favored Outcomes: How Errors Impact Learning Processes

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES