Netherlands

Humans are remarkably adept at inferring the causes of events in our environment; doing so often requires incorporating information from multiple sensory modalities. For instance, if a car slows down in front of us, inferences about why they did so are rapidly revised if we also hear sirens in the distance. Here, we investigate the ability to reconstruct others&#39; actions and events from the past by integrating multimodal information. Participants were asked to infer which of two agents performed an action in a household setting given either visual evidence, auditory evidence, or both. We evaluate our task on humans, a large language model (GPT-4), and a large multimodal model (GPT-4V). We find that humans are relatively accurate overall and perform best when given multimodal evidence, seeming to put more emphasis on visual evidence than on auditory evidence. GPT-4&#39;s overall accuracy closely matches that of humans in all modalities, but is only weakly correlated with human accuracy across trials, suggesting different reasoning mechanisms. Meanwhile, GPT-4V has lower accuracy and exhibits no evidence of incorporating multimodal information. People&#39;s ability to reconstruct the behavior of others relies on successfully integrating evidence across different senses. Such multimodal reasoning presents an intriguing challenge for multimodal AI systems.

**Authors:**

Sarah A Wu: Stanford University; Erik Brockbank: Stanford University; Hannah Cha: Stanford University; Jan-Philipp Fränken: University of Edinburgh; Emily Jin: Stanford University; Zhuoyi Huang: Stanford; Weiyu Liu: Stanford University; Ruohan Zhang: Stanford University; Jiajun Wu: Stanford University; Tobias Gerstenberg: Stanford University

CogSci 2024

Whodunnit? Inferring what happened from multimodal evidence

The 46th Annual Meeting of the Cognitive Science Society was an in-person meeting held in Rotterdam, The Netherlands at the Postillion Hotel & Conference Centre.

**ON-DEMAND PROGRAM ACCESS AFTER THE CONFERENCE** 
Recordings of the invited program are now available. You can access them by clicking on the 'Schedule' icon on the left. Select view by week and navigate to the day and time slot of the recording you wish to view. Click on the time slot to access the recording.

Recordings available (click on each recording to view):

**Thursday, July 25** 
0900: [Keynote Speaker Morgan Barense on Enhancing real-world event memory](https://underline.io/events/465/sessions/17997/lecture/99084-dynamics-between-minds-and-the-environment) 
1000: [Gleitman Award Winner Isabelle Dautriche on Language Foundations: Insights from acquisition, communication, cognition, and more](https://underline.io/events/465/sessions/18000/lecture/99085-gleitman-talk) 
1415: [Invited Symposium: Dynamics between minds and the environment](https://underline.io/events/465/sessions?eventSessionId=18016&searchGroup=lecture) 
1700: [Rumelhart Prize Presentation Speaker Alison Gopnik on Exploit, explore, empower: Three ages and three intelligences](https://underline.io/events/465/sessions/18028/lecture/99161-rp1-rumelhart-prize-presentation) 

**Friday, July 26** 
0900: [C.L. de Carvalho-Heineken Prize Keynote Speaker Kia Nobre on Focusing in memory](https://underline.io/events/465/sessions/18035/lecture/99162-hp1-c-l-de-carvalho-heineken-prize-keynote-address) 
1030: [Rumelhart Symposium: Childhood as exploration](https://underline.io/events/465/sessions/18038/lecture/100346-childhood-as-exploration) 
1415: [Elman Prize Symposium](https://underline.io/events/465/sessions?eventSessionId=18053&searchGroup=lecture) 
1600: [Invited Symposium: Dynamics between minds](https://underline.io/events/465/sessions?eventSessionId=18065&searchGroup=lecture) 
1745: [Keynote Speaker Andrea E. Martin on Neural dynamics encode the structure and statistics of language](https://underline.io/events/465/sessions/18077/lecture/99271-neural-dynamics-encode-the-structure-and-statistics-of-language) 

**Saturday, July 27** 
0900: [Keynote Speaker Gregor Schöner on How higher cognition emerges from the dynamics of strongly interacting neural populations](https://underline.io/events/465/sessions/18083/lecture/99273-dynamics-within-the-mind) 
1000: [Glushko Talks](https://underline.io/events/465/sessions?eventSessionId=18086&searchGroup=lecture) 
1500: [Invited Symposium: Dynamics within minds](https://underline.io/events/465/sessions?eventSessionId=18100&searchGroup=lecture) 


#### **CogSci 2024 Program Book**
<div style="position:relative;padding-top:0;width:900px;height:500px;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/cble/" seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


**The Program Book can be downloaded [here](https://drive.google.com/file/d/1yRVp1cBqRnAbdYi4GXWV_pZeyWNsRquv/view?usp=sharing).**

#### **Letter from the President**

Welcome to the 2024 Meeting of the Cognitive Science Society in Rotterdam! CogSci 2024 brings together a large community of cognitive scientists who have traveled here from around the world, as well as a smaller group of remote presenters. I would like to extend an especially warm welcome to our first-time CogSci meeting attendees, and hope that 
their experience inspires them to join our future meetings for many years to come.

I want to thank this year’s conference co-chairs, Larissa K Samuelson, Stefan Frank, Mariya Toneva, Allyson Mackey, and Eliot Hazeltine, for putting together an amazing series of invited talks and symposia around the theme of Dynamics of Cognition. The co-chairs also deserve credit for handling a record-breaking number of submissions and producing a truly exciting conference program 

Beyond the invited talks and symposia dedicated to this year’s conference theme, I would like to highlight the presentations and events honoring the 
Rumelhart, Elman, Gleitman and C .L . de Carvalho Heineken prize winners, including the Rumelhart reception on Thursday. As you may know, we are recording all of the invited talks and symposia, and all prize-winners’ talks, and will be making this content available after the conference. 

The Cognitive Science Society relies on the volunteer efforts of our Governing Board members who work on matters related to our membership; conference policies; diversity and inclusion, international, and outreach initiatives; prizes and much more. We welcome interest from Society members who would like to become more involved in what we do. The 
Society is deeply grateful to Rick Dale and Andrea Bender, who serve as editors of the two Society journals, Cognitive Science and Topics in Cognitive Science respectively. We especially acknowledge our editors’ generosity in donating their stipends each year to D&I and outreach efforts. 

This year’s Annual Meeting depends critically on the support of Marischal De Armond and his team at Podium Conference Specialists: Sharon Zwack, Cendrine De Vis, Sarah-Kate Burke, and Rachel Waller worked tirelessly to secure the venue and handled a multitude of logistical and organizational issues, from arranging coffee breaks and receptions to making sure that a huge number of posters fits comfortably into the available space. Finally, and importantly, I would like to acknowledge the Cognitive Science Society Executive Officer, Erica Wojcik, for managing the complex and ever expanding sphere of our Society’s activities.

I hope you enjoy this year’s meeting and the many different opportunities to engage with the vibrant community of cognitive scientists gathered here. I also hope you get a chance to explore the great city of Rotterdam with its many attractions, museums, restaurants and scenic views. If you’d like to talk more about our Society, or simply want to say hello, please stop by!

![](https://assets.underline.io/markdown_image/1/image/3c66d0e21fff30e204df1138ffd15173.jpeg)

**Anna Papafragou** 
President, Cognitive Science 
Society 2023-2024

**CODE OF CONDUCT** 
By attending the CogSci 2024 Conference, you are required to adhere to the society’s [Code of Conduct](https://drive.google.com/file/d/16-6KkptF0Gn3ZYGJDlpqpwTdImw45Ng0/view?usp=drive_link).

**ABOUT THE COGNITIVE SCIENCE SOCIETY** 
The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education.

The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

To access the site please register [**here**](https://cognitivesciencesociety.org/registration/). 

Please register!

The 46th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Humans are remarkably adept at inferring the causes of events in our environment; doing so often requires incorporating information from multiple sensory modalities. For instance, if a car slows down in front of us, inferences about why they did so are rapidly revised if we also hear sirens in the distance. Here, we investigate the ability to reconstruct others' actions and events from the past by integrating multimodal information. Participants were asked to infer which of two agents performed an action in a household setting given either visual evidence, auditory evidence, or both. We evaluate our task on humans, a large language model (GPT-4), and a large multimodal model (GPT-4V). We find that humans are relatively accurate overall and perform best when given multimodal evidence, seeming to put more emphasis on visual evidence than on auditory evidence. GPT-4's overall accuracy closely matches that of humans in all modalities, but is only weakly correlated with human accuracy across trials, suggesting different reasoning mechanisms. Meanwhile, GPT-4V has lower accuracy and exhibits no evidence of incorporating multimodal information. People's ability to reconstruct the behavior of others relies on successfully integrating evidence across different senses. Such multimodal reasoning presents an intriguing challenge for multimodal AI systems.

**Authors:**

Sarah A Wu: Stanford University; Erik Brockbank: Stanford University; Hannah Cha: Stanford University; Jan-Philipp Fränken: University of Edinburgh; Emily Jin: Stanford University; Zhuoyi Huang: Stanford; Weiyu Liu: Stanford University; Ruohan Zhang: Stanford University; Jiajun Wu: Stanford University; Tobias Gerstenberg: Stanford University

poster

Mandarin speakers have different space-time mappings than English speakers, but how Mandarin-speaking children spatialize time is unknown. We explored the development of 3D time-space representations in Chinese children aged 4 to 6. 145 Mandarin-speaking children, divided into three conditions (Exp1: horizontal, vertical, and Exp2: sagittal axes), undertook a MTL task for ten picture stories. We analysed their choices in 3-step temporal events, intending to test their sequential and directional preference of time (e.g., order vs. disorder; left-to-right vs. right-to-left). The results showed that Chinese children acquired sequential temporal representations on the horizontal and vertical axes at age 4, similar to English-speaking children. However, their directional preferences appeared earlier than English children (Exp1). Furthermore, the sagittal axis had different patterns: sequentiality emerged only at age 5, with no significant directional preference observed in the 3-6 age group. These findings emphasize that language and culture impact children's conceptualization of time.

**Authors:**

Jiayu Jiang: Syracuse University; Yan Gu: UCL

Understanding Time in Children's Mind: Development of Mental Timelines on Three-dimensional Axes

Essentialist beliefs about categories (e.g., intuitions that categories like “girl” or “tiger” reflect real natural structure in the world) emerge early in development across diverse cultural contexts, but the processes by which they develop have rarely been examined. We tested if the basic conceptual and explanatory biases that children rely on to build intuitive theories of the world contribute to the emergence of essentialism across early childhood. Consistent with this possibility, children who deferred to experts regarding category labels, endorsed single and intrinsic causes for object functions, and generated over-hypotheses about structure based on limited evidence developed more essentialist beliefs across childhood (with some variation across domains of thought). Together, these data reveal that the development of essentialist beliefs is shaped by basic conceptual biases that underlie how children construct intuitive theories about the world.

**Authors:**

Michelle Wang: New York University; Yian Xu: Kennesaw State University; Kelsey Moty: New York University; Marjorie Rhodes: New York University

The Cognitive Precursors of Early Developing Essentialist Beliefs

Adults expect others’ choices will be biased by investments of effort, time, or money. However, children do not similarly consider past investments when anticipating others’ actions. We examined whether prompting children about effort and emotion impacts their predictions about sunk costs. Children aged 5 to 7 years (N = 180) saw scenarios where a character collected two identical objects, one easy to obtain and the other difficult. Before children were asked which of the two objects the character will keep (sunk cost prediction), they were either asked an effort, sadness, or a control prompt. Children in the effort and sadness prompts selected the high-cost objects, suggesting they expected the character to be biased by sunk costs. However, similar to previous findings, children in the control prompt condition selected objects at chance-level. These findings suggest that if prompted, young children can anticipate others will be biased by sunk costs.

**Authors:**

Claudia G. Sehl: University of Waterloo; Stephanie Denison: University of Waterloo ; Ori Friedman: University of Waterloo

Do children predict the sunk cost bias if prompted to consider effort and emotion?

People often experience regret when we consider counterfactuals to our past actions, which can help us improve our future behaviours. However, existing developmental measures of regret typically involve no means of foreseeing the eventual outcome, which means that any reported experiences of regret may not aid children in making better choices in similar future situations. We investigated if 4- to 9-year-olds (N = 144) experienced stronger regret towards a choice where they could have foreseen the eventual outcome. Children selected one box each from two pairs of boxes, with both selected boxes leading to sub-optimal outcomes. Critically, one pair of boxes had windows on the bottom, such that children could have apparently foreseen the sub-optimal outcome of their choice if only they had first looked underneath the boxes. Not until 8 years of age did many children feel worse about the box selection with the foreseeable outcome.

**Authors:**

Alicia K Jones: The University of Queensland; Shalini Gautam: Boston College; Jonathan Redshaw: University of Queensland

"I should have known!" How foreseeability influences children's experiences of regret

Long COVID is defined as the persistence of COVID-19 symptoms for more than 12 weeks following infection (NICE, 2022). It is estimated to affect nearly 2 million people in the UK (ONS, 2023). Long COVID patients experience symptoms affecting multiple organ systems (Davis et al., 2023; Raveendran et al., 2021) including the CNS And Cognitive symptoms (e.g. Davis et al, 2021; Guo et al., 2022a and deficits (e.g. Guo et al., 2022b; Hampshire et al., 2021) have been demonstrated in adult sufferers. Despite the condition occurring in 13% of children who contract COVID-19 there is little research on the cognitive impacts of Long COVID in paediatric samples. This study explores memory (item- and associative) and language (semantic and syntactic) across 80 6-12 year olds with and without history of covid infection, relating these to parent-reported cognitive symptoms including brain fog and short-term memory problems.

**Authors:**

Lucy G Cheke: University of Cambridge; Sabine Yeung: University of Cambridge; Seraphina Zhang: University of Cambridge; Emma JL Weisblatt MRCPsych PhD: University of Cambridge; Mirjana Bozic: University of Cambridge; Zoe Hemsley: University of Cambridge; Panyuan Guo: University of Cambridge

The Impact of COVID infection on Cognition in 6-12 Year Old Children

Children are keen explorers of the world: They systematically explore surprising findings and test hypotheses during play. However, less is known about whether toddlers are similarly driven to learn about the self. Here, we ask whether toddlers are intrinsically motivated to explore their own competence. In ongoing work, 2-year-olds (N = 12) play Montessori practical life games along with their parents; toys were verified to be equally appealing and challenging to toddlers in an independent norming experiment (N = 14). Within each pair, parents guide toddler’s hands while playing with one toy, which provides ambiguous information about the toddler’s competence, and take turns playing with the other toy independently, which provides unambiguous information. Toddlers are then offered both toys to freely explore independently. Preliminary results show that toddlers explored the ambiguous toy first in 71% of the 31 trials, suggesting that toddlers seek out opportunities to learn about their competence.

**Authors:**

Bella Fascendini: Princeton University; Bonan Zhao: Princeton University; Natalia A Vélez: Princeton University

Are toddlers intrinsically motivated to explore their own competence?

Previous work has found that when adults are asked for "the first thing that comes to mind", they will provide something that falls between the descriptive average and the prescriptive ideal. In two experiments, we tested whether children would also be influenced by prescriptive information in their first-to-mind judgments, but also when they were asked to predict a randomly sampled item. In Experiment 1, providing information about whether being longer or shorter made a fictional tool better or worse led children to provide judgments that were biased toward the prescriptively 'best' tool, regardless of what they were asked for, while adults ignored prescriptive information when asked for a random sample. Experiment 2 replicated this result but further showed that the effect was specifically driven by information about which objects were prescriptively good, and did not also arise when the only salient information was about which objects were prescriptively bad.

**Authors:**

Jonathan F. Kominsky: Central European University; Joshua Knobe: Yale University; Elizabeth Bonawitz: Harvard Graduate School of Education

Children use positive prescriptive information when asked to predict random samples

Cognitive development has been hypothesised to be stagelike between the ages of 5-8 years (e.g., Piaget). Yet, cognition varies from moment to moment, in every task, for every child. Studies have demonstrated that cognitive variability is non-trivial, non-random, and meaningful, but attempts for systematic and large-scale longitudinal measurements of cognitive variability have scarcely been undertaken. 
This project’s goal is to create a more detailed empirical record and dynamical account of intra-individual variability in cognitive development of children. We aim to do this with a 3-year longitudinal and multimodal data collection starting at 5 years of age. Half-yearly measurements will be complemented with periods of daily measurements. Our ultimate aim is to build a variability corpus in which we can study variability patterns and developmental transitions, and to connect our findings to “Piaget’s new theory”. Our poster will present our methodology and results from a pilot study.

**Authors:**

Marije B. ten Den: University of Groningen; Lisette De Jonge-Hoekstra: University of Groningen; Marijn van Dijk: University of Groningen; Jeremy Trevelyan Burman PhD: University of Groningen; Ralf F.A. Cox: University of Groningen

Not stages, but variability ranges? Cognitive variability bridging complexity science and 'Piaget's new theory'

Recent evidence suggests that infants interpret giving as indicative of a relationship based on long-term balance. Such a relational model should induce the tracking of the participating agents as individuals irrespective of the temporary role they occupy, as these are expected to alternate over time in reciprocal fashion. We tested this hypothesis in a label-mapping paradigm by examining whether 14-month-old infants interpreted a trained label as referring to the individual pointed at or the action role it occupied within a giving event. Across four eye-tracking experiments, we found systematic evidence for the first interpretation. Tellingly, no consistent mapping was produced when infants were familiarized with object-displacing actions not resulting in a social transfer (i.e., disposing). Such an early propensity to map labels onto identity-preserving information over transitory action roles supports the hypothesis that infants conceptualize the participants of a giving event as individuals reciprocating acts of transfer over time.

**Authors:**

Denis Tatone: Central European University; Gergely Csibra: Central European University; Jun Yin: Ningbo University

The observation of giving induces infants to track individuals

Children learn words that name objects (“ball”) and those that name abstract concepts (“story”). One view of learning is that different inputs matter for different words (Snedeker & Gleitman, 2004). That is, many argue that although the observational contexts in which words occur are sufficient for learning object names, they are not for learning abstract “hard words” (Gleitman et al., 2005). This study revisited the contributions of observational contexts to learning one type of hard word: nouns denoting non-basic level object categories (“hard nouns” like “friend”; Kako, 2005). In a new artificial learning paradigm, we reveal that although observational contexts were insufficient for full hard noun learning, they afforded learners partial knowledge that allowed them to succeed in some learning tasks. These data highlight how observational contexts may lay the foundation for learning hard nouns, and underscore how definitions of learning impact our understanding of how the input shapes it.

**Authors:**

Kosta Boskovic: University of Connecticut; Sumarga H. Suanda: University of Connecticut

Premium content

Downloads

Next from CogSci 2024

Understanding Time in Children's Mind: Development of Mental Timelines on Three-dimensional Axes

Similar lecture

Procedural Dilemma Generation for Moral Reasoning in Humans and Language Models

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES