Canada

It has been conjectured that verb learning is hard because verb meanings are not readily &quot;packaged&#39;&#39; from the physical world. To provide new empirical evidence on this account, we analyzed egocentric video collected from natural toy-play interaction and focused on the naming events when action verbs were uttered in parent speech. Using the Human Simulation Paradigm, we showed egocentric videos of those naming events to adult observers and asked them to guess the target verb in parent speech. We found that adult observers used many different verbs to describe the same visual event, and only one of them matched with the verb in parent speech. We analyzed mismatched verbs and identified several sources of mismatch, and found that all of the mismatched verbs are relevant to the target verb, but they capture different properties (temporal, semantic, etc) of the visual event. We also found that different naming events for the same verb also differ in terms of the degree of ambiguity. Taken together, the results in the present paper provided new evidence from the child’s view, showing that verb learning is hard not only because multiple possible meanings are embedded in each learning situation, but also because these candidate meanings expand across multiple dimensions of the physical world, overlap with each other, and relate to the target meaning in many different ways.

CogSci 2022

Grounding Action Verbs in Egocentric Visual Perception

Join us for CogSci2022! Our 44th Annual Conference will be centered around the theme of diversity, with an accompanying range of keynotes and invited speakers who themselves highlight the diversity of our field. Our three symposia are Cognition across Diverse Populations, Cognition within Diverse Societies, and Global Challenges for Cognitive Science. The conference is being offered in a fully hybrid environment for onsite and online attendees.
 
 [**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/07/Cogsci-2022-PAAG-Detailed-July-12final-1.pdf)
 [**The Detailed Program can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/08/DeArmond-CogSci2022-program-v113-final.pdf)

This conference requires a registration to attend. Please use the link below to secure your ticket. The website will be available to all registered attendees on July 25th. 

[Register to Attend](https://cognitivesciencesociety.org/registration/)

Registration Is Required

The 44th Annual Meeting of the Cognitive Science Society highlighted research on the theme Cognitive Diversity in addition to the full breadth and diversity of research topics offered by the society’s membership. 





It has been conjectured that verb learning is hard because verb meanings are not readily "packaged'' from the physical world. To provide new empirical evidence on this account, we analyzed egocentric video collected from natural toy-play interaction and focused on the naming events when action verbs were uttered in parent speech. Using the Human Simulation Paradigm, we showed egocentric videos of those naming events to adult observers and asked them to guess the target verb in parent speech. We found that adult observers used many different verbs to describe the same visual event, and only one of them matched with the verb in parent speech. We analyzed mismatched verbs and identified several sources of mismatch, and found that all of the mismatched verbs are relevant to the target verb, but they capture different properties (temporal, semantic, etc) of the visual event. We also found that different naming events for the same verb also differ in terms of the degree of ambiguity. Taken together, the results in the present paper provided new evidence from the child’s view, showing that verb learning is hard not only because multiple possible meanings are embedded in each learning situation, but also because these candidate meanings expand across multiple dimensions of the physical world, overlap with each other, and relate to the target meaning in many different ways.

poster

**Abstract:**

Even simple, ambiguous signals can have a rich interpretation when viewed in the context of an interaction in a shared environment. We create a model called Paternalistic Communication by combining an existing model of overloaded language -- Rational Speech Acts (RSA) -- with a full agent model of Theory of Mind (ToM). This integration allows signals to be processed in conjunction with common ground in a principled manner dependent on task-dependent action utilities. This modeling perspective treats communication as a way to coordinate diverging perspectives in a cooperative setting. Under Paternalistic Communication, a speaker decides what to say by predicting their partner’s reaction based on the information in common ground and then evaluates those reactions using their own mind which may contain additional information. We demonstrate the flexibility and performance of Paternalistic Communication in a case study with ambiguous signaling through a set of simulations.

**List of authors and affiliations:**

Stephanie Stacy: University of California - Los Angeles; Aishni Parab: University of California, Los Angeles; Max Kleiman-Weiner: Massachusetts Institute of Technology; Tao Gao: UCLA

Overloaded Communication as Paternalistic Helping

Literature has suggested that self-faces are processed differently at various stages of information processing. Although mechanisms like familiarity, implicit positive attitude, emotional arousal, dual-coding, and dopamine reward pathway have been theorized to explain this effect, it may share a fundamental basis in the attentional mechanism resulting in perceptual prioritization for self-face. In this study, we have assessed the attentional bias resulting from the self-face (over other familiar and unfamiliar faces), by using face pairs as cues before a dot-probe task. We looked at reaction time and its underlying latent variables as a function of face pairs used as cues. We find that both self-face and familiar face result in a faster reaction time for subsequent stimuli at cued locations. Though self-face shows this advantage for both short and long cue-time, a familiar face shows the advantage only for longer cue-time. We also found that drift rate bias is found for the location where self-face is presented. Familiar faces show a prior bias (z) as the reason for underlying advantage. We conclude that although, self-face, as well as familiar faces, might bias the processing of subsequent stimuli the underlying latent factor might differ.

Attentional Bias for Self-Face: Investigation using Dri Diffusion Modelling

People detect painful expressions more easily in members of their racial ingroup than outgroup. Here, we wanted to investigate this racial bias with a machine learning model trained to detect activations of different action units of painful facial expressions. We examined whether the model detected higher action unit activation for European than African faces when trained on datasets with mostly White faces. To control for confounding variables, pictures of faces were generated with the FaceGen Modeller. Results revealed that there exist differences in the visual detectability of some facial muscle activations due to skin color or other race-dependent facial features. Despite the bias towards European looking faces in the training data, some activations were more easily detectable in African faces. Thus, neither the perceptual detectability, nor the larger exposure to own-race faces seems to solely explain the racial bias in pain detection.

Exploring the Racial Bias in Pain Detection with a Computer Vision Model

Categorical perception involves our perceptual system creating sharp boundaries along an objectively continuous stimulus property, such as the discrete colors of the rainbow being perceived despite continuous change in wavelength. The same mechanism is thought to take place in facial emotion perception. But how are emotions at these boundaries perceived? We presented participants with morphed emotional faces made by blending different emotional expressions in equal proportions. Next, we asked participants to respond freely to these ambiguous face morphs and examined these responses via natural language processing methods. The results showed that participants used many more labels than those related to the categories which went into the morphs. These results can inform theories on categorical facial perception as well as the mental representation of facial expressions.

Categorizing Ambiguous Facial Expressions

Inferences of other’s emotion states are influenced by multiple sources including target cues such as facial movements and the situational context. Our understanding of how information from these cues is integrated is limited, however. We examined whether people integrate information from faces and situations to infer emotions as predicted by an existing model of affective cognition. We applied a Bayesian cue-integration model to a dataset that includes a variety of complex social situations that reflect the heterogeneity of emotion contexts in social lives. Results indicate that when viewing both faces and situations, situation information alone predicted people’s inference about emotions better than Bayesian cue-integration model. However, there was some variability in this pattern across emotion categories as the Bayesian cue-integration model best predicted inferences for emotion categories of amusement and happiness. These findings better our understanding of the interplay between facial and situational cues in informing emotion inferences.

Modeling Cue-integration in Emotion Inferences

An examination of emotion recognition and response to music can isolate perception and experience of emotion from the potentially confounding effects of other social cues (e.g., faces). Participants aged 5-6-years-old listened to clips of calm, scary, and sad music and either identified the emotional content of the music or reported on the feelings elicited by the music clip. Children correctly identified the emotions and reported feeling the emotions conveyed in music above chance. Accurately recognizing and resonating with the emotion conveyed were correlated, although the relationship varied as a function of child characteristics. Specifically, children whose parents reported them as showing more prosocial behavior had significantly greater alignment between emotion recognition and resonation. Results provide new insights into emotion perception in the absence of direct social signals and provide evidence that children’s ability to perceive and resonate with the emotion conveyed through music differs depending on key socioemotional characteristics.

Alignment of Knowing Versus Feeling the Emotion in Music During Middle-Childhood

Children increasingly rely on the internet for information. In this study, children ages 7-10 (n=80) indicated whether a human source or Google could answer questions involving past, present, or future events, and which informant would be better able to do so. Children indicated that Google could accurately answer questions more frequently than the human could, and they were least likely to indicate that either informant could answer questions about the future. Children selected Google as the better informant across all question types, but they did so most frequently in the future condition. Children’s responses also varied such that as the age of the participants increased, they judged the person as less able to answer questions about current events and Google as better able to do so. Children believe that search engines can accurately answer questions more often than a person can, perhaps reflecting their exposure to digital learning environments.

Children’s Judgments About Asking for Past, Present, and Future Information from Google and a Person

What information guides individuals to trust an algorithm? We examine this question across three experiments that consistently found explanations and relative performance information increased trust in an algorithm relative to a human expert. Strikingly however, in only 23% of responses (414/1800) did an individual’s preferred agent for a task (e.g., driving a car) change from human to algorithm. Thus, initial preferences were ‘sticky’ and largely resistant to large shifts in trust. We discuss theoretical and practical implications of this work and identify important contributions to our understanding of how summaries of information can improve people’s willingness to trust decision aid algorithms.

Trusting algorithms: performance, explanations, and sticky preferences

How do people build up trust with artificial agents? Here, we study a key component of interpersonal trust: people's ability to evaluate the competence of another agent across repeated interactions. Prior work has largely focused on appraisal of simple, static skills; in contrast, we probe competence evaluations in a rich setting with agents that learn over time. Participants played a video game involving physical reasoning paired with one of four artificial agents that suggested moves each round. We measure participants' decisions to accept or revise their partner's suggestions to understand how people evaluated their partner's ability. Overall, participants collaborated successfully with their agent partners; however, when revising their partner's suggestions, people made sophisticated inferences about the competence of their partner from prior behavior. Results provide a quantitative measure of how people integrate a partner's competence into their own decisions and may help facilitate better coordination between humans and artificial agents.

How do people incorporate advice from artificial agents when making physical judgments?

Deductive domains are typical of many cognitive skills in that no single problem-solving strategy is always optimal for solving all problems. It was shown that students who know how and when to use each strategy (StrTime) outperformed those who know neither and stick to the default strategy (Default). In this work, students were trained on a logic tutor that supports a default forward-chaining and a backward-chaining (BC) strategy, then a probability tutor that only supports BC. We investigated three types of interventions on teaching the Default students how and when to use which strategy on the logic tutor: Example, Nudge and Presented. Meanwhile, StrTime students received no interventions. Overall, our results show that Nudge outperformed their Default peers and caught up with StrTime on both tutors.

Grounding Action Verbs in Egocentric Visual Perception

Downloads

Next from CogSci 2022

Overloaded Communication as Paternalistic Helping

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES