United States

Robots must be able to communicate naturally and efficiently, e.g., using concise referring forms like it, that, and the ⟨N’⟩. Recently researchers have started working on Referring Form Selection (RFS) machine learning algorithms but only evaluating them offline using traditional metrics like accuracy. In this work, we investigated how a cognitive status-informed RFS computational model might fare in actual human-robot interactions in a human-subjects study (N=36). Results showed improvements over a random baseline in task performance, naturalness, understandability, and mental workload. However, the model was not perceived to outperform a simple, naive, non-random baseline (constant use of indefinite noun phrases). We contribute several key research directions for further development of cognitive status-informed RFS models, the inclusion of multi-modality, and further development of testbeds.


CogSci 2023

Evaluating Cognitive Status-Informed Referring Form Selection for Human-Robot Interactions

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Androids that strongly but imperfectly resemble humans in shape can elicit negative emotions in people, a phenomenon known as the "uncanny valley," which has been replicated in laboratory experiments. Recently, the accuracy of face recognition utilizing machine learning has increased, raising the question of whether machine learning can replicate the uncanny valley. Using FaceNet as a representative face recognition algorithm, we examined the similarity of face recognition to human evaluation and its replication of the uncanny valley. The results revealed a strong correlation between machine learning and human evaluation of human-like shapes. However, it is evident that only certain aspects of the uncanny valley were replicated. Furthermore, visualization of the activation maps suggests that localized regions, such as the mouth and chin, acted as the basis for judgment. These findings support the idea that human and machine learning have distinct areas of attention.

Does Machine Learning Replicate the Uncanny Valley? An Example using FaceNet

Working Memory (WM) plays a key role in natural language understanding and generation. To enable a human-like breadth and flexibility of language understanding and generation capabilities, cognitive systems for language-capable robots should feature a human-like WM system in a similarly central role. However, it is still quite unclear how robotic WM should be designed, as a variety of models of human WM have been proposed in cognitive psychology. Moreover, human reliance on WM during language production is sometimes to help the speaker rather than to help hearers. Thus, it is unclear whether different robotic WM systems might harm certain dimensions of interaction for the sake of the robot speaker's ostensible ease of cognitive processing. In this paper we demonstrate how different models of human WM can be implemented into robot cognitive architectures. Our results suggest that these models can be effective in terms of accuracy, perceived naturalness, and perceived human-likeness.

Forget About It: Entity-Level Working Memory Models for Referring Expression Generation in Robot Cognitive Architectures

Event cognition research has typically considered events to be contiguous in time, with defined starts and ends. However, people sometimes engage in more than one event at the same time. If this happens frequently, then theories of event cognition may require modification. This research study aims to estimate how often people engage in multitasking in daily life. Ninety-seven participants were asked whether they had been multitasking at four time points during the last 24 hours. Forty-five per cent of responses reported multitasking with a diverse range of event structures. Twenty-one per cent of reports specifically listed multiple overlapping activities. The prevalence of multitasking suggests that theories of event cognition need to be expanded to accommodate non-contiguous and simultaneous events.

The prevalence of multitasking presents challenges for theories of event segmentation

The ability to represent and reason about other people’s perspectives (‘social cognition’, or ‘Theory of Mind’ [ToM]) plays an important role in daily life. It is not currently clear how different types of interaction may impact subsequent perspective taking, and whether this differs with age. Here, we examine how experiences of social inclusion vs. social exclusion (manipulated using the Cyberball task) influence later perspective-taking efficiency, in younger (19-30 years; N = 100; M = 24.69) and older (65-80 years; N = 100; M = 69.63) participants. As expected, significant age differences were found in perspective-taking abilities, with younger participants out-performing older participants. This was specific to false-belief trials, with no difference in memory trials performance. Counter to hypothesized, no interaction between impact of social experience (inclusion vs. exclusion) and age was found, indicating that both younger and older adults are similarly impacted by social experiences. Implications will be discussed.

The Impact of Social Inclusion vs. Exclusion on Subsequent Social Cognition Abilities in Younger and Older Adults

People care about others’ welfare to varying degrees, captured by the welfare tradeoff ratio (WTR), or the weight placed on another person’s welfare compared to one’s own. People can infer another person’s WTR toward themselves and reciprocate by adjusting their own WTR toward that person. However, the evolutionary origin of such a capacity is unclear. In games with perfect information, a heuristic strategy with tit-for-tat-like reciprocity is unbeatable, and the additional computation of inferring the opponent’s WTR confers no benefit. Here we show that in games with asymmetric information, where the actor has more accurate information about the payoff structure than the observer, the heuristic strategy is prone to errors from misperception, while reciprocity based on the inference of WTRs is robust. These findings suggest that asymmetric information about social decisions, a realistic modification to the game environment, may have contributed to the evolution of people’s understanding of others’ WTRs.


The evolution of reciprocity based on welfare tradeoff ratios in games with asymmetric information

Children's perception of gender stereotypes influences their gender role representations, leading to preferences regarding gender-toy associations and gender-based beliefs. Exposure to literary stories may shape children’s gender-role representation by either enforcing stereotypes or countering them. The current study investigates one type of literature that young children are often exposed to: folklore stories with goddesses, which contain abundant and varied gender information across different cultures. Twenty-eight 4-to-5-year-olds were randomly assigned to hear either a gender role typical or atypical goddess story. Toy stereotyping ratings and confidence levels of responses were obtained before and after the story exposure. This is ongoing study provides a preliminary summary of investigating the impact of story reading on young children’s gender stereotype representation. It also provides insight regarding the need for and ways to measure impacts of story-reading.

Story Reading and Children’s Gender Stereotype Representation

We rated video recordings of participants playing a charades-like semiotic game, where they took turns miming stories and picking the sequence of pictures that best suited what was mimed. The stories had one (male or female) or two characters (male and female). Although there was no pressure to disambiguate between the characters, most of the participants did so by producing iconic or indexical gender markers. Our analysis shows that when the gender of the characters in a story (the referent environment) matched the gender of the participants (the physical environment), the participants were more likely to use indexes rather than icons (p < 0.05, β = 1.4247). These results complement research on the role of the environment in the emergence of language-like structures and their systematicity in silent gesture tasks (e.g. Nölle et al. 2018).


Does the environment shape gender marking in mimed stories?

Past studies have shown that adult listeners can classify gender from children’s speech – long before anatomical differences between the sexes emerge. This suggests that gender differences in speaking style are learned early in life through socialization. Yet, no study to date has asked whether children also perceive gender cues in child speech. In Study 1, we found that 3.5- to 6-year-olds were able to detect gender from their age-matched peers’ speech, and that their performance was moderated by parents’ gender attitudes. In Study 2, we examine whether the effect of parental socialization is mirrored in children's production. We predicted that how easy children's gender can be perceived from their speech is moderated by parents’ gender attitudes. Our findings support our prediction. Taken together, our results demonstrate that sensitivity to gender cues emerges early in life, and that parental socialization contributes to the development of both gendered speech perception and production.

It Starts at Home: The Role of Parental Socialization on the Development of Gendered Speech

During referential communication, speaker choices regarding the syntactic encoding of their expressions can modulate the linear ordering of the properties necessary to identify the referent. We investigated whether such syntactic choices are influenced by the informativity of these properties in a given visual context, as quantified by Referential Entropy Reduction (RER). In two experiments, a maze-based sentence completion task was used to examine whether informativity of a particular property (animal or action) influenced the decision to produce pre- versus post-nominal modifications when describing animal-performing-action referents in a visual scene. While many participants used a fixed strategy, informativity did significantly influence linearization for the remaining participants, consistent with a maximal informativity strategy in which the high RER property is encoded first. This suggests that speakers who vary their encodings are indeed sensitive to the informativity of properties in a visual scene, preferring syntactic linearization in which informative properties appear early.


Does informativity modulate linearization preferences in reference production?

This paper presents a challenge to enactivist approaches to cognition (e.g. Ward, D., Silverman, D. & Villalobos, M. 2017): how can enactivists integrate data and results from research methods used in developmental psychology and animal cognition studies that heavily rely on representations, given enactivism's aversion to representations? I discuss theoretical commitments behind forms of looking time studies that have been used to probe into the cognitive abilities of infants and animals, focusing on the Violation of Expectation (VoE) paradigm (Ginnobili & Olmos 2021), to illustrate why these methods might pose an especially difficult problem for enactivists and their re-interpretation of cognitive science in terms of dynamic coupling between organism and environment. I argue that despite the lack of clarity regarding how the notion of expectation should be applied to the minds of neonates and nonhuman animals, there is an inherently representational aspect to expectation, given that it embodies satisfaction conditions.


Premium content

Evaluating Cognitive Status-Informed Referring Form Selection for Human-Robot Interactions

Next from CogSci 2023

Does Machine Learning Replicate the Uncanny Valley? An Example using FaceNet

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES