United States

In assessing the sense of agency (SoA) in the field, researchers are often confronted with the limitations of conventional methods (e.g., retrospectivity, task-disruption). Generally, facial analysis offers a promising possibility to overcome these obstacles by directly collecting objective data in real-time and without any need for contact. This paper examines the applicability of face tracking for SoA assessment by investigating whether facial expressions can be used to derive information about the extent to which users feel in control in interactions with technical systems. In a laboratory experiment (N = 27) based on the dot control task, we manipulated SoA via task difficulty and explored the effects on facial expression patterns. Three action units in the region of the mouth were observed to be characteristic facial indicators of SoA. These findings support the suitability of facial expressions for SoA evaluation as well as the potential of face tracking for monitoring SoA in various practical fields of application (e.g., automated driving).

CogSci 2023

Are Facial Expressions Predictors for the Sense of Agency in a Dot Control Task?

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Multimodal social signals are used when a third person perceives the relationship between multiple people. In this study, in order to determine whether people can detect improvisation played by skilled actors from audio-visual information, improvised and scripted play stimuli were presented online and participants were asked to identify which was the improvised play. Participants (n＝216) were randomly divided into three conditions (Audiovisual, Visual only, and Auditory only) and asked about their confidence level. Results showed significantly higher rate of correct response in Audiovisual and Auditory-only conditions than in Visual-only condition. In addition, examination of the free-text responses revealed that those who answered correctly in the audiovisual condition relied primarily on pauses in conversation to identify. These results suggest that in the performances of skilled actors, an improvisation is manifested in auditory signals as social information, which can be detected from the third person.

An on-line study about recognition of improvisation theatre using audio-visual information

We cannot help but form a first impression every time we meet someone unfamiliar to us. While the existing literature has almost exclusively focused on facial impressions, we know from our everyday experiences that other aspects of a person can also inform the judgements we make. Here, I use a data-driven approach to reveal the underlying structure of first impressions attributed to human voices, first names and those based on multimodal information (face, voice and name). Approximately 40,000 spontaneous descriptors were used to establish the most common first impression traits and Exploratory Factor Analyses revealed a number of fundamental dimensions, capturing judgements of approachability, confidence, and intelligence. Surprisingly, multimodal first impressions were primarily driven by voice cues and this was observed even when the presentation order of the three cues (faces, voices, names) was systematically manipulated, providing evidence for the remarkable role of voices in the formation of first impressions.

Multimodal Person Evaluation: First Impressions from Faces, Voices, and First Names

In this study, we investigate the priming effect of camera-induced objectifying-gaze, operationalised via sexualised music video(MV), on university students when looking at target female images in traditional (saree, salwar) and western (shirt-pant, short-dress) attires in the Indian demographics. We showed participants two videos where female leads wore unrevealing Indian attire (saree) and revealing western clothing in sexualised and traditional MVs respectively. We quantified gaze as fixation-duration and visit-count (revisit frequency) metrics. Our findings report that there is a priming effect of sexualised media. Female images in all attires are equally objectified and body-biased gaze is more pronounced in sexualised MV priming, short-dress is more objectified in traditional MV priming, and saree is less objectified overall, which confirms the social stereotypes on attire and respectability — traditionally clad (vs. non-traditional) women are seen as more moral and worthy of respect and protection compared to women in hypersexualised and objectified roles.

The Objectifying Gaze: Impact of Sexualized Media on Viewer Gaze Behavior towards (Non)Traditional Attire

Combining perceptual learning techniques with adaptive learning algorithms has been shown to accelerate the development of expertise in medical and STEM learning domains. Virtually all adaptive learning systems have relied on simple accuracy data that does not take into account response bias, a problem that may be especially consequential in multi-category perceptual classifications. We investigated whether adaptive perceptual learning in skin cancer screening can be enhanced by incorporating signal detection theory (SDT) methods that separate sensitivity from criterion. SDT-style concepts were used to alter sequencing, and separately to define mastery (category retirement). SDT retirement used a running d’ estimate calculated from a recent window of trials based on hit and false alarm rates. We used a Skin Cancer PALM (perceptual adaptive learning module) to train classification of 10 cancerous and non-cancerous skin lesion types. Four adaptive conditions varied either the type of adaptive sequencing (standard vs. SDT) or retirement criteria (standard vs. SDT). A control condition presented didactic instruction on dermatologic screening in video form, including images, standard classification schemes, and detailed explanations. All adaptive conditions robustly outperformed the control condition in both learning efficiency and fluency (large effect sizes). Use of SDT retirement criteria produced greater learning efficiency at both immediate and delayed posttests (medium effect sizes). SDT and standard adaptive sequencing did not differ. SDT enhancements to adaptive perceptual learning procedures have potential to enhance learning efficiency.

Connecting Adaptive Perceptual Learning and Signal Detection Theory in Skin Cancer Screening

Scatterplot research has identified factors that impact people’s perception of correlation magnitudes, yet much less is known about how people reason about data represented in scatterplots. We investigated how people make correlational and causal inferences based on scatterplots with and without outliers. In Experiment 1 and 2, participants viewed scatterplots matched in overall correlational magnitude depicted, but half had an outlier. In Experiment 3, the scatterplots in the two conditions were matched in the correlation magnitude depicted by all the dots excluding the outlier. For each scatterplot, participants stated their endorsement for correlational (X and Y change together) and causal statements (X changes Y). Only when outliers further strengthened an already moderate to strong relationship, people endorsed related correlational statements more and showed a stronger causality bias. Altogether we demonstrate that the impact of outliers in scatterplots on visual reasoning depends on the strength of the relationship depicted.

Processing Scatterplots: Impact of Outliers on Correlational and Causal Inferences

How humans visually cluster points is potentially relevant to perception, information visualization, and other areas. However, there are relatively few empirical studies and computational models of this ability. Here, we propose a new competitive clustering_ model that uses neurally plausible mechanisms such as hebbian learning and lateral inhibition. We evaluate its fit to the data from a behavioral study of visual clustering, as well as the fits of two categorization models (the Rational model and SUSTAIN) and $K$-Means. We find that people are highly reliable in their clusterings of the same stimulus on two occasions, suggesting they are using a stable strategy. The models were generally successful at predicting human clusterings and were able to replicate the qualitative performance profile of human reliability over number of points and over cluster structure. The performance of the competitive clustering model motivates further investigation of its computational properties and empirical validity.


Models of human visual clustering

Previous work shows that visual perception of biological motion is supported by a specialized neural system and is distinct from non-biological or mechanical motion. However, there has been no systematic characterization of what construes mechanical motion, especially in the context of complex actions performed by humans and non-human agents like robots. In the present study, we proposed four different methods that disrupt biological motion and investigated to what extent the altered motion stimuli were perceived mechanical by human observers (N=230). The methods manipulated the pattern of motion trajectories, their timing, and the number of the limbs used in the performed actions. The results show that the motion stimuli were found most mechanical when the pattern of the motion trajectory was changed. We also found that as the number of limbs whose motion trajectory was changed increased, the perceived mechanicalness increased.

Visual perception of mechanical motion: A comparison of methods that disrupt biological motion

The McGurk effect is a multisensory phenomenon, an audiovisual illusion that shows how speech sounds may interfere with the visual sense. In case of an incongruency between a speech sound and a human face articulating the speech sound, human interlocutors tend to perceive an audiovisual percept different from the sound percept and the visual percept. For example, if the speech sound is ba and the visual presentation of the articulation is fa, human interlocutors usually perceive a third sound va. What can we learn about audiovisual integration using an avatar face? To what extent do we observe the McGurk effect? The present study reports an empirical investigation that tested the extent of the effect in an avatar face designed for the purpose of the study. Our findings suggest that the fidelity of realistic avatar designs may be evaluated by employing human responses to audiovisual illusions on avatar faces.

Multisensory Integration of Audiovisual Information in Avatar Faces: An Experimental Investigation of McGurk Effect

The new Sans Forgetica (SF) typeface, creates perceptual disfluency through fragmented letters, which make it difficult for readers to use good continuation and perceptual completion to identify letters. We compare CRT performance and font legibility ratings when CRT problems are presented in SF or Arial. In Experiment 1, participants rated legibility of both fonts after solving CRT problems. Participants who solved the problems in SF rated Arial to be harder to read than those who solved the problems in Arial, suggesting that after prolonged use of SF, participants viewed Arial as being closer in legibility to SF. In Experiment 2, participants rated legibility of both fonts before and after solving CRT problems. The patterns of legibility ratings were very similar across the Arial and CRT font conditions, suggesting that SF was not a metacognitive cue for difficulty. These legibility ratings help explain the null effects of SF on CRT performance.


Cognitive Reflection Test Performance with Arial and Sans Forgetica Font

In 1998/1999 three participants trained for up to 74 hour long sessions to find a target in visual displays of 1, 2, or 4 objects. There were four targets and four foils that never changed. Displays occurred simultaneously, or the objects occurred successively, or the features of each object occurred successively. When successive, the SOAs were short (17, 33, or 50 ms) so the displays appeared simultaneous, making it likely that the search strategy was the same in all conditions. A 2004 publication examined only the simultaneous condition and found evidence for serial search as well as some small amount of automatic attention to targets, and occasional early or late search termination. A 2021 publication examined only the displays with single objects, obtaining evidence for dynamic perception of features. Building on these results we present a simple dynamic model that explains the main processes operating in all the experimental conditions.

Premium content

Are Facial Expressions Predictors for the Sense of Agency in a Dot Control Task?

Next from CogSci 2023

An on-line study about recognition of improvisation theatre using audio-visual information

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES