United States

The McGurk effect is a multisensory phenomenon, an audiovisual illusion that shows how speech sounds may interfere with the visual sense. In case of an incongruency between a speech sound and a human face articulating the speech sound, human interlocutors tend to perceive an audiovisual percept different from the sound percept and the visual percept. For example, if the speech sound is ba and the visual presentation of the articulation is fa, human interlocutors usually perceive a third sound va. What can we learn about audiovisual integration using an avatar face? To what extent do we observe the McGurk effect? The present study reports an empirical investigation that tested the extent of the effect in an avatar face designed for the purpose of the study. Our findings suggest that the fidelity of realistic avatar designs may be evaluated by employing human responses to audiovisual illusions on avatar faces.

CogSci 2023

Multisensory Integration of Audiovisual Information in Avatar Faces: An Experimental Investigation of McGurk Effect

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

The new Sans Forgetica (SF) typeface, creates perceptual disfluency through fragmented letters, which make it difficult for readers to use good continuation and perceptual completion to identify letters. We compare CRT performance and font legibility ratings when CRT problems are presented in SF or Arial. In Experiment 1, participants rated legibility of both fonts after solving CRT problems. Participants who solved the problems in SF rated Arial to be harder to read than those who solved the problems in Arial, suggesting that after prolonged use of SF, participants viewed Arial as being closer in legibility to SF. In Experiment 2, participants rated legibility of both fonts before and after solving CRT problems. The patterns of legibility ratings were very similar across the Arial and CRT font conditions, suggesting that SF was not a metacognitive cue for difficulty. These legibility ratings help explain the null effects of SF on CRT performance.


Cognitive Reflection Test Performance with Arial and Sans Forgetica Font

In 1998/1999 three participants trained for up to 74 hour long sessions to find a target in visual displays of 1, 2, or 4 objects. There were four targets and four foils that never changed. Displays occurred simultaneously, or the objects occurred successively, or the features of each object occurred successively. When successive, the SOAs were short (17, 33, or 50 ms) so the displays appeared simultaneous, making it likely that the search strategy was the same in all conditions. A 2004 publication examined only the simultaneous condition and found evidence for serial search as well as some small amount of automatic attention to targets, and occasional early or late search termination. A 2021 publication examined only the displays with single objects, obtaining evidence for dynamic perception of features. Building on these results we present a simple dynamic model that explains the main processes operating in all the experimental conditions.

Dynamic Modeling of Visual Search

Psycholinguistic accounts of pronoun resolution have emphasized the influence of discourse-level and lexical cues (e.g., order-of-mention/coherence relations/implicit causality). Less work has considered situation-specific pragmatic reasoning effects, which we explore here using communicative perspective-taking. Expts1-2 used sentences like “Molly [asked Ana if]/[told Ana that] she remembers when lecture starts” to assess readers’ judgments for antecedents of ambiguous pronouns. We predicted that characters ASKING about the subordinate clause information should entail readers interpreting the pronoun as co-referring with the main-clause object, whereas TELLING should entail main-clause subject interpretations. This is because we do not tell people what they already know nor ask people something that we should know that they do not. The results corroborated our predictions: Participants picked the “perspectivally-congruent” antecedent 99.8% of the time, with no order-of-mention effect (which would predict stronger effects for TELL, where the antecedent is the first-mentioned character). Expts3-4 corroborated the results through self-paced reading tasks.

Drawing on Perspective Cues in Pronoun Resolution

Lab-based research on insight has typically explored solutions on specially designed puzzles, but there is growing interest in exploring insight in more naturalistic contexts. Anecdotally computer programmers report experiences that sound similar to common insight phenomena such as Aha! and restructuring . To better understand moments of insight in computer programming, we conducted a protocol study where we asked intermediate-level computer-programming students to think-aloud as they solved three problems that required non-obvious solutions. Participants began their solutions on a whiteboard and then implemented their solutions in C++. This yielded several data sources including the trace data and the quality of their programmed solutions. This data was used to test whether and when programmers experienced Aha! or restructuring while solving these problems. This data was also used to test whether common findings in insight problem solving research such as the Aha-Accuracy effect and the theorized impasse-insight sequence are present when computer programming.


Exploring Restructuring and Aha! in the Context of Computer Programming

The purpose of this study is to examine how young adults use their metacognitive abilities such as cognitive strategies and self-checking during the mathematics problem-solving process. The study group consisted of 12 young adults selected from three different faculties. A multimodal mixed-methods design was employed, where participants were asked to think out loud while solving ten mathematical problems. The experimental process was recorded with the use of eye-tracking, which was utilized to evaluate the active use of metacognitive sub-skills. The findings from the experimental process revealed that there is a significant difference between the amount of reflection of young adults’ and pupils' cognitive strategy and self-checking skill levels on their responses to mathematics problem solving process in favor of young students.

Differential Effect of Young Adults’ and Students’ MetaCognitive Skills in Mathematics Problem Solving Process

Collaborative problem solving capability (CPS) has become more important recently. Accurate measurement of CPS is necessary for developing training programs. However, most existing CPS measurement tools have two weaknesses. They do not account for individual contributions to the output. Also, it is hard to utilize new problems. Addressing these weaknesses, we modified the PISA 2015 framework to account for individual contributions to the output and introduced peer-assessment. To test the validity of our proposition, we had fifty participants perform two tasks in different groups, and analyzed the results. We found that the score from PISA 2015 and our modified score showed a high correlation. And, peer-assessed modified scores approximated expert-assessed modified scores. Furthermore, peer-assessed modified scores between the tasks showed a significant correlation, suggesting that it is consistent across different tasks and groups. These results suggest that the modified method and adopting peer-assessment can be used to measure CPS.

Measuring Collaborative Problem Solving Capability Using Peer Assessment

Recent work has suggested that caregivers adjust their speech to their child's individual lexical knowledge when playing a reference game. Whether caregivers display similar scaffolding during shared reading interactions remains unexplored. We hypothesized that similar adjustments may characterize caregivers’ communication during shared reading interactions and support children’s word learning. Caregivers and their 4- and 5-year-olds engaged in shared reading with two custom storybooks with embedded low-frequency words. Immediately before the session, caregivers were asked whether their child understood or produced those words via a vocabulary checklist. The shared reading interactions were video-recorded and transcribed. Caregivers’ communication was coded offline for repetitions, questions, comments, and gestures in relation to target words. Children’s word learning outcomes were measured after the shared reading episodes. Results will clarify whether and how caregivers adjust their communication to their perceived child's individual lexical knowledge, and whether this adjustment enhances word learning in naturalistic settings.

Do caregivers’ communicative adjustments enhance 4- and 5-year-olds' word learning? Testing the generalizability of the fine-tuning hypothesis to naturalistic shared reading interactions

This study investigated the effect of interleaving and epistemological beliefs on the reading comprehension of multiple conflicting documents. Especially, it is assumed that interleaving may highlight the differences through discrimination and contrast, thus improving the integration of source and content information. A one-factor experiment was conducted (n = 179) to examine the effect of interleaving and spacing on reading comprehension. In addition, a parallel chain mediation model was established based on the Documents Model Framework. Path analysis results indicated an interleaving effect which took place through germane cognitive load and viewpoint description, while spacing enhanced source memory and source-content links. Furthermore, justification for knowing moderated the effect of interleaving on germane cognitive load. Participants with high level of justification reported the lowest germane cognitive load in the interleaved condition. Potential mechanism of interleaving in multiple document reading was discussed.

Integrating in Interleaved Conflicting Documents: Content Preference and Epistemic Beliefs Matter

Popular culture consists of elements consumed by the general public, such as movies, television shows, and books, that reflect the attitudes, practices, beliefs, and cultural objects prevalent in society. Popular culture tends to influence numerous facets of one’s lifestyle, and both empirical and anecdotal evidence suggests that it plays a role in determining the career of nascent professionals. We conducted an anonymized survey to assess the effect of popular culture on pursuing a career in Science, Technology, Engineering, Mathematics, and Medicine (STEMM). Results from the path analysis identified the direct effect of different aspects of popular media on the perceptions of social standing in choosing a STEMM career.

Exploring the relationship between popular culture and perceptions of social standing in Science, Technology, Engineering, Mathematics, and Medicine (STEMM) careers: a pilot study using path analysis

In 2021, charitable giving accounted for about 2.1% of the national gross domestic product in the U.S., with 67% of donations coming from individual donors. Despite the profound impact of this industry on the economy, with respect to its provision of services and goods as well as employment opportunities and wages, much is still unknown about the psychological processes supporting charitable giving. One major theoretical distinction emerging in the literature is that between acts of charity motivated by a consideration of the needs of recipients vs. those prompted by the needs of the giver. Here, we investigated how socio-affective constructs of “simulated vs. genuine compassion” impact the inclination to make public vs. anonymous charitable donations and assessed how information about social norms influences those differences.

Premium content

Multisensory Integration of Audiovisual Information in Avatar Faces: An Experimental Investigation of McGurk Effect

Next from CogSci 2023

Cognitive Reflection Test Performance with Arial and Sans Forgetica Font

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES