Canada

The presence of semantic information in multivariate patterns of neural activity has been explored as a method of measuring knowledge and learning. Using fMRI, we investigated whether novice learners of American Sign Language (ASL) showed overlapping representations of semantic categories for words presented in a well-known (English) or newly learned (ASL) language. We find evidence of neural patterns that were partially shared between sign and speech in novice participants. This result provides evidence for the influence of even brief learning on neural representations in cross-modality language processing.

CogSci 2022

Overlapping semantic representations of sign and speech in novice sign language learners

poster

Join us for CogSci2022! Our 44th Annual Conference will be centered around the theme of diversity, with an accompanying range of keynotes and invited speakers who themselves highlight the diversity of our field. Our three symposia are Cognition across Diverse Populations, Cognition within Diverse Societies, and Global Challenges for Cognitive Science. The conference is being offered in a fully hybrid environment for onsite and online attendees.
 
 [**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/07/Cogsci-2022-PAAG-Detailed-July-12final-1.pdf)
 [**The Detailed Program can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/08/DeArmond-CogSci2022-program-v113-final.pdf)

This conference requires a registration to attend. Please use the link below to secure your ticket. The website will be available to all registered attendees on July 25th. 

[Register to Attend](https://cognitivesciencesociety.org/registration/)

Registration Is Required

The 44th Annual Meeting of the Cognitive Science Society highlighted research on the theme Cognitive Diversity in addition to the full breadth and diversity of research topics offered by the society’s membership. 





The effect of prioritization on information in working memory
has primarily been examined in tasks containing a single type
of stimulus and with one item that is prioritized. However,
many theories of working memory posit different types of
components for the maintenance of verbal or visuospatial
information. This study examined differences between
prioritized and nonprioritized items as well as word and image
stimuli. Participants completed an association learning task in
which working memory demands were varied along with the
number of items to be prioritized. Following a short delay,
retention was tested. Prioritization effects were identified
during both the learning and testing phases of the experiment,
and the impact of prioritization was moderated by working
memory demands of the task. Significant differences in
accuracy between word and image stimuli were only observed
in the testing phase, with accuracy for verbal information being
worse. While prioritization improved accuracy and response
times during learning, it led to decreases in the testing phase.

Examining Prioritization in Working Memory for Verbal and Visual Stimuli

As voice-AI technology becomes commonplace in today’s world, speech synthesis technology is rapidly becoming more naturalistic. While previous studies investigated the intelligibility of synthetic speech, it is not clear how the naturalness of a synthetic voice affects listeners’ memory of the meaning content of a spoken message. The present study investigates how listeners’ memory of semantic gist is affected when participants are exposed to a naturalistic synthetic or a roboticized synthetic voice. Participants completed a Deese-Roediger-McDermott (DRM) task to assess recognition accuracy for semantically related word lists. The naturalistic and robotic synthetic exposure voices showed similar levels of recognition accuracy across conditions. However, both synthetic voices resulted in worse recognition accuracy compared to previous research on DRM tasks when the lists were read by human talkers. These findings inform the development of synthetic voices used in information delivery contexts and point to future directions for memory research with synthetic voices.

Investigating the Effect of Synthetic Voice Naturalness on Gist Memory

Temporal Binding (TB) is standardly regarded as an implicit measure of the sense of agency (Haggard, 2017). Though the TB effect is robust, an underlying mechanism has not been agreed upon (Hoerl et al., 2020). Here we propose a memory process as an explanation for the observed error in two publicly available datasets. We first replotted the data and found that on average, across both experiments, participants overestimate the length of the shortest timing interval and underestimate the longest interval, a classic regression to the mean pattern. Summary statistics extracted from the data from each experiment were then used as parameters in a simple Bayesian model of memory. Model simulations reproduced the behavioral data for almost all timing intervals and experimental trial-types across two experiments. Adjusting one of the parameters in the model (prior mean for actions) resulted in an improved qualitative fit. We suggest that other more likely sources of error, apart from experienced agency, may account for this result.

Bayesian rational memory model simulates temporal binding effect

This study focuses on naming and rating ambiguously categorized objects to more accurately represent the real-life naming scenarios that language-users encounter. We examined 407 images of common object categories from a previous study collected in 2013 with monolingual undergraduate students in the United States and China. Of those images, we selected 150 that both (1) balanced the mean and range of naming variables between languages and (2) had low between-language correlations on the same variables. We elicited object names and typicality ratings for these 150 images from a more age-, education-, and geographically-diverse sample of English monolinguals across the US. Naming measures were significantly correlated between current and previous English samples, suggesting high consistency of these object categories within language, especially typicality ratings when conditioned on the object name typicality is not a strictly conceptual measure. Cross-language correlations were low, illustrating the unique categorization patterns between languages.

Lexical Categorization Variables for English and Mandarin Quantify Ambiguity in Object Naming

Memory is limited in capacity, which means that we must choose what information to prioritize for storage. Part of knowing what to prioritize is predicting future needs. For example, if you view a 3D object, later on you may wish to recall exactly how it was oriented. Alternatively, you might need to remember its shape, independent of viewpoint. Given this kind of uncertainty, a good strategy would be to store multiple kinds of information about the objects we observe, and then decode in a task-dependent manner. We tested whether people apply these strategies in the specific domain of short-term memory for novel faces. To test whether people store various kinds of information about a face, and then decode in a task-dependent manner, we modeled their responses in a memory task using features (extracted from deep neural networks) that varied in how much 3D information they carried. We found strong evidence for a mixed-storage strategy, which did not vary in response to task demands. Our results suggest that in order to fully understand resource allocation and retrieval strategies in human memory, it may be critical to consider not just the distribution over tasks in people's natural environments, but also task uncertainty at the time of encoding.

Coding Strategies in Memory for 3D Objects: The Influence of Task Uncertainty

Recent work in judgment and decision making has focused on which actions people consider when solving open-ended problems and found that the actions that come to mind tend to have particular features, such as having a high historical value. Here, we pursue the idea that the process of generating actions for decision-making tasks may actually reflect more general mechanisms for generating kinds of things. We provide evidence that what comes to mind may simply be a reflection of participants sampling from the most relevant part of the representational space they use to encode the type of thing they are generating. In this paper, we (1) introduce an approach for empirically describing a category in terms of the features that people use to represent category members, and for locating category members within that feature space, (2) show that certain locations in a category's feature space predict an item's likelihood of coming to mind, (3) introduce an approach for understanding the relevance of various features to people's representations of category members, and (4) show that features which are most involved in people's representations of category members are also predictors of what comes to mind within a category. We close by proposing that features that are most relevant to our representations of category members and predict coming to mind are those for which it has been historically useful to have information about during past experiences with the category in question.

What comes to mind? Samples from relevance-based feature spaces

Testing the testing effect with featural and relational categories
Theoreticians have recently proposed that the benefits of testing depend on the extent to which the to-be-learned concepts are interconnected, with testing effects emerging from low element interconnectivity and becoming weaker with greater element interconnectivity. We tested this idea by using a classification task and manipulating the amount of element interconnectivity in the categories that subjects learned. Some subjects learned featural categories, which lack element interconnectivity, whereas others learned relational categories, which are defined by how its elements are interconnected. This factor was crossed with type of training, wherein some subjects learned through classification (i.e., testing) and others through observation (i.e., studying). For featural categories, it was predicted that classification training would produce better learning than observational training, whereas this benefit should become weaker or disappear for relational categories. However, preliminary results show that both types of training lead to similar learning for both category types.

Testing the testing effect with featural and relational categories

We investigated neural networks’ ability to generalize during visual object recognition. In three experiments, we show that while basic multilayer neural networks easily learn to classify the objects on which they are trained, they show serious difficulties transferring that knowledge to novel items. However, our experiments also show that when the previously trained networks are then trained on the novel items, they learn to respond correctly to the novel items much faster than untrained networks. This shows that these networks are learning abstract representations that go beyond the simple items on which they were trained. We argue that this demonstrates that regarding abstract rule learning, the problem with neural networks is not their inability to learn abstractions, but their ability to apply that knowledge when classifying new objects.

Generalization and Transfer Learning in Neural Networks Performing Shape, Size, and Color Classification

Holistic and analytic thinking styles are well-documented in cultural psychology. However, recent studies suggest that language potentially mediates the influence of culture on thinking styles. The overarching goal of this study is to examine how verbal labels impact people’s thinking styles. Study 1 sought to examine whether thinking styles in a classic triad task could depend on verbal or pictorial formats. Although we observed a significant correlation between performance in verbal and picture triad tasks, more participants were classified as holistic thinkers with a verbal compared to a picture triad task. In Study 2, we examined whether participants could shift their thinking styles in the verbal triad task after being primed to focus on categorical associations. We found that females were influenced by this prime and displayed more analytic thinking. Our results suggest that language can influence thinking styles and that thinking styles are context-dependent.


Verbal Labels Affect Holistic and Analytic Thinking Styles in Native English Speakers

Basic category learning mechanisms are largely thought to be universal (Shepard, 1987). However, research has shown differences in processing styles between Eastern and Western cultures (Nisbett et al., 2001; Norenzayan et al., 2002), though this has not always been replicated (Murphy et al., 2017). To explore category learning styles and effects of culture, this online study paired the Shepard, Hovland, and Jenkins tasks (Shepard et al., 1961) with the Analysis-Holism scale (Choi et al., 2007). These tasks vary in complexity and optimal strategy to test the reliance on single feature rules, disjunctive rules, and family resemblance. Cultural differences for holistic approaches in non-Western samples were expected when categories could be acquired in more ways than one. The results show expected learning trends for the SHJ tasks, signifying reliable data collection. Additionally, of the six tasks, higher holistic thinking was significantly correlated with the family resemblance task (Type IV).

Premium content

Overlapping semantic representations of sign and speech in novice sign language learners

Next from CogSci 2022

Examining Prioritization in Working Memory for Verbal and Visual Stimuli

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES