Canada

A notional machine is an abstract representation of how a computer works. Prior research shows that students often form incomplete or inaccurate mental models of the notional machine, reflecting misunderstandings and causing programming errors. To date, however, there is very little work on how to help students acquire accurate models of the notional machine. To fill this gap, this research focuses on novice programmers and experimentally investigates the impact of direct instruction of the notional machine compared to standard instruction that does not include the notional machine. The instruction is embedded in an online computer tutor that we implemented and compared to a version of the tutor that does not refer to the notional machine but otherwise includes the same materials. In particular, both tutor versions include instructional videos followed by self-explanation prompts. We will present data from an evaluation of both tutor versions.

CogSci 2022

Self-Explaining the Notional Machine to Improve Novice Programmers’ Learning and Mental Models, Supported by a Computer Tutor System

poster

Join us for CogSci2022! Our 44th Annual Conference will be centered around the theme of diversity, with an accompanying range of keynotes and invited speakers who themselves highlight the diversity of our field. Our three symposia are Cognition across Diverse Populations, Cognition within Diverse Societies, and Global Challenges for Cognitive Science. The conference is being offered in a fully hybrid environment for onsite and online attendees.
 
 [**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/07/Cogsci-2022-PAAG-Detailed-July-12final-1.pdf)
 [**The Detailed Program can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/08/DeArmond-CogSci2022-program-v113-final.pdf)

This conference requires a registration to attend. Please use the link below to secure your ticket. The website will be available to all registered attendees on July 25th. 

[Register to Attend](https://cognitivesciencesociety.org/registration/)

Registration Is Required

The 44th Annual Meeting of the Cognitive Science Society highlighted research on the theme Cognitive Diversity in addition to the full breadth and diversity of research topics offered by the society’s membership. 





The dual mechanisms of control framework describes two modes of goal-directed behavior: proactive control (goal maintenance) and reactive control (goal activation on task demands). Shifts between these modes may explain variations in user performance in computing tasks. Although these mechanisms are relevant to the design of interactive systems, and particularly educational systems, their relation to human-computer interaction (HCI) is under-researched. We propose a manipulation to induce proactive or reactive control in the context of mathematical problem solving on an online tutoring system. We present two experiments where students solved problems using either proactive or reactive control. Study 1 validates the manipulation by investigating behavioral measures that reflect usage of the intended strategy and assesses whether either cognitive control mode impacted learning. Study 2 investigates whether alternating between control modes during problem solving affects student performance.

Eliciting Proactive and Reactive Control during Use of an Interactive Learning Environment

Online data collection has become a prominent option due to the COVID-19 pandemic. It is crucial to understand to what extent online studies can be compared with face-to-face studies, particularly in multimodal language research on which the modes of communication have a crucial effect. This study investigated multimodal communication across face-to-face and videoconferencing settings, focusing on gesture production and speech disfluency in a daily routine description task (N=64). Results suggested that overall disfluency rate was higher for those who communicated via videoconferencing than those who communicated face-to-face. The use of specific disfluency types also differed across the two settings, signaling an interplay between cognitive and communicative strategies. Overall gesture frequency and iconic gesture use were comparable across the two settings. Iconic gesture use negatively predicted the overall disfluency rate, regardless of the setting. Using different contexts is required to understand whether multimodal language differs between face-to-face and online communication.

Multimodal Communication in Virtual and Face-to-Face Settings: Gesture Production and Speech Disfluency

Gestures facilitate speech production by helping speakers reduce cognitive load. Studies on gesture-speech interaction mostly examined the effect of representational gestures on spatial contexts. However, abstract deictics (e.g., pointing at objects that are not visually present) might also have a role in facilitating cognitive processes. The present study investigated the effect of gestures on disfluency rates by presenting a narrative task in three conditions: spontaneous, restricted, and encouraged gesture use. We found that disfluency rates across three conditions did not significantly differ. The use of abstract deictics in the spontaneous gesture use condition was a significant predictor of disfluency rates in the gesture restricted condition. Results indicate that gestures’ facilitative roles might be manifested differently depending on the context. Abstract deictics might also benefit speakers, especially in a narrative context. Studying abstract deictics can provide new insights on gesture and speech production interaction.

Gesture and Speech Disfluency in Narrative Context: Disfluency Rates in Spontaneous, Restricted, and Encouraged Gesture Conditions

Myriad linguistic cues, subtle and otherwise, have been shown to shape thinking and reasoning. Such linguistic framing effects are attributed variously to irrational biases, affective processes, and analogy. Across two studies, we explored pragmatic reasoning (PR) as a potential unifying mechanism underlying disparate framing effects. In Study 1, we adapted a measure of PR used in attribute framing research, assessing the ability to infer information subtly communicated by a speaker’s choice of frame. Our measure captured individual differences in PR distinct from cognitive reflection and mentalizing ability. In Study 2, PR predicted the effect of identity labels on attitudes towards illegal immigration, but not other framing effects. Two such effects were driven, however, by participants’ explicit recognition of the influence of the framing language. These results suggest that framing effects may indeed be mediated by PR, but that our measure of PR does not fully capture its mediating role.

Exploring the Role of Pragmatic Reasoning in Linguistic Framing Effects

Accessibility plays a major role in speech production. Here we investigate and measure four factors that influence speakers to produce one word over another more optimal word form. Three experiments asked participants to label images of insects and instruments. Participants were incentivized to produce an accurate specific label (e.g., bee), over a more general label (e.g., insect), so that specific labels were more optimal. Each of three experiments manipulated a different factor that could influence accessibility – word frequency, priming, and interference – and all experiments additionally varied whether labels had to be produced under time pressure or not. Results showed that each variable significantly influenced the accessibility of labels: participants produced more specific labels when those labels were higher frequency, when they were primed, when a visually-similar label had not been primed, and when participants were unconstrained by time pressure. These findings demonstrate that multiple factors influence the accessibility of familiar words during production, regularly leading participants to rely on “good-enough” rather than optimal options to convey their message.

Accessibility factors that lead to good-enough language production

Sentence comprehension involves simultaneous processes such as maintaining and integrating different types of verbal representations. As such, it has been argued that sentence comprehension relies on working memory (WM). Some findings suggest that semantic (word meaning) WM rather than phonological (speech sound) WM is critical for comprehension. This study took a case-series multiple regression approach to examine the relationship between sentence comprehension and WM for 56 individuals with aphasia. We examined the independent contribution of phonological and semantic WM in predicting comprehension for higher WM target sentences relative to matched lesser WM sentences, while also controlling for single word processing. We found that only semantic WM had a significant contribution to comprehension for three contrasts. However, for the fourth contrast of trials requiring syntactic processing with those requiring only lexical processing, both WM contributions were significant. The possible backup role of phonological WM for comprehension of role reversals is discussed.

Semantic Working Memory Predicts Relative Clause Sentence Comprehension: A Case Series Approach

Neural networks struggle in continual learning settings from catastrophic forgetting: when trials are blocked, new learning can overwrite the learning from previous blocks. Humans learn effectively in these settings, in some cases even showing an advantage of blocking, suggesting the brain contains mechanisms to overcome this problem. Here, we build on previous work and show that neural networks equipped with a mechanism for cognitive control do not exhibit catastrophic forgetting when trials are blocked. We further show an advantage of blocking over interleaving when there is a bias for active maintenance in the control signal, implying a tradeoff between maintenance and the strength of control. Analyses of map-like representations learned by the networks provided additional insights into these mechanisms. Our work highlights the potential of cognitive control to aid continual learning in neural networks, and offers an explanation for the advantage of blocking that has been observed in humans.

A Neural Network Model of Continual Learning with Cognitive Control

Recent work has shown that the brain organizes abstract, non-spatial relationships between entities into map-like representations. However, an animal’s objectives often depend on only a subset of the features of the environment. Under these circumstances, cognitive control – the capacity to flexibly select the features most relevant in the current context – becomes paramount. Here, we explore the relationship between cognitive control and the geometry of map-like representations by combining fMRI with neural network modeling. We find that brain areas including hippocampus and entorhinal cortex spontaneously organize pairwise relationships into 2D map-like representations, and that this 2D structure was controlled by compressing task-irrelevant dimensions in areas of prefrontal and parietal cortex. Our neural network model reproduced these findings and additionally predicted warping in the geometry along a context-invariant axis. This prediction was confirmed with fMRI, which showed that the degree of warping was correlated with individual differences in cognitive control.

The Geometry of Map-Like Representations under Dynamic Cognitive Control

Distributed vector representations are a key bridging point between connectionist and symbolic representations of cognition. It is unclear how uncertainty should be modelled in systems using such representations. One may place vector-valued distributions over vector representations, although that may assign non-zero probabilities to vector symbols that cannot occur. In this paper we discuss how bundles of symbols in Vector Symbolic Architectures (VSAs) can be understood as defining an object that has a relationship to a probability distribution, and how statements in VSAs can be understood as being analogous to probabilistic statements. We sketch novel designs for networks that compute entropy and mutual information. In this paper we restrict ourselves to operators proposed for Holographic Reduced Representations, and representing real-valued data. However, we suggest that the methods presented in this paper should translate to any VSA where the dot product between fractionally bound symbols induces a valid kernel.

Fractional Binding in Vector Symbolic Architectures as Quasi-Probability Statements

Models in cognitive science are often restricted for the sake of interpretability, and as a result may miss patterns in the data that are instead classified as noise. In contrast, deep neural networks can detect almost any pattern given sufficient data, but have only recently been applied to large-scale data sets and tasks for which there already exist process-level models to compare against. Here, we train deep neural networks to predict human play in 4-in-a-row, a combinatorial game of intermediate complexity, using a data set of 10,874,547 games. We compare these networks to a planning model based on a heuristic function and tree search, and make suggestions for model improvements based on this analysis. This work provides the foundation for estimating a noise ceiling on massive data sets as well as systematically investigating the processes underlying human sequential decision-making.

Self-Explaining the Notional Machine to Improve Novice Programmers’ Learning and Mental Models, Supported by a Computer Tutor System

Downloads

Next from CogSci 2022

Eliciting Proactive and Reactive Control during Use of an Interactive Learning Environment

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES