United States

In this paper we study human sequential behavior by integrating cognitive, evolutionary, and computational approaches. Our work centers around the emergence of shared vocabularies in the Embodied Communication Game (ECG). In this game, participant pairs solve a shared task without access to conventional means of communication, enforcing the emergence of a new communication system. This problem is solved typically by negotiating a shared set of sequential signals that acquire meaning through interactions. Individual differences in Personal Need for Structure (PNS) have been found to influence how this process develops. We trained deep learning models to mimic the emergence of these systems and used hyperparameter optimization to approximate latent human cognitive variables to explain human behavior. We demonstrate that models based on bidirectional LSTM networks are better at capturing human behavior than unidirectional LSTM networks. This suggests that human sequence processing in the ECG is influenced by expected future states. The approximated variables cannot explain the effects of PNS, but we do provide evidence suggesting that random and uncertainty-directed exploration strategies are combined to develop optimal behavior.

CogSci 2023

Modeling Human Sequential Behavior with Deep Neural Networks in Emergent Communication

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

There is an apparent discrepancy between visual perception, which is colorful, complete, and in high resolution, and the saccadic, and spatially heterogeneous retinal input data. In this work, we computationally emulated foveated color maps and intensity channels as well as intra-saccadic motion data using a neuromorphic event camera. We used a convolutional neural network, U-Net, and adversarial optimization to demonstrate how retinal inputs can be used for the reconstruction of colorful images in high resolution. Our model may set the groundwork toward the development of biologically plausible neural networks for computational vision perception.

Perceptual colorization of the peripheral retinotopic visual field using adversarially-optimized neural networks

Deep convolutional neural networks (DCNNs) have been found to demonstrate hierarchical mapping to human brain regions on tasks such as object recognition. However, it remains unclear if such hierarchical mapping also applies to action recognition, which involves dynamic visual information processing. Here, we compared action representations of two-stream DCNNs to the human visual system. Five visual areas that are associated with object and action processing were selected. Nine human action categories were adopted from three semantic classes to examine the action representations of both DCNNs and human visual areas. In two fMRI experiments, actions were presented in the forms of Maya-rendered videos and point-light biological motion videos. Results showed that although two-stream DCNNs demonstrated hierarchical representations of actions as layers grow deeper, DCNNs lack a hierarchical mapping to human visual areas. Consistently across different video displays and DCNN pathways, only the top DCNN layers demonstrated high similarity to representations in the human visual system. The results suggest that the dynamic representations of human actions may be different in DCNNs compared to the human visual system, even after big-data training.

Mapping between the Human Visual System and Two-stream DCNNs in Action Representation

It has been argued that semantic systems reflect pressure for efficiency, and a current debate concerns the cultural evolutionary process that produces this pattern. We consider efficiency as instantiated in the Information Bottleneck (IB) principle, and a model of cultural evolution that combines iterated learning and communication. We show that this model, instantiated in neural networks, converges to color naming systems that are efficient in the IB sense and similar to human color naming systems. We also show that iterated learning alone, and communication alone, do not yield the same outcome as clearly.

Iterated learning explains aspects of color naming beyond those explained by efficiency

How do humans get better at planning? Previous work postulated that the improvement of cognitive strategies occurs through feedback-based metacognitive reinforcement learning (MCRL). However, it is not clear whether and, if so, how people can learn planning strategies without reinforcement. To answer these questions, we experimentally investigated the effect of frequency of feedback on people's ability to learn adaptive planning strategies. We found that participants receiving feedback only 25\% of the time nonetheless learned about as well as participants receiving constant feedback. Quantitative modelling of the data revealed that state-of-the-art MCRL models cannot explain this finding. However, extending these models by a mechanism generating an additional learning signal through self-evaluation of plan quality can account for people's ability to learn planning strategies without feedback. The findings of this research have implications for the design of learning environments and enabling people and machines to self-sufficiently improve their strategies in naturalistic settings.

Learning planning strategies without feedback

In psychology, simple tasks that are designed to tap a particular feature of cognition are used without considering the other mechanisms that might be at play. For example, the delayed-match-to-sample (DMTS) task is often used to examine short-term memory; however, there are a number of cognitive mechanisms interacting to produce the observed behaviour, such as decision-making and attention processes. As these simple tasks form the basis of more complex psychological experiments and theories, it is critical to understand what strategies might be producing the recorded behaviour. The current paper seeks to extend the GEMS methodology, a system that generates models of cognition using genetic programming, for differing DMTS experimental conditions. This allows us to investigate the strategies that participants might be using, while looking at differences in strategy depending on task variations. We also discuss possible weaknesses with the methodology.

Genetic Programming for Developing Simple Cognitive Models

Neural Probabilistic Learner and Sampler (NPLS) is an algorithm
that has simulated children’s non-symbolic probability
learning from visual stimuli such as collections of different
colors of marbles. Although NPLS closely simulates the
cognitive process of probability learning, the training of such
learning algorithms often uses binary encoding of inputs that
represent the perceived visual stimuli, avoiding simulation of
the visual perception of the stimuli. Here, the computer vision
technique You Only Look Once (YOLO) (Jocher et al.,
2021; Redmon et al., 2016), is integrated into the workflow
of an NPLS simulation probability learning experiments with
children. YOLO is a convolutional neural network (CNN) designed
to detect objects. The model’s performance on marble
datasets is tested through an analysis of precision and recall.
Results indicate that the YOLO model, when trained sufficiently,
outputs predictions on marble image datasets with high
accuracy and precision. We also analyze YOLO’s suitability as
a biologically plausible model of visual processing, interfering
with YOLO’s training process by shortening the training time
to examine the effects of perceptual errors on simulated probabilistic
reasoning.


A Perceptual Front-End for Probability Learning: Object Detection with YOLO

Autonomy support is an interpersonal tone of support and understanding. Teachers' instructional behaviors with autonomy support facilitate students' inner motivational resources and enhance their engagement in learning activities. Identifying these behaviors is vital in terms of teaching effects on students' academic performance and behaviors. However, existing methods heavily rely on psychological questionnaires or observers, which are time consuming and difficult to apply in large scale teaching scenarios. In this work, we combine deep neural networks with psychology to achieve automatically identifying the behaviors. To this end, we create the first Chinese instructional behavior dataset for deep learning, and then construct a BERT-based network for modeling psychological features and identifying behaviors. Since frequencies of different behaviors are highly imbalanced, we further design a novel loss function to adaptively optimize training weights for each behavior. Extensive experiments verify the effectiveness of our method and explain how the behaviors are identified.

Automatically Identifying Teachers' Autonomy Support using Text Classification on Imbalanced Data

We propose the predictive forward-forward (PFF) algorithm for conducting credit assignment in neural systems. Specifically, we design a novel, dynamic recurrent neural system that learns a directed generative circuit jointly and simultaneously with a representation circuit, integrating learnable lateral competition and elements of predictive coding, an emerging and viable neurobiological process theory of cortical function, with the forward-forward (FF) adaptation scheme. Furthermore, PFF efficiently learns to propagate learning signals and updates synapses with forward passes only, eliminating key structural and computational constraints imposed by a backprop-based scheme. Besides computational advantages, the PFF process could prove useful for understanding the learning mechanisms behind biological neurons that use local signals despite missing feedback connections. We run experiments on image data and demonstrate that the PFF procedure works as well as backprop, offering a promising brain-inspired learning algorithm for classifying, reconstructing, and synthesizing data patterns.

The Predictive Forward-Forward Algorithm

Sentiment and emotion embody intrinsic experience of human cognition, both of which have the profoundly influence on the cognitive processes in humans. The commonality between sentiment and emotion leads to the fact that sentiment classification and emotion recognition are two correlated tasks in cognitive science and artifical intellignce (AI). One task improves its performance by leveraging similarities and differences across tasks. However, recent works have often treated them as two separately different tasks, where their commonalities have largely been neglected. To fill this gap, we propose a multi-task interactive graph attention network with position encodings, termed MIP-GAT. The main proposal is a multi-interactive graph interaction layer where a syntactic dependency connection, a cross-task connection and position encodings are constructed and incorporated into a unified graphical structure. Empirical evaluation on two benchmarking datasets, i.e., CMU-MOSEI and GoEmotions, shows the effectiveness of the proposed model over state-of-the-art baselines. In addition, we also explore the superiority and limitations of the proposed model.

MIP-GAT: A Multi-Task Interactive Graph Attention Network with Position Encodings for Joint Sentiment Classification and Emotion Recognition

Humans can successfully interpret images even when they have been distorted by significant image transformations. Such images aid in differentiating existing computational models for perception because models which predict similar results for typical stimuli that may diverge when confronted with atypical stimuli. In this work, we propose an explanation of people's ability to perceive a specific class of degraded stimuli that requires extreme generalization capabilities: Mooney, or two-tone, faces. We provide a parametric and controllable method to generate Mooney faces and a hypothesis about how the human visual system might be able to perceive such extreme precepts. We show that Mooney faces can be approximated under extreme illumination conditions; a computer graphics perspective on the Mooney image formation process suggests inverse graphics as a model for perceptual generalization and possibly everyday perception. With various visual examples, we argue that Mooney faces are important in the understanding the role of generative models and inverse graphics in human visual processing.

Premium content

Modeling Human Sequential Behavior with Deep Neural Networks in Emergent Communication

Next from CogSci 2023

Perceptual colorization of the peripheral retinotopic visual field using adversarially-optimized neural networks

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES