United States

Deep convolutional neural networks (DCNNs) have been found to demonstrate hierarchical mapping to human brain regions on tasks such as object recognition. However, it remains unclear if such hierarchical mapping also applies to action recognition, which involves dynamic visual information processing. Here, we compared action representations of two-stream DCNNs to the human visual system. Five visual areas that are associated with object and action processing were selected. Nine human action categories were adopted from three semantic classes to examine the action representations of both DCNNs and human visual areas. In two fMRI experiments, actions were presented in the forms of Maya-rendered videos and point-light biological motion videos. Results showed that although two-stream DCNNs demonstrated hierarchical representations of actions as layers grow deeper, DCNNs lack a hierarchical mapping to human visual areas. Consistently across different video displays and DCNN pathways, only the top DCNN layers demonstrated high similarity to representations in the human visual system. The results suggest that the dynamic representations of human actions may be different in DCNNs compared to the human visual system, even after big-data training.

CogSci 2023

Mapping between the Human Visual System and Two-stream DCNNs in Action Representation

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

It has been argued that semantic systems reflect pressure for efficiency, and a current debate concerns the cultural evolutionary process that produces this pattern. We consider efficiency as instantiated in the Information Bottleneck (IB) principle, and a model of cultural evolution that combines iterated learning and communication. We show that this model, instantiated in neural networks, converges to color naming systems that are efficient in the IB sense and similar to human color naming systems. We also show that iterated learning alone, and communication alone, do not yield the same outcome as clearly.

Iterated learning explains aspects of color naming beyond those explained by efficiency

How do humans get better at planning? Previous work postulated that the improvement of cognitive strategies occurs through feedback-based metacognitive reinforcement learning (MCRL). However, it is not clear whether and, if so, how people can learn planning strategies without reinforcement. To answer these questions, we experimentally investigated the effect of frequency of feedback on people's ability to learn adaptive planning strategies. We found that participants receiving feedback only 25\% of the time nonetheless learned about as well as participants receiving constant feedback. Quantitative modelling of the data revealed that state-of-the-art MCRL models cannot explain this finding. However, extending these models by a mechanism generating an additional learning signal through self-evaluation of plan quality can account for people's ability to learn planning strategies without feedback. The findings of this research have implications for the design of learning environments and enabling people and machines to self-sufficiently improve their strategies in naturalistic settings.

Learning planning strategies without feedback

In psychology, simple tasks that are designed to tap a particular feature of cognition are used without considering the other mechanisms that might be at play. For example, the delayed-match-to-sample (DMTS) task is often used to examine short-term memory; however, there are a number of cognitive mechanisms interacting to produce the observed behaviour, such as decision-making and attention processes. As these simple tasks form the basis of more complex psychological experiments and theories, it is critical to understand what strategies might be producing the recorded behaviour. The current paper seeks to extend the GEMS methodology, a system that generates models of cognition using genetic programming, for differing DMTS experimental conditions. This allows us to investigate the strategies that participants might be using, while looking at differences in strategy depending on task variations. We also discuss possible weaknesses with the methodology.

Genetic Programming for Developing Simple Cognitive Models

Neural Probabilistic Learner and Sampler (NPLS) is an algorithm
that has simulated children’s non-symbolic probability
learning from visual stimuli such as collections of different
colors of marbles. Although NPLS closely simulates the
cognitive process of probability learning, the training of such
learning algorithms often uses binary encoding of inputs that
represent the perceived visual stimuli, avoiding simulation of
the visual perception of the stimuli. Here, the computer vision
technique You Only Look Once (YOLO) (Jocher et al.,
2021; Redmon et al., 2016), is integrated into the workflow
of an NPLS simulation probability learning experiments with
children. YOLO is a convolutional neural network (CNN) designed
to detect objects. The model’s performance on marble
datasets is tested through an analysis of precision and recall.
Results indicate that the YOLO model, when trained sufficiently,
outputs predictions on marble image datasets with high
accuracy and precision. We also analyze YOLO’s suitability as
a biologically plausible model of visual processing, interfering
with YOLO’s training process by shortening the training time
to examine the effects of perceptual errors on simulated probabilistic
reasoning.


A Perceptual Front-End for Probability Learning: Object Detection with YOLO

Autonomy support is an interpersonal tone of support and understanding. Teachers' instructional behaviors with autonomy support facilitate students' inner motivational resources and enhance their engagement in learning activities. Identifying these behaviors is vital in terms of teaching effects on students' academic performance and behaviors. However, existing methods heavily rely on psychological questionnaires or observers, which are time consuming and difficult to apply in large scale teaching scenarios. In this work, we combine deep neural networks with psychology to achieve automatically identifying the behaviors. To this end, we create the first Chinese instructional behavior dataset for deep learning, and then construct a BERT-based network for modeling psychological features and identifying behaviors. Since frequencies of different behaviors are highly imbalanced, we further design a novel loss function to adaptively optimize training weights for each behavior. Extensive experiments verify the effectiveness of our method and explain how the behaviors are identified.

Automatically Identifying Teachers' Autonomy Support using Text Classification on Imbalanced Data

We propose the predictive forward-forward (PFF) algorithm for conducting credit assignment in neural systems. Specifically, we design a novel, dynamic recurrent neural system that learns a directed generative circuit jointly and simultaneously with a representation circuit, integrating learnable lateral competition and elements of predictive coding, an emerging and viable neurobiological process theory of cortical function, with the forward-forward (FF) adaptation scheme. Furthermore, PFF efficiently learns to propagate learning signals and updates synapses with forward passes only, eliminating key structural and computational constraints imposed by a backprop-based scheme. Besides computational advantages, the PFF process could prove useful for understanding the learning mechanisms behind biological neurons that use local signals despite missing feedback connections. We run experiments on image data and demonstrate that the PFF procedure works as well as backprop, offering a promising brain-inspired learning algorithm for classifying, reconstructing, and synthesizing data patterns.

The Predictive Forward-Forward Algorithm

Sentiment and emotion embody intrinsic experience of human cognition, both of which have the profoundly influence on the cognitive processes in humans. The commonality between sentiment and emotion leads to the fact that sentiment classification and emotion recognition are two correlated tasks in cognitive science and artifical intellignce (AI). One task improves its performance by leveraging similarities and differences across tasks. However, recent works have often treated them as two separately different tasks, where their commonalities have largely been neglected. To fill this gap, we propose a multi-task interactive graph attention network with position encodings, termed MIP-GAT. The main proposal is a multi-interactive graph interaction layer where a syntactic dependency connection, a cross-task connection and position encodings are constructed and incorporated into a unified graphical structure. Empirical evaluation on two benchmarking datasets, i.e., CMU-MOSEI and GoEmotions, shows the effectiveness of the proposed model over state-of-the-art baselines. In addition, we also explore the superiority and limitations of the proposed model.

MIP-GAT: A Multi-Task Interactive Graph Attention Network with Position Encodings for Joint Sentiment Classification and Emotion Recognition

Humans can successfully interpret images even when they have been distorted by significant image transformations. Such images aid in differentiating existing computational models for perception because models which predict similar results for typical stimuli that may diverge when confronted with atypical stimuli. In this work, we propose an explanation of people's ability to perceive a specific class of degraded stimuli that requires extreme generalization capabilities: Mooney, or two-tone, faces. We provide a parametric and controllable method to generate Mooney faces and a hypothesis about how the human visual system might be able to perceive such extreme precepts. We show that Mooney faces can be approximated under extreme illumination conditions; a computer graphics perspective on the Mooney image formation process suggests inverse graphics as a model for perceptual generalization and possibly everyday perception. With various visual examples, we argue that Mooney faces are important in the understanding the role of generative models and inverse graphics in human visual processing.

Perception of Mooney Faces: Extreme Generalization through Inverse Rendering?

Cross-lingual adaptation has shown effectiveness in low-resource spoken language understanding (SLU) systems.
However, existing approaches often show unsatisfactory performance on intent detection and slot filling, especially for distant languages,
which are dramatically different from source language in scripts, morphology, or syntax.
To solve the aforementioned challenges, we propose Latent Dialogue Action (LaDA) layer to 
optimize decoding strategy.
The model consists of an extra latent dialogue action layer. It enables our model to enhance a system’s ability to
handle conversations in complex multi-lingual intent and slot value of distant languages.
To our best knowledge, our work is the first
comprehensive study of the use of latent variables for cross-lingual SLU
policy optimization during decode stage. LaDA
achieves state-of-the-art results in zero-shot and few-shot adaptation both on public datasets.

LaDA: Latent Dialogue Action For Zero-shot Cross-lingual Neural Network Language Modeling

In this work, we propose an interpretable knowledge tracking model (KVFKT) based on key value and forgetting. This model maintains the students’ knowledge state matrix in each memory unit, and also maintains the corresponding forgetting matrix to track the last review timestamp of students’ knowledge points. In addition, this model introduces the IRT two parameter model from the perspective of educational psychology to further enhance the interpretability of the model. Finally, the model is tested on four real-world datasets, and the experimental results prove the advantages of the method described in this paper.

Premium content

Mapping between the Human Visual System and Two-stream DCNNs in Action Representation

Next from CogSci 2023

Iterated learning explains aspects of color naming beyond those explained by efficiency

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES