United States

To develop computational agents that better communicate using their own emergent language, we endow the agents with an ability to focus their attention on particular concepts in the environment.
Humans often understand a thing or scene as a composite of concepts and those concepts are further mapped onto words.
We implement this intuition as attention mechanisms in Speaker and Listener agents in a referential game and show attention leads to more compositional and interpretable emergent language.
We also demonstrate how attention helps us understand the learned communication protocol by investigating the attention weights associated with each message symbol and the alignment of attention weights between Speaker and Listener agents.
Overall, our results suggest that attention is a promising mechanism for developing more human-like emergent language.

CogSci 2023

Emergent Communication with Attention

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Ontology is the basis of knowledge representation, and it is necessary to translate ontologies that are normally expressed in English into other languages in order to achieve exchange across languages. Building a domain-specific translation system is essential due to the extremely focused words used and the inadequacy of contextual information. In this paper, we introduce disentangled representations under cross-lingual agreement to alleviate the aforementioned issues. We introduce semantic and language representations and integrate extra losses to induce disentangled representations that capture different information. To reduce the gap between the ontology label and the hypothesis generated by the translation model, we further integrate adversarial learning. In order to guide the generation of translation candidates, the semantic matching strategy is incorporated into the decoding phase. Experiments on the four English-to-German ontologies of different domains show that the proposed method achieves improvements over the baselines.


Improving Ontology Translation from Disentangled Semantic and Language Representations

Graph neural networks (GNNs) have become a \emph{de facto} paradigm for graph representation learning. Generally, GNNs are trained in an end-to-end manner with supervision, requiring considerable task-specific labeled data. To reduce the labeling burden, recent works leverage self-supervised tasks to pre-train an expressive GNN model on abundant unlabeled data and finetune the trained model on downstream datasets with only a few labels. However, existing GNN pre-training approaches only concentrate on a single view for graph self-supervised learning while ignoring the rich semantic information in graphs, leading to the lack of sample utilization efficiency during the pre-training process. To tackle such challenges, we propose a multi-view graph contrastive encoding for graphs during GNN pre-training, called MVRACE. The critical insight is that we construct node and graph-level views to capture local attribute information and global structure in a graph. Concretely, the node-level view utilizes graph centrality and encodes the $r$-ego network to capture the local-whole relationship in a graph. The graph-level view aims to encode graph pairs to explore different graph structures and empower the discrimination ability of the GNN encoder. In addition, we combine multi-views with a joint contrastive loss function to integrate node- and graph-semantic information simultaneously. Comprehensive experiments on multiple domain datasets demonstrate that our approach can significantly yield competitive performance compared to state-of-the-art methods.


MVRACE: Multi-view Graph Contrastive Encoding for Graph Neural Network Pre-training

Infants explore their complex physical and social environment in an organized way. To gain insight into what intrinsic motivations may help structure this exploration, we create a virtual infant agent and place it in a developmentally-inspired 3D environment with no external rewards. The environment has a virtual caregiver agent with the capability to interact contingently with the infant agent in ways that resemble play. We test intrinsic reward functions that are similar to motivations that have been proposed to drive exploration in humans: surprise, uncertainty, novelty, and learning progress. These generic reward functions lead the infant agent to explore its environment and discover the contingencies that are embedded into the caregiver agent. The reward functions that are proxies for novelty and uncertainty are the most successful in generating diverse experiences and activating the environment contingencies. We also find that learning a world model in the presence of an attentive caregiver helps the infant agent learn how to predict scenarios with challenging social and physical dynamics. Taken together, our findings provide insight into how curiosity-like intrinsic rewards and contingent social interaction lead to dynamic social behavior and the creation of a robust predictive world model.

Developmental Curiosity and Social Interaction in Virtual Agents

Predictability is a prerequisite for effective human control of artificial intelligence (AI). The inability to predict malfunctioning of AI, for example, impedes timely human intervention. In this paper, we empirically investigate how AI’s predictability compares to the predictability of humans in a real-effort task. We show that humans are worse at predicting AI performance than at predicting human performance. Importantly, participants are not aware of the differences in relative predictability of AI and overestimate their prediction skills. These results raise doubts about the human ability to effectively exercise control of AI — at least in certain contexts.

Human control redressed: Comparing AI-to-human vs. human-to-human predictability in a real-effort task

Large language models have been shown useful in multiple
domains including conversational agents, education, and explainable AI. ChatGPT is a large language model developed
by OpenAI as a conversational agent. ChatGPT was trained on
data generated by humans and by receiving human feedback.
This training process results in a bias toward humans’ traits
and preferences. In this paper, we stress multiple biases of
ChatGPT, and show that its responses are closer to human responses than to fully rational responses. We Begin by showing
a very high correlation between the frequency of digits generated by ChatGPT and humans’ favorite numbers, with the
most frequent digit generated by ChatGPT, matching humans'
most favorable number, 7. We continue by showing that ChatGPT’s responses in several social experiments are much closer
to those of humans’ than to those of fully rational agents. Finally, we show that several cognitive biases, known in humans,
are also present in ChatGPT’s responses.



ChatGPT: More Human-Like Than Computer-Like, but Not Necessarily in a Good Way

Lexical Decision Task is one of the most widely used experimental paradigms for studying word recognition that allows researchers to make inferences about lexical processing and lexical representations of words and non-words. Evidence Accumulation Models (EAM) have been successfully used to model this task. Despite the accurate prediction of participants’ reaction time and accuracy, these models lack a mechanism for representing the lexical features of words and non-words. Incorporating lexical features directly into an EAM can open up new and better ways to study lexical processing and lexical representations. For this purpose, we developed two models by combining FastText and BERT models, with the race-diffusion model. In this framework representations of words and non-words are generated by FastText or BERT models and transformed into the race-diffusion model’s drift rate. Results show that a combination of FastText and race diffusion model is a promising approach for modeling the lexical decision task.


Modeling Human Lexical Decision-Making with Artificial Neural Networks and Evidence Accumulation Models

Recent advances in large language models (LLMs) have shown that it is possible to derive expressive meaning representations by leveraging linguistic associations in large-scale natural language data. These potentially Gestalt representations have enabled state-of-the-art performance for many practical applications. It would appear that we are on a pathway to empirically deriving a robust and expressive computable semantics. A key question that arises is how far can language data alone enable computers to understand the necessary truth about the physical world? Attention to this question is warranted because our future interactions with intelligent machines depends on how well our techniques correctly represent and process the concepts (objects, properties, and processes) that humans commonly observe to be true. After reviewing existing protocols, we explore this question using a novel and tightly controlled reasoning test and compare human norms against the LLMs. Our findings highlight the categories of common sense relations models could learn directly from data and areas of weakness which require moving beyond associative learning.


On Reality and the Limits of Language Data: Aligning LLMs with Human Norms

Widely considered a cornerstone of human morality, trust shapes many aspects of human social interactions. In this work, we show that artificial intelligence (AI) agents learn to trust ― and rationally adapt the extent to which they trust ― by interacting with their counterparts in the trust game (Berg, Dickhaut, & McCabe, 1995), the canonical task for studying trust in behavioral and brain sciences. Leveraging reinforcement learning (RL) to train our AI agents, we systematically investigate learning trust under various parameterizations of this task and show that trust can arise from pure self-interest. Our work sheds new light on the computational underpinnings of trust, taking us a step closer to developing AI agents whose values are aligned with human values.


AI Agents Learn to Trust

Conversational AI devices are increasingly present in our daily lives and used by both adults and children to ask questions, play and learn. However, their appearance oddly blurs the line between agents and objects—they both look and behave like inanimate objects yet engage in polite helpful conversations—raising questions about how humans conceptualize such novel entities. In this work we explore adult and children's attributions of the AI's "mental states" through a variant of a false belief task. We establish that, while adults successfully differentiate the individual minds of people from the shared "minds" of conversational AI agents, children through age 8 reason about AI agent minds much as they do human minds: as separate entities that do not share information. These findings suggest conversational AI agents are rather unintuitive to young children, (i.e. understanding develops sometime after age 8) and we discuss implications of this for childhood AI education.


Theory of AI Minds: Children's reasoning about the "mental states" of conversational AI agents

Explicit similarity judgments tend to emphasize relational information more than do difference judgments. We propose and test the hypothesis that this asymmetry arises because human reasoners represent the relation different as the negation of the relation same, so that processing difference is more cognitively demanding than processing similarity. For both verbal comparisons between word pairs, and visual comparisons between sets of geometric shapes, we asked participants to select which of two options was either more similar to or more different from a standard. On unambiguous trials, one option was unambiguously more similar to the standard; on ambiguous trials, one option was more featurally similar to the standard, whereas the other was more relationally similar. Given the higher cognitive complexity of assessing relational similarity, we predicted that detecting relational difference would be particularly demanding. We found that participants (1) had more difficulty accurately detecting relational difference than they did relational similarity on unambiguous trials, and (2) tended to emphasize relational information more when judging similarity than when judging difference on ambiguous trials. The latter finding was captured by a computational model of comparison that weights relational information more heavily for similarity than for difference judgments. Our results provide convergent evidence for a representational asymmetry between the relations same and different.


Premium content

Emergent Communication with Attention

Next from CogSci 2023

Improving Ontology Translation from Disentangled Semantic and Language Representations

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES