United States

The study of emergent languages in deep multi-agent simulations
has become an important research field. While targeting
different objectives, most studies focus on analyzing properties
of the emergent language—often in relation to the agents’
inputs—ignoring the influence of the agents’ perceptual processes.
In this work, we use communication games to investigate
how differences in perception affect emergent language.
Using a conventional setup, we train two deep reinforcement
learning agents, a sender and a receiver, on a reference game.
However, we systematically manipulate the agents’ perception
by enforcing similar representations for objects with specific
shared features. We find that perceptual biases of both sender
and receiver influence which object features the agents’ messages
are grounded in. When uniformly enforcing the similarity
of all features that are relevant for the reference game,
agents perform better and the emergent protocol better captures
conceptual input properties.

CogSci 2021

Why and how to study the impact of perception on language emergence in artificial agents

<center>Louis Stokes Regional Center of Excellence for the Study of STEM Interventions<center>

<CENTER>Translating Critical, Strengths-Based Conceptual Frameworks to STEM Programming Design & Dissemination<CENTER>


 **Sponsored in Partnership by:**
 National Science Foundation, Understanding Interventions, University of Michigan and University of Missouri
 
 
 		
	
Dear Colleagues,

Welcome to the inaugural Louis Stokes Regional Center of Excellence for the Study of STEM Interventions ‎‎(LSCRE ‎‎‎‎SOSI) Summer Institute. SOSI is an NSF funded initiative that reflects a collaboration of STEM and STEM Education Scholars ‎and Program Administrators from the University of Missouri-Columbia, Understanding ‎Interventions, Inc., and the University of Michigan.‎ We offer training and development for STEM program administrators, faculty, and other ‎stakeholders ‎in critical, strengths-based theories and evidence-based practices. Key to our work is providing the tools and avenues for our participants to inform their practice and contribute to the body of translational research that facilitate ‎underrepresented students’ success in STEM. Beyond the Summer Institutes, SOSI, through the Understanding Interventions Index, supports resources for access to the STEM research-to-practice translational literature, a database of journals that publish these works, relevant published measures, and the Understanding Interventions Journal as an outlet for dissemination. This work also extends to webinars and workshops held throughout the year as an integral part of the collaboration between SOSI and its partners. 

This 2021 Summer Institute is designed to address participant needs across a range of experience in our concurrent sessions. Concurrent Sessions 1 are tailored for those of us who have professional training outside the social and behavioral sciences but are looking to develop proficiency, scholarship, and utilize methodologies that test theories like Community Cultural Wealth (CCW), Phenomenological Variant of Ecological Systems Theory (PVEST) and Social Cognitive Career Theory (SCCT). Concurrent Sessions 2 are tailored to those with some experience in these areas and are looking to engage more deeply. Some sessions like those that address measures, implementation, and outcomes will be joint. For the current topic sessions, participants will not be limited a specific concurrent session but will have the flexibility to attend the session that they feel most appropriate to their level of experience. 

We look forward to an exciting and productive institute as we all leverage our varied experiences, perspectives, and ideas to this growing community of practice. We would also like to extend our deep gratitude to the National Science Foundation for its generous support of this work.


Best,

SOSI Leadership Team

SOSI Virtual Summer Institute 2021

poster

### Looking for Cogsci 2023? Please [click here](https://underline.io/events/405/reception) to switch to 2023 edition.


**Dear Colleagues,**

Welcome to the 43rd Annual Conference of the Cognitive Science Society! 

This year’s conference brings together the latest research from all Cognitive Science and also highlights this year’s theme **Comparative Cognition: Animal Minds.** 

The program features four plenary speakers: Friederike Range (University of Veterinary Medicine, Vienna), Onur Güntürkün (Ruhr University Bochum), Julia Fischer (University of Göttingen), and Erich Jarvis (The Rockefeller University). The keynotes and invited symposia showcase the broad spectrum of ideas that encompass the conference theme of comparative cognition. 

Further, the program includes the Rumelhart Prize keynote presentation by Susan Goldin-Meadow (University of Chicago), and symposia to celebrate the winner of the Elman Prize: Wei Ji Ma (New York University), the Heineken Award winner Robert Zatorre (McGill University), and the Robert J. Glushko Dissertation Prizes session.

The program committee for CogSci 2021 received 1,025 submissions, including 710 full papers, 275 member abstracts, 8 publication-based talks, as well as 15 proposals for symposia, and 17 for workshops and tutorials. After a rigorous review process, the committee selected 152 papers for oral presentation and inclusion in the conference proceedings (14.8%), 332 papers for poster presentation and inclusion in the proceedings (32.4%), and 175 papers for poster presentation with inclusion of abstracts in the proceedings (17.1%) We also selected 275 submitted member abstracts. In addition, we accepted 6 publication-based talks, 12 symposia, and 9 workshops and tutorials to make for an exciting and inclusive program.

We hope that you enjoy the conference this year! 

**Your Program Chairs,**
![](https://assets.underline.io/uploads/markdown_image/1/image/99984908caba5e6453e5fc5ba34f3507.jpg)

**Learn more about the Vienna Cognitive Science Hub at the University of Vienna, hosts of CogSci 2021**

<iframe width="560" height="315" src="https://www.youtube.com/embed/464h1m3iU68" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

To review the proceedings for the conference or review the searchable program, please visit the [website](https://cognitivesciencesociety.org/program/).

The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education. The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

Due to information processing constraints and cognitive limitations, humans must form limited representations of complex decision making tasks. However, the mechanisms by which humans generate representations of task-relevant stimulus remain unclear. We develop a model that seeks to account for the formation of these representations using a β-variational autoencoder (β-VAE) trained with a novel utility based learning objective. The proposed model forms latent representations of decision making tasks that are constrained in their information complexity. We show through simulation that this approach can account for important phenomena in human economic decision making tasks. This model provides a method of forming task-relevant representations that can be used to make decisions in a human-like manner.

Modeling Capacity-Limited Decision Making Using a Variational Autoencoder

We model the sense-making process of diagrams as conceptual blends of the diagrams’ geometric configurations with apt image schemas.We specify image schemas and geometric configurations with typed FOL theories. In addition, for the latter, we utilise some Qualitative Spatial Reasoning formalisms. Using an algebraic specification language, we can compute the conceptual blends of image schemas and geometry as category-theoretic colimits.We show through several examples how this model captures the sort of direct inferences we confer to diagrammatic representations due to our embodied cognition. We argue that this approach to sense-making might be of value for the design and application of diagrammatic and graphical visualisations, as well as for AI in general.

Modelling the Sense-Making of Diagrams Using Image Schemas

Human learning transfer takes advantage of important cognitive building blocks such as an abstract representation of concepts underlying tasks and causal models of the environment. One way to build abstract representations of the environment when the task involves interactions with others is to build a model of the opponent that may inform what actions they are likely to take next. In this study, we explore opponent modelling and its role in learning transfer by letting human participants play different games against the same computer agent, who possesses human-like theory of mind abilities with a limited degree of iterated reasoning. We find that participants deviate from Nash equilibrium play and learn to adapt to the opponent's strategy to exploit it. Moreover, we show that participants transfer their learning to new games and that this transfer is moderated by the level of sophistication of the opponent. Computational modelling shows that it is likely that players start each game using a model-based learning strategy that facilitates generalisation and opponent model transfer, but then switch to behaviour that is consistent with a model-free learning strategy in the later stages of the interaction.

Transfer of learned opponent models in repeated games

Most theoretical accounts of non-instrumental information seeking suggest that the magnitude of rewards has a direct influence on the attractiveness of the information. Specifically, the magnitude of rewards is assumed to be proportional to the strength of information seeking (or avoidant) behaviour. In a series of experiments using numerical and pictorial stimuli, we explore the extent to which observed information seeking behaviour tracks these predictions. Our findings indicate a robust independence of information seeking from outcome magnitude and valence with preferences for information largely remaining constant across different reward valence and magnitudes. We discuss these results in the context of current computational models with suggestions for future theoretical and empirical work.

Invariance of Information Seeking Across Reward Magnitudes

To glean information from social networks, people must be adept at distinguishing real evidence from hearsay. Here, we investigate human inferences within a simulated social network. We introduce a novel social learning task in which participants infer their surrounding social network structure while also using social communications to aid judgments about the shared environment. We find that the majority of adults correctly identified independent and acyclic network structures, though they struggled to identify cyclic communications. Comparison of judgments with several social inference models supports a naïve social learning account that integrates social evidence with the learner’s own observations in a structure insensitive way. This suggests that people are capable of using communication patterns to identify social influences, but they may still be misled by the distortions of evidence that these network dynamics can produce.

Know your network: Sensitivity to structure in social learning

One of the consequences of the pandemic is that throughout 2020 virtual interactions largely replaced face-to-face interactions. Though there are few studies of how social media impacts body image perception across genders, research suggests that socializing through a virtual self-body image might have distinct implications for men and women. On an online study, we examined whether type of social pressure and body-ideal exert distinct pressures on members of the X, Y, and Z generations. Results showed media pressure affected body image satisfaction significantly more than other kinds of social pressure across genders and generations, with young males reporting a higher impact compared to older males. Males experienced more pressure to be muscular and women to be thin, especially for the younger generation.Future research should focus on social media as a potential intervention tool for the detection and prevention of body image disorders in both young female and male adults.

Body Image During Quarantine; Generational Effects of Social Media Pressure on Body Appearance Perception

It is an ancient insight that human actions are driven by desires. Yet it misses one mental representation, intention, with which agents regulate conflicting desires by committing to an admissible plan. Here we demonstrate four behavioral signatures of intention only observed in humans: disruption resistance as sticking with a plan despite setbacks; exclusiveness as avoiding paths with temptations of re-planning; deliberation as the gradual emergence of a commitment plan; temporal leap as forming future plans before finishing the current one. Humans were compared against an optimal model formulated as Markov Decision Process (MDP), who acts only to maximize expected future rewards. Conflicting desires are defined as a reward function returning positive rewards for multiple states. It showed none of the behavioral signatures of intention. These results reveal that humans regulate conflicting desires with intentions, which directly drive actions.

Intention beyond Desire: Humans Spontaneously Commit to Future Actions

Deep Neural Network representations correlate very well with neural responses measured in primates' brains and with psychological representations of human similarity judgement tasks, making them possible models for human behavior-related tasks. This study investigates whether DNNs can learn an implicit association (between colors and emotions) for images. An experiment was conducted in which subjects were asked to select a color for a given emotion-inducing image. These human responses (decision probabilities) were modeled on neural networks using representations extracted from pre-trained DNNs for the images and colors (a square of the color). The model presented showed a fuzzy linear relationship with the decision probabilities. Finally, this model was presented as a model for emotion classification tasks, specifically with very few training examples, showing an improvement in accuracy from a standard classification model. This analysis can be of relevance to psychologists studying these associations and AI researchers modelling emotional intelligence in machines.

Emotion-Color Association in Biologically Inspired Deep Neural Networks

A critical aspect of the way humans recognize and understand meaning in sequential data is the ability to identify abstract structural repetitions. We present a novel approach to discovering structural repetitions within sequences that uses a multiple Smith-Waterman self-alignment. We illustrate our approach in the context of finding different forms of structural repetition in music composition. Feature-specific alignment scoring functions enable structure finding in primitive features such as rhythm, melody, and lyrics. These can be compounded to create scoring functions that find higher-level structure including verse-chorus structure. We demonstrate our approach by finding harmonic, pitch, rhythmic, and lyrical structure in symbolic music and compounding these viewpoints to identify the abstract structure of verse-chorus segmentation.

Inferring Structural Constraints in Musical Sequences via Multiple Self-Alignment

It is a long standing challenge to devise a formal model of ACT-R as a basis for formal reasoning on ACT-R. The ACT-R architecture is a composition of components (such as modules) with predefined interfaces between components and predefined interactions on the interfaces. Reasoning over the correctness of a formal model of ACT-R benefits from the separation of abstraction levels i.e. reasoning on the level of interfaces and interactions between components in isolation from the concrete behaviour of each component. 
We propose a formal semantics of ACT-R that preserves the structural properties of architectural components, i.e., the interfaces of modules to the remaining architecture as well as communication between modules within the architecture. We demonstrate how our new formal semantics of ACT-R serves to prove the correctness of the timed automaton based operational semantics for ACT-R (TA-ACT-R) on the level of architectural components.

Why and how to study the impact of perception on language emergence in artificial agents

Downloads

Next from CogSci 2021

Modeling Capacity-Limited Decision Making Using a Variational Autoencoder

Similar lecture

A Dive into the Ins and Outs of PVEST and its Application for STEM Programming Research and Design

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES