Canada

Speakers&#39; referential expressions often depart from communicative ideals in ways that help illuminate the nature of pragmatic language use. Patterns of overmodification, in which a speaker uses a modifier that is redundant given their communicative goal, have proven especially informative in this regard. It seems likely that these patterns are shaped by the environment a speaker is exposed to in complex ways. Unfortunately, systematically manipulating these factors during human language acquisition is impossible. In this paper, we propose to address this limitation by adopting neural networks (NN) as learning agents. By systematically varying the environments in which these agents are trained, while keeping the NN architecture constant, we show that overmodification is more likely with environmental features that are infrequent or salient. We show that these findings emerge naturally in the context of a probabilistic model of pragmatic communication.

CogSci 2022

Color Overmodification Emerges from Data-Driven Learning and Pragmatic Reasoning

Speakers' referential expressions often depart from communicative ideals in ways that help illuminate the nature of pragmatic language use. Patterns of overmodification, in which a speaker uses a modifier that is redundant given their communicative goal, have proven especially informative in this regard. It seems likely that these patterns are shaped by the environment a speaker is exposed to in complex ways. Unfortunately, systematically manipulating these factors during human language acquisition is impossible. In this paper, we propose to address this limitation by adopting neural networks (NN) as learning agents. By systematically varying the environments in which these agents are trained, while keeping the NN architecture constant, we show that overmodification is more likely with environmental features that are infrequent or salient. We show that these findings emerge naturally in the context of a probabilistic model of pragmatic communication.

poster

Join us for CogSci2022! Our 44th Annual Conference will be centered around the theme of diversity, with an accompanying range of keynotes and invited speakers who themselves highlight the diversity of our field. Our three symposia are Cognition across Diverse Populations, Cognition within Diverse Societies, and Global Challenges for Cognitive Science. The conference is being offered in a fully hybrid environment for onsite and online attendees.
 
 [**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/07/Cogsci-2022-PAAG-Detailed-July-12final-1.pdf)
 [**The Detailed Program can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/08/DeArmond-CogSci2022-program-v113-final.pdf)

This conference requires a registration to attend. Please use the link below to secure your ticket. The website will be available to all registered attendees on July 25th. 

[Register to Attend](https://cognitivesciencesociety.org/registration/)

Registration Is Required

The 44th Annual Meeting of the Cognitive Science Society highlighted research on the theme Cognitive Diversity in addition to the full breadth and diversity of research topics offered by the society’s membership. 





In this work we introduce a structured signaling game, an extension of the classical signaling game with a similarity structure between meanings in the context, along with a variant of the Rational Speech Act (RSA) framework which we call structured-RSA (sRSA) for pragmatic reasoning in structured domains. We explore the behavior of the sRSA in the domain of color and show that pragmatic agents using sRSA on top of semantic representations, derived from the World Color Survey, attain efficiency very close to the information theoretic limit after only 1 or 2 levels of recursion. We also explore the interaction between pragmatic reasoning and learning in multi-agent reinforcement learning framework. Our results illustrate that artificial agents using sRSA develop communication closer to the information theoretic frontier compared to agents using RSA and just reinforcement learning. We also find that the ambiguity of the semantic representation increases as the pragmatic agents are allowed to perform deeper reasoning about each other during learning.

Pragmatic Reasoning in Structured Signaling Games

Empirical studies have demonstrated that when comprehenders are faced with informationally redundant utterances, they may make pragmatic inferences to accommodate the informationally redundant utterance (Kravtchenko & Demberg, 2015. Previous work has also shown that the strength of these inferences depends on prominence of the redundant utterance – if it is stressed prosodically, marked with an exclamation mark, or introduced with a discourse marker such as “Oh yeah”, atypicality inferences are stronger (Kravtchenko & Demberg, 2015; 2022; Ryzhova & Demberg, 2020).
The goal of the present paper is to demonstrate how both the atypicality inference and the effect of prominence can be modelled using the rational speech act (RSA) framework. We show that atypicality inferences can be captured by introducing joint reasoning about the habituality of events, following Degen, Tessler, and Goodman (2015); Goodman and Frank (2016). However, we find that joint reasoning models principally cannot account for the effect of differences in utterance prominence. This is because prominence markers do not contribute to the truth-conditional meaning. We then proceed to demonstrate that leveraging a noisy channel model, which has previously been used to model low-level acoustic perception (Bergen & Goodman, 2015), can successfully account for the empirically observed patterns of utterance prominence.

Modeling atypicality inferences in pragmatic reasoning

Generalizations are a fundamental linguistic tool for efficiently passing along information. To interpret the intended strength of a generalization, listeners rely on prior knowledge. Experienced and inexperienced listeners may interpret the same generalization differently, potentially leading to miscommunication. Speakers could mitigate such miscommunication by avoiding generalizations that inexperienced listeners are likely to misinterpret. However, experienced speakers may struggle to understand the perspective of an inexperienced listener. The present study examined whether experienced speakers adjust their use of generalizations based on the expertise of their intended audience. Results showed that any such adjustments are minimal and insufficient to avoid miscommunication as operationally defined. Future research may clarify the practical impact of such miscommunication by examining how generalizations are used in relation to speakers’ and listeners’ goals.

Adjusting the Use of Generalizations Based on Audience Expertise

Rap lyrics are a popular and understudied domain of human culture. In particular, improvised rap lyrics provide a unique window into the cognitive and linguistic constraints of creative language production. Although the vast majority of rap lyrics are written (premeditated), improvising lyrics has long been a core element of hip-hop culture. Very few efforts have investigated the neural underpinnings of improvised rap, and none so far have focused on the language output itself. This project compares phonemic, rhyme, and semantic features of written and improvised rap lyrics from 7 expert rappers in order to uncover related phonological structures. Here, I demonstrate that the phonemes of these two modes of production seem to be drawn from different distributions. In addition, across various metrics, improvised lyrics from these experts display smaller phonological structures and less variation than written lyrics from those same artists, all while consistently exhibiting large rhyming patterns (3+ syllable).

What’s Different About Improvised Rap?

Young children tend to prioritize objects over layouts in their drawings, often juxtaposing “floating” objects in the picture plane instead of grounding those objects in drawn representations of the extended layout. In the present study, we explore whether implicitly directing children’s attention to elements of the extended layout through a drawing’s communicative goal—to indicate the location of a hidden target to someone else—might lead children to draw more layout information. By comparing children’s drawings to a different group of children’s verbal descriptions, moreover, we explore how communicative medium affects children’s inclusion of layout and object information. If attention modulates children’s symbolic communication about layouts and objects, then children should both draw and talk about layouts and objects when they are relevant to the communicative task. If there are challenges or advantages specific to either medium, then children might treat layouts and objects differently when drawing versus describing them. We find evidence for both of these possibilities: Attention affects what children include in symbolic communication, like drawings and language, but children are more concise in their inclusion of relevant layout or object information in language versus drawings.

Young children’s drawings and descriptions of layouts and objects

We examined the signatures of music improvisation following a targeted manipulation that involves a mirroring task. In 18 pairs participants completed a solo music improvisation performance using a percussion instrument (cajon), a mirroring task with a partner, and joint music improvisation. Across pairs, we manipulated three different types of mirroring to examine its effects on coordination during joint music improvisation (Hierarchical mirroring, partner A leading and partner B following; Turn-Taking mirroring, partner A leading the first half and partner B leading the latter half; and Egalitarian mirroring, partners co-creating spontaneous movement together). From audio recordings, we extracted signatures of interpersonal coordination from the acoustic performances using cross-correlation. There were only marginally significant differences in interpersonal coordination during improvisation, with pairs in the Turn-Taking condition exhibiting lower levels of cross-correlation than in the Egalitarian condition. This work provides some initial theoretical insights about downstream effects of interpersonal dynamics on music improvisation.

Improving Improv: Effects of Interpersonal Coordination on Music Improvisation

The Rescorla-Wagner model has seen widespread success in modelling not only its original target of animal learning, but also several areas of human learning. However, despite its success, a number of studies with humans have found effects that are not predicted by the model, thus inspiring proposals for modifications to the model. One such proposal, by Van Hamme and Wasserman (1994, VHW), is that humans not only learn from present cues to all (present and absent) outcomes, as in the original model, but also learn from the absence of cues. They set out to test this hypothesis with a causal rating experiment. However, behaviour in learning studies may depend on the task. We propose that error-driven learning should be considered to be a form of implicit learning and that the results of VHW’s contingency judgement task might stem from explicit strategies involving logic and reasoning. The present study investigates this question by a) running simulations with both the original and modified versions of the model; b) replicating the VHW experiment (Experiment 1); and c) extending the experiment with new stimuli and by including unseen stimuli following the learning phases (Experiment 2). Simulations show that the VHW modified model predicts that cues learnt at the beginning will be unlearnt when absent over the following blocks, so that they become negative predictors over time. In contrast, the original RW predicts that the absent cues remain steady (positive) predictors over the blocks. Results showed no significant difference in cue assignment between training and test, in line with the original RW model. Moreover, predictive cues in the training phase showed significantly higher ratings than a new cue introduced in the test phase, at least in some cases, also partially supporting the original RW. We propose that in the development of human learning theory, attention should be paid to whether the behaviour (or other learning data) to be modelled results from implicit learning or involves higher level cognitive processes. We suggest that the RW may best capture implicit error-driven learning.

Does error-driven learning occur in the absence of cues? Examination of the effects of updating connection weights to absent cues

This paper presents a connectionist network simulation of Livesey and McLaren’s (2009) results. In that paper they showed that participants with post-discrimination gradients that were initially peak shifted became monotonic as they were exposed to the full range of test stimuli. While the authors suggest that this is the result of rule-based processes ‘taking over’ responding, we show how a connectionist network with an attentional parameter and realistic activation functions for the input can simulate both the peak shifted and monotonic gradients. Although we do not infer that the monotonic gradient obtained in peak shift paradigms is entirely the result of associative, rather than propositional processes, we suggest that perhaps it is a change in the allocation of attention, in conjunction with the underlying representational structures used for the stimuli that facilitates rule induction in this case.

Modelling Dual-Processes in a Connectionist Network

In the color-word contingency learning paradigm, each word appears more often in one color (HI contingency) than in other colors (LO contingency). Despite the words being irrelevant, responses to the relevant colors quickly become faster to HI than to LOs—the contingency learning effect. Across four experiments (N = 1,490), the number of response-irrelevant word stimuli linked to each of the three response-relevant colors varied from 1 to 2 to 4. Our prediction, derived from the Parallel Episodic Processing (PEP) 2.0 model, was borne out: The magnitude of the contingency learning effect declined monotonically as more words were linked as HIs to each color. Inconsistent with the PEP model, however, we observed changes in response times not only in HI but also in LO trials, indicating a need to amend the model. Associative learning may therefore be a function of prioritizing high probability items at the expense of low probability items.

Contingency learning decreases when associations are shared

The human voice effectively communicates a range of emotions with nuanced variations in acoustics. Existing emotional speech corpora are limited in that they are either (a) highly curated to induce specific emotions with predefined categories that may not capture the full extent of emotional experiences, or (b) entangled in their semantic and prosodic cues, limiting the ability to study these cues separately. To overcome this challenge, we propose a new approach called 'Genetic Algorithm with People' (GAP), which integrates human decision and production into a genetic algorithm. In our design, we allow creators and raters to jointly optimize the emotional prosody over generations. We demonstrate that GAP can efficiently sample from the emotional speech space and capture a broad range of emotions, and show comparable results to state-of-the-art emotional speech corpora. GAP is language-independent and supports large crowd-sourcing, thus can support future large-scale cross-cultural research.


Premium content

Color Overmodification Emerges from Data-Driven Learning and Pragmatic Reasoning

Downloads

Next from CogSci 2022

Pragmatic Reasoning in Structured Signaling Games

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES