United States

We present a computational (LSTM) model that learns to produce English (3sg and -bare) verb inflection when trained on English child-directed speech (CDS). The model is trained on input containing morphemized verbs and learns to predict the next word given a preceding sequence of words. The model produces the type of error (-bare for -3s) shown by English-learning children while avoiding errors that children do not often make (-3s for -bare). The model also shows the same type of sensitivity to input statistics that has been reported in English-learning children. Finally, we manipulated the length of the sequences the model is trained on, and show that this results in the delayed acquisition of -3sg forms that is characteristic of English-learning children with Developmental Language Disorder (DLD). Taken together these results suggest that input-driven learning is a major determinant of both typical and impaired acquisition of English verb inflection.

CogSci 2023

Simulating children’s verb inflection errors in English using an LSTM language model

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

A central challenge of cross-situational word learning is retaining word-referent mappings across exposures. We evaluate Memory-Bound Pursuit (MBP), a hypothesis-testing model of cross-situational word-learning which aims to account for learners’ memory constraints via a single parameter targeting the number of words that can be learned concurrently. Here, we show that by varying this parameter with age, MBP can capture both children’s and adults’ cross-situational word-learning success under varying levels of ambiguity. We also present new experimental findings supporting novel predictions made by MBP about the retention of word-referent mappings across intervening exposures. These findings suggest that MBP provides a strong baseline model of cross-situational word learning, capturing both developmental trends and experimental evidence of memory limitations for word learning.

Memory as a computational constraint in cross-situational word learning

The human lexicon expresses a wide array of objects with a limited set of named categories. Previous work has suggested that semantic categories are structured compactly to enable informative communication. Informativeness is typically quantified with respect to the entirety of a semantic domain and not at the level of individual words. We develop a measure of name informativeness in an information-theoretic framework grounded in visual object representations derived from natural images. We use computer vision models to characterize informativeness of individual names with respect to large-scale data in a naturalistic setting. We show that our informativeness measure predicts degrees of specificity in lexical categories more precisely than alternative measures based on entropy and frequency. We also show that name informativeness jointly captures within-category similarity and distinctiveness across categories. Our analyses suggest how the variability of names in the broad lexicon may be understood through the lens of information theory.

Quantifying informativeness of names in visual space

Computational models of morphology acquisition have played a central role in debates over the nature of morphological representations. Recent advances in natural language processing have renewed these debates with the advent of powerful artificial neural network architectures of morphological inflection. However, the actual suitability of these advanced neural models as models of human morphology acquisition remains uncertain. We argue that much of this confusion stems from inconsistent methods of training and evaluation. In this work, we demonstrate that more careful data set creation and an evaluation combining quantitative analysis and comparison with human development will put the evaluation of neural models on firmer ground.

Re-Evaluating the Evaluation of Neural Morphological Inflection Models

This study examines the reliability and validity of using two types of crowd-sourced judgments to collect lexical diversity scores. Scaled and pairwise comparison approaches were used to collect data from non-expert Amazon Mechanical Turk workers. The reliability of the lexical diversity ratings for the crowd-sourced raters was assessed along with those from trained raters using a variety of reliability statistics. The validity of the ratings was examined by 1) comparing crowd-sourced and trained ratings, 2) comparing crowd-sourced and trained ratings to ratings of language proficiency, and 3) by using an objective measure of lexical diversity to predict the crowd-sourced and trained ratings. The results indicate that scaled crowd-sourced ratings showed strong reliability in terms of text and rater strata and showed fewer misfitted texts than the trained raters. The scaled crowd-sourced ratings were also strongly predicted by lexical diversity features derived from the texts themselves.

Crowd-Sourcing Human Ratings of Linguistic Production

Words that regularly fill the same sentential slots are said to be paradigmatically related. Paradigmatic relations may be retained through a direct association or a latent representation at encoding, or by reinstating context during retrieval. We paired proper names by embedding them into two instances of the same sentence frame, each in a separate list, yielding blocks of two study-cloze sessions. The pairing between proper names was fixed across twelve blocks. In the static condition, the same sentence frames were used across blocks, while in the dynamic condition sentence frames changed for each block. Interference should accrue in both conditions if paradigmatic relations are based on a direct association or overlap in a latent representation, however, if paradigmatic relations are mediated by retrieved context then changing the sentence frame should release interference. Our results are consistent with a context-mediation account of paradigmatic relations.

Paradigmatic formation through context-mediation

What makes some sentences more memorable than others? In this work, we build on past work in image and word memorability and conduct a large-scale sentence memorability experiment with responses from 500 participants for 2,500 target sentences. We find that sentences vary in their intrinsic memorability. We predict the memorability of sentences from a variety of word- and sentence-level features, including representations from artificial large language models and the information-theoretic measure of surprisal. We find that the best predictor of sentence memorability is the presence of individually memorable words, but that in the absence of such words, sentence-level features predict sentence memorability. This suggests that otherwise forgettable words can together create memorable compositional meanings that remain in memory.

Context-sensitive features predict sentence memorability in the absence of memorable words

The relevance of Broca's area to language grammar processing is well established. Its relevance to implicitly learned non-linguistic sequential rules has also been demonstrated. Previous work by our lab has shown this is true even if subjects are unaware they are being trained on, and tested for, sensitivity to these rules. We extend this work to show that the degree of differential activation in Broca's area for implicitly learned grammatical vs. ungrammatical sequences is correlated across subjects with their behavioral evidence of their degree of learning of these rules as assessed by reaction time differences. Broca's is among the three regions showing the highest association with degree of learning. This finding further underscores the relevance of Broca's for non-linguistic rule learning, and suggests that the original function of this area in our pre-human ancestors may have been the implicit learning of any kind of sequential patterns in the environment.

Degree of learning of an artificial grammar correlates with differential fMRI activation of Broca's area

Reference to events using pronouns like 'it' or demonstratives like 'that' has been difficult to study, because unlike in reference to people or objects, the ground truth of interpretation is harder to establish. In this paper, we introduce a new task to understand the roles of three parameters in event reference resolution: The referential expressions themselves, sentential aspect, and discourse structure. We find that different referring expressions reliably refer to specific parts of a discourse, and confirm previous findings that sentential aspect influences referential accessibility; that the structure of a discourse itself has a major effect on reference resolution, and most notably that, contrary to predictions in the literature, some events that are not on the right-frontier of a discourse are nevertheless available for reference. Our findings contribute to a growing literature on anaphor resolution that finds that the parameter structure that determines reference resolution is multifaceted, but predictable.

Discourse structure affects reference resolution to events in English: Evidence from a new paradigm

Repetition of linguistic structure plays a role in both language comprehension and production. Previously encountered structures are processed faster, and speakers tend to reuse them in new utterances—a phenomenon known as structural priming. According to the prevailing interpretation of structural priming, linguistic input activates an underlying mental representation based on constituents, a syntactic unit derived from rule-based grammars (e.g., [[he]NP [hears [a sound]NP]VP]S). Here we ask whether structural priming can occur for non-constituent parts-of-speech fragments, such as pronoun verb determiner (e.g., he hears a). Across two preregistered phrasal decision experiments, we show that structural priming can occur at the level of three-word parts-of-speech sequences and in the absence of constituents. Using corpus analysis, we further show that structural priming of non-constituents also occurs in real-life dialogue. These results imply that constituent structure is not a necessary prerequisite for structural priming and provide a challenge to contemporary approaches to grammar.

Does he hears a prime I prefer the? Testing a novel chunk-based perspective on structural priming

The lesser known G(rammatical)-Maze task (Forster, Guerrera, & Elliot, 2009) is arguably a better choice than self-paced reading (Mitchell, 2004) for detecting difficulty from word to word in online sentence processing over crowdsourcing platforms. In G-Maze, a participant must choose between each successive word in a sentence and a distractor word that does not make sense based on the preceding context. If a participant chooses the distractor as opposed to the actual word, then the trial ends and they may not complete the sentence. Thus, G-Maze automatically filters out data from inattentive participants, and more effectively localizes differences in processing difficulty. Still, the effort required to pick contextually inappropriate distractors for hundreds of words might cause an experimenter to hesitate before picking this method. To save experimenters this time and effort, Boyce, Futrell, and Levy (2020) developed A(uto)-Maze, a tool that automatically generates distractors using a computational language model. We now introduce the next generation of A-Maze: T(ransformer)-Maze. Transformer models are the current state of the art in natural language processing, and thousands, pretrained in a variety of languages, are freely available on the internet, specifically through Huggingface’s Transformers package (Wolf et al., 2020). In our validation experiment, T-Maze proves itself to be as effective as G-Maze with handmade materials, run in a lab. This tool thus allows psycholinguists to easily gather high-quality online sentence processing data in many different languages.

Premium content

Simulating children’s verb inflection errors in English using an LSTM language model

Next from CogSci 2023

Memory as a computational constraint in cross-situational word learning

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES