Learning verbs is challenging because it is difficult to infer the precise meaning of a verb when there are a multitude of relations that one can derive from a single event. To study this verb learning challenge, we used children's egocentric view collected from naturalistic toy-play interaction as learning materials and investigated how visual and linguistic information provided in individual naming moments as well as cross-situational information provided from multiple learning moments can help learners resolve this mapping problem using the Human Simulation Paradigm. Our results show that learners benefit from seeing children's egocentric views compared to third-person observations. In addition, linguistic information can help learners identify the correct verb meaning by eliminating possible meanings that do not belong to the linguistic category. Learners are also able to integrate visual and linguistic information both within and across learning situations to reduce the ambiguity in the space of possible verb meanings.

Human Learners Integrate Visual and Linguistic Information Cross-Situational Verb Learning

Infants learn the meaning of words from accumulated experiences of real-time interactions with their caregivers. To study the effects of visual sensory input on word learning, we recorded infant's view of the world using head-mounted eye trackers during free-flowing play with a caregiver. While playing, infants were exposed to novel label-object mappings and later learning outcomes for these items were tested after the play session. In this study we use a classification based approach to link properties of infants' visual scenes during naturalistic labeling moments to their word learning outcomes. We find that a model which integrates both highly informative and ambiguous sensory evidence is a better fit to infants' individual learning outcomes than models where either type of evidence is taken alone, and that raw labeling frequency is unable to account for the word learning differences we observe. Here we demonstrate how a computational model, using only raw pixels taken from the egocentric scene image, can derive insights on human language learning.

In-the-Moment Visual Information from the Infant's Egocentric View Determines the Success of Infant Word Learning: A Computational Study

Numerous studies in cognitive development have provided converging evidence that Joint Attention (JA) is crucial for children to learn about the world together with their parents. However, a closer look reveals that, in the literature, JA has been operationally defined in different ways. For example, some definitions require explicit signals of “awareness” of being in JA—such as gaze following, while others simply define JA as shared gaze to an object or activity. But what if “awareness” is possible without gaze following? The present study examines egocentric images collected via head-mounted eye-trackers during parent-child toy play. A Convolutional Neural Network model was used to process and learn to classify raw egocentric images as JA vs not JA. We demonstrate individual child and parent egocentric views can be classified as being part of a JA bout at above chance levels. This provides new evidence that an individual can be “aware” they are in JA based solely on the in-the-moment visual information. Moreover, both models trained on child views and those trained on parent views leveraged the visual properties associated with visual object holding to improve classification accuracy—suggesting a critical role for object handling in not only establishing JA, as shown in previous research, but also in inferring the social partner’s attentional state during JA.

Modeling joint attention from egocentric vision


**For a full list of posters in this session, click the See All link on the right side of the page.**

Poster Session 2 -  E: Linguistics

poster

### Looking for Cogsci 2023? Please [click here](https://underline.io/events/405/reception) to switch to 2023 edition.


**Dear Colleagues,**

Welcome to the 43rd Annual Conference of the Cognitive Science Society! 

This year’s conference brings together the latest research from all Cognitive Science and also highlights this year’s theme **Comparative Cognition: Animal Minds.** 

The program features four plenary speakers: Friederike Range (University of Veterinary Medicine, Vienna), Onur Güntürkün (Ruhr University Bochum), Julia Fischer (University of Göttingen), and Erich Jarvis (The Rockefeller University). The keynotes and invited symposia showcase the broad spectrum of ideas that encompass the conference theme of comparative cognition. 

Further, the program includes the Rumelhart Prize keynote presentation by Susan Goldin-Meadow (University of Chicago), and symposia to celebrate the winner of the Elman Prize: Wei Ji Ma (New York University), the Heineken Award winner Robert Zatorre (McGill University), and the Robert J. Glushko Dissertation Prizes session.

The program committee for CogSci 2021 received 1,025 submissions, including 710 full papers, 275 member abstracts, 8 publication-based talks, as well as 15 proposals for symposia, and 17 for workshops and tutorials. After a rigorous review process, the committee selected 152 papers for oral presentation and inclusion in the conference proceedings (14.8%), 332 papers for poster presentation and inclusion in the proceedings (32.4%), and 175 papers for poster presentation with inclusion of abstracts in the proceedings (17.1%) We also selected 275 submitted member abstracts. In addition, we accepted 6 publication-based talks, 12 symposia, and 9 workshops and tutorials to make for an exciting and inclusive program.

We hope that you enjoy the conference this year! 

**Your Program Chairs,**
![](https://assets.underline.io/uploads/markdown_image/1/image/99984908caba5e6453e5fc5ba34f3507.jpg)

**Learn more about the Vienna Cognitive Science Hub at the University of Vienna, hosts of CogSci 2021**

<iframe width="560" height="315" src="https://www.youtube.com/embed/464h1m3iU68" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

To review the proceedings for the conference or review the searchable program, please visit the [website](https://cognitivesciencesociety.org/program/).

CogSci 2021

The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education. The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

**Session Chair: **Daphna Buchsbaum 
**Technical Session Chair: **Alon Hafri 
**Session Description: ** The session is comprised of 15 minute talks followed by up to 5 minutes of live Q&A time for each presenter.

IP8: Development models

technical paper

**Session Chair: **Stephanie Denison 
**Technical Session Chair: **Tiffany Doan 
**Session Description: ** The session is comprised of 15 minute talks followed by up to 5 minutes of live Q&A time for each presenter.

Andrei Amatuni

3

SHORT BIO

Presentations

Human Learners Integrate Visual and Linguistic Information Cross-Situational Verb Learning

In-the-Moment Visual Information from the Infant's Egocentric View Determines the Success of Infant Word Learning: A Computational Study

Modeling joint attention from egocentric vision

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES