United States

When you see a glass fall off the table, you can predict it will break without seeing it hit the ground. Similarly, if you hear the glass shatter, you can infer what happened without seeing anything at all. Our knowledge of the causal mechanisms structuring our world allows us to make impressively accurate inferences based on incomplete information spread across multiple sensory modalities. In this work, we study the cognitive processing that supports this remarkable behavior. We utilize the Plinko domain, an intuitive physics setup where marbles are dropped into a box from one of three holes, colliding with obstacles as they fall to the ground. Participants judge where they think the ball fell from based on visual and auditory evidence. We track participants&#39; eye-gaze to gain deeper insight into their mental processes, and develop models that characterize the computational processes underlying participant behavior.

CogSci 2021

Eye-Tracking Multi-Modal Inference

### Looking for Cogsci 2023? Please [click here](https://underline.io/events/405/reception) to switch to 2023 edition.


**Dear Colleagues,**

Welcome to the 43rd Annual Conference of the Cognitive Science Society! 

This year’s conference brings together the latest research from all Cognitive Science and also highlights this year’s theme **Comparative Cognition: Animal Minds.** 

The program features four plenary speakers: Friederike Range (University of Veterinary Medicine, Vienna), Onur Güntürkün (Ruhr University Bochum), Julia Fischer (University of Göttingen), and Erich Jarvis (The Rockefeller University). The keynotes and invited symposia showcase the broad spectrum of ideas that encompass the conference theme of comparative cognition. 

Further, the program includes the Rumelhart Prize keynote presentation by Susan Goldin-Meadow (University of Chicago), and symposia to celebrate the winner of the Elman Prize: Wei Ji Ma (New York University), the Heineken Award winner Robert Zatorre (McGill University), and the Robert J. Glushko Dissertation Prizes session.

The program committee for CogSci 2021 received 1,025 submissions, including 710 full papers, 275 member abstracts, 8 publication-based talks, as well as 15 proposals for symposia, and 17 for workshops and tutorials. After a rigorous review process, the committee selected 152 papers for oral presentation and inclusion in the conference proceedings (14.8%), 332 papers for poster presentation and inclusion in the proceedings (32.4%), and 175 papers for poster presentation with inclusion of abstracts in the proceedings (17.1%) We also selected 275 submitted member abstracts. In addition, we accepted 6 publication-based talks, 12 symposia, and 9 workshops and tutorials to make for an exciting and inclusive program.

We hope that you enjoy the conference this year!<br><br>

**Your Program Chairs,**
![](https://s3.eu-west-1.amazonaws.com/underline.prod/uploads/markdown_image/1/image/99984908caba5e6453e5fc5ba34f3507.jpg)

**Learn more about the Vienna Cognitive Science Hub at the University of Vienna, hosts of CogSci 2021**

<iframe width="560" height="315" src="https://www.youtube.com/embed/464h1m3iU68" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

To review the proceedings for the conference or review the searchable program, please visit the [website](https://cognitivesciencesociety.org/program/).


The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education. The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

The AGI conference series is the only major conference series devoted wholly and specifically to the creation of AI systems possessing general intelligence at the human level and ultimately beyond. By gathering together active researchers in the field, for presentation of results and discussion of ideas, we accelerate our progress toward our common goal.

Underline hosted the two main days of AGI 2020 on Wednesday June 24th and Thursday June 25th. Both days were **live-streamed** and you can read the full program bellow. All presentations are available for a **asynchronous viewing**. All presenters are grouped into panels. You can also hear guest talks by **Ben Goertzel** (SingularityNET), **Valery Tarasov** (Semenov Institute of Chemical Physics), **Tatiana Shavrina** (Sberbank) and **Joscha Bach** (MIT Media Lab). 

AGI-20

The AGI conference series is the only major conference series devoted wholly and specifically to the creation of AI systems possessing general intelligence at the human level and ultimately beyond. By gathering together active researchers in the field, for presentation of results and discussion of ideas, we accelerate our progress toward our common goal.

When you see a glass fall off the table, you can predict it will break without seeing it hit the ground. Similarly, if you hear the glass shatter, you can infer what happened without seeing anything at all. Our knowledge of the causal mechanisms structuring our world allows us to make impressively accurate inferences based on incomplete information spread across multiple sensory modalities. In this work, we study the cognitive processing that supports this remarkable behavior. We utilize the Plinko domain, an intuitive physics setup where marbles are dropped into a box from one of three holes, colliding with obstacles as they fall to the ground. Participants judge where they think the ball fell from based on visual and auditory evidence. We track participants' eye-gaze to gain deeper insight into their mental processes, and develop models that characterize the computational processes underlying participant behavior.

poster

We investigated the subjective experience of loneliness during COVID-19 by analyzing social media postings from March 2020 to January 2021. We collected text data from loneliness-related subgroups of Reddit and sampled 12787 posts that were written in ten consecutive days from each month. The results suggest that when individuals express their loneliness, they show an internal focus of attention on their emotions, desires, and cognitive appraisals rather than an external focus of attention on situations or other people. Linguistic markers of emotions expressed by lonely individuals included depression, anxiety, anger, hate, helplessness, and sadness. Also, loneliness-related topics were generally about their internal states pertinent to various social relationships, interpersonal interaction deficits, and their own lives in broad time perspectives. COVID-19 related loneliness was associated with negative appraisal of one’s situation and reaching out for new relationships online.

Web-scraping the Expression of Loneliness during COVID-19

To convey meaning, language relies on hierarchically organized, long-range relationships spanning words, phrases, sentences, and discourse. As the distances between elements in language sequences increase, the strength of the long range relationships between those elements decays following a power law. This power-law relationship has been attributed variously to long-range sequential organization present in language syntax, semantics, and discourse structure. However, non-linguistic behaviors in numerous phylogenetically distant species, ranging from humpback whale song to fruit fly motility, demonstrate similar long-range statistical dependencies. Therefore, we hypothesized that long-range statistical dependencies in speech may occur independently of linguistic structure. To test this hypothesis, we measured long-range dependencies in speech corpora from children (aged 6 months -- 12 years). We find that adult-like power-law statistical dependencies are present in human vocalizations prior to the production of complex linguistic structure. These linguistic structures cannot, therefore, be the sole cause of long-range statistical dependencies in language.

Long-range sequential dependencies precede complex syntactic production in language acquisition

The meaning of a word is acquired using other words. The current study explores how Japanese native-speakers and adult second-language (L2) learners of Japanese apply the meanings of verbs that belong to the same semantic domain, focusing on the semantic domains of “kai (to open),” “guan (to close),” “kan (to see/to look at/to watch),” and “chuan (to wear)” in Chinese. A comparison of the results from native speakers and L2 learners revealed a marked dissimilarity between the patterns of their semantic domains. Native-speakers were able to use all of the words correctly, whereas L2 learners made errors that were heavily influenced by the word concepts of their native language. Our findings suggest two main points: first, it is possible that adult L2 learners have their own semantic domain influenced by their native language; second, it is important to systematically investigate words that belong to the same domain as a whole.

How do the Concepts of Native Language Influence Second Language Learning? : Evidence from the Reconstruction of Word Semantic Domain

Speakers use different language to communicate with partners in different communities. But how do we learn and represent which conventions to use with which partners? In this paper, we argue that solving this challenging computational problem requires speakers to supplement their lexical representations with knowledge of social group structure. We formalize this idea by extending a recent hierarchical Bayesian model of convention formation with an intermediate layer explicitly representing the latent communities each partner belongs to, and derive predictions about how conventions formed within a group ought to extend to new in-group and out-group members. We then present evidence from two behavioral experiments testing these predictions using a minimal group paradigm. Taken together, our findings provide a first step toward a formal framework for understanding the interplay between language use and social group knowledge.

Respect the code: Speakers expect novel conventions to generalize within but not across social group boundaries

This study investigated how a group of twenty seven Wuhan citizens (coronavirus patients and their family member, medical staff, college students, social workers, teacher, journalist) employed metaphors to communicate about their experiences of COVID-19 pandemic through in-depth individual interviews. The analysis of metaphors captured the different kinds of emotional states and psychological conditions of the research participants, focusing on their mental imagery of COVID-19, extreme emotional experiences, and symbolic behaviors under the pandemic. The results show that multiple metaphors were used to construe emotionally-complex, isolating experiences of the COVID-19 pandemic. Most metaphorical narratives are grounded in embodied sensorimotor experiences such as body parts, batting, fighting, hit, weight, temperature, spatialization, motion, violence, light, and journey. Embodied metaphors were manifested in both verbal expressions and nonlinguistic behaviors (e.g., patients’ symptom of obsessive-compulsive behaviors). These results suggest that the bodily experiences of the pandemic, the environment, and the psychological factors combine to shape people’s metaphorical thinking processes.

Embodied Metaphor in Communication about Experiences of COVID-19 Pandemic

Previous literature suggests that the language processor generates expectations about upcoming material. Several studies have found evidence for a prediction error cost in cases where the comprehender encountered not the predicted word but a plausible unexpected continuation instead. This cost is argued to be a result of an inhibitory process that suppresses activation of the originally predicted word. Others have found no such evidence for a prediction cost. In a probe recognition memory task, we find evidence for interference from an incorrectly predicted word, and in a self-paced reading study, we find evidence for facilitation when the originally predicted word is encountered later on. Taken together, our results provide evidence against a strong version of the suppression account, in which incorrectly predicted words are fully inhibited. Instead we argue in favor of a passive lingering activation account, in which activation for the disconfirmed prediction gradually decays over time.

Unexpected guests: When disconfirmed predictions linger

Infant vocabulary development is inevitably dependent on the speech they hear in their environment. This paper reports an investigation of the vocabulary statistics that predict a word’s age of acquisition, focusing on frequency and contextual diversity as derived from child-directed speech data along with associative norms generated by adults. Age of acquisition is operationalised using parental-report of infant word knowledge in a British English-speaking population. The work can be considered an extension of Hills, Maouene, Riordan, and Smith (2010) using a fully British English dataset. We found significant effects of both word frequency and word associations on age of acquisition. Interestingly, the strength of these predictors differed between word classes, with frequency being the strongest predictor for nouns and associations the strongest predictor for function words.

Exploring the variable effects of frequency and semantic diversity as predictors for a word's ease of acquisition in different word classes

Learning verbs is challenging because it is difficult to infer the precise meaning of a verb when there are a multitude of relations that one can derive from a single event. To study this verb learning challenge, we used children's egocentric view collected from naturalistic toy-play interaction as learning materials and investigated how visual and linguistic information provided in individual naming moments as well as cross-situational information provided from multiple learning moments can help learners resolve this mapping problem using the Human Simulation Paradigm. Our results show that learners benefit from seeing children's egocentric views compared to third-person observations. In addition, linguistic information can help learners identify the correct verb meaning by eliminating possible meanings that do not belong to the linguistic category. Learners are also able to integrate visual and linguistic information both within and across learning situations to reduce the ambiguity in the space of possible verb meanings.

Human Learners Integrate Visual and Linguistic Information Cross-Situational Verb Learning

Human communication involves far more than words; speakers’ utterances are often accompanied by various kinds of emotional expressions. How do listeners represent and integrate these distinct sources of information to make communicative inferences? We first show that people, as listeners, integrate both verbal and emotional information when inferring true states of the world and others' communicative goals, and then present computational models that formalize these inferences by considering different ways in which these signals might be generated. Results suggest that while listeners understand that utterances and emotional expressions are generated by a balance of speakers’ informational and social goals, they additionally consider the possibility that emotional expressions are noncommunicative signals that directly reflect the speaker’s internal states. These results are consistent with the predictions of a probabilistic model that integrates goal inferences with linguistic and emotional signals, moving us towards a more complete formal theory of human communicative reasoning.

Integrating emotional expressions with utterances in pragmatic inference

In isolated word reading, readers have the best performance when fixating between the beginning and center of a word, i.e., the optimal viewing position (OVP). Also, perceptual expertise literature suggests that both global and local processing are important for visual stimulus recognition. Here we showed that in lexical decision, higher similarity to an eye movement pattern that focused at the OVP and better local processing ability predicted faster response time (RT), in addition to verbal working memory and lexical knowledge. Also, this eye movement pattern was associated with longer RT in naming isolated single letters, suggesting conflicting visual abilities required for identifying isolated letters and letter strings. In contrast, word and pseudoword naming RT, and lexical decision and naming accuracy, were predicted by lexical knowledge but not eye movement pattern or global-local processing abilities. Thus, visual processing abilities are important factors accounting for isolated word reading fluency not involving naming.

Eye-Tracking Multi-Modal Inference

Downloads

Next from CogSci 2021

Web-scraping the Expression of Loneliness during COVID-19

Similar lecture

Cognitive Machinery and Behaviours

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES