Contrary to the notions of language as an amodal system, natural human communication is multimodal and combines speech, gestures, writing, and pictures. To account for this, recent work has proposed that our vocal, bodily, and graphic modalities persist in parallel in a multimodal language faculty, and both unimodal and multimodal expressions arise out of emergent states of a shared architecture. Such a model carries different expectations for the ways in which modalities may be similar or different from each other, and how they may interact. I will highlight these properties specifically for our graphic modality, which I argue can manifest in full visual languages when displaying both a systematic lexicon and complex grammar. I will use analysis of a corpus of several hundred annotated comics to show distinctive patterns that suggest they are drawn in different visual languages. Yet, I will also show that consistent “universal” linguistic principles persist across this structural diversity. Finally, I will argue that a multimodal language faculty requires us to change our conception of linguistic relativity, and I will show how subtle structures of spoken languages permeate across to visual languages. Altogether, this work argues for a multimodal basis of linguistic structure, and heralds a reconsideration of what constitutes the language system.

The multimodal language faculty and the visual languages of comics

Language has been characterized as a “unique” facet of human cognition with complex syntactic features like anaphora and distance dependencies. However, visual narratives, like comics, have been argued to use similar sequencing mechanisms. These narrative structures include “refiner” panels that “zoom in” on the contents of another panel. Similar to linguistic anaphora, refiners co-referentially connect inexplicit information in one unit (refiner/pronoun) to a more informative “antecedent.” Also, refiners can follow their antecedents (anaphoric) or precede them (cataphoric) with either proximal or distant connections. We explored these constraints of order and distance on visual narrative refiners by measuring event-related brain potentials (ERPs) to wordless comic strips. Anaphoric refiners evoked late sustained negativities (Nref) while distant anaphoric refiners attenuated N400s compared to all others, and all distance dependencies evoked leftward negativities. These responses are consistent with (neuro)cognitive responses shown to anaphora in language, suggesting domain-general constraints on the sequencing of referential dependencies.

Anaphoric distance dependencies in the sequential structure of wordless visual narratives

Information density in visual narrative units and sequences

Opening + Keynote 1: Dr. Neil Cohn

keynote

## Welcome to EMNLP 2022!
I am delighted to welcome you to EMNLP 2022! I believe this conference has been complicated beyond any precedent. Over the past year, it’s been thrilling to see the organization team approach each new puzzle with creativity and enthusiasm. We hope that those participating in Abu Dhabi as well as those joining remotely will leave the conference feeling newly inspired by the program and newly connected to our ever-growing community. Following EMNLP 2021 and major NLP conferences since, EMNLP 2022 is “hybrid,” serving both virtual and in-person participants.

Our key innovations for EMNLP 2022 include:

* EMNLP 2022 is “hybrid” in a second sense, as well: we allowed both direct and rolling review paper submissions, building on the pilot experiment of EMNLP 2021, which considered a small number of ARR submissions. 
* Familiar from NAACL but new to EMNLP, we’ve added an industry track.
* During the conference, “portals” will link virtual poster sessions to in-person conference participants during poster sessions each day.
* The first ACL-family conference in the United Arab Emirates.

 *Message from Noah A. Smith, University of Washington and Allen Institute for AI, Seattle, Washington, USA* 
***EMNLP 2022 General Chair***
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/9eec7d4a287ee18c278b08229290aa83.png)](https://drive.google.com/file/d/1OlPv6QBeo62VVTughj2jkiLeyHd1WnUt/view)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/a3db7a768409f05192210d98601edb25.png)](https://emnlp2022.rocket.chat/)

To access this site you need to register. Please register [here](https://2022.emnlp.org/registration/).

Register here

EMNLP 2022

Welcome!
EMNLP 2022 will take place in Abu Dhabi from December 7th to December 11th, 2022. And it will be held in hybrid mode, both online and offline.


**For a full list of posters in this session, click the See All link on the right side of the page.**

Poster Session 3 -  E: Linguistics

poster

### Looking for Cogsci 2023? Please [click here](https://underline.io/events/405/reception) to switch to 2023 edition.


**Dear Colleagues,**

Welcome to the 43rd Annual Conference of the Cognitive Science Society! 

This year’s conference brings together the latest research from all Cognitive Science and also highlights this year’s theme **Comparative Cognition: Animal Minds.** 

The program features four plenary speakers: Friederike Range (University of Veterinary Medicine, Vienna), Onur Güntürkün (Ruhr University Bochum), Julia Fischer (University of Göttingen), and Erich Jarvis (The Rockefeller University). The keynotes and invited symposia showcase the broad spectrum of ideas that encompass the conference theme of comparative cognition. 

Further, the program includes the Rumelhart Prize keynote presentation by Susan Goldin-Meadow (University of Chicago), and symposia to celebrate the winner of the Elman Prize: Wei Ji Ma (New York University), the Heineken Award winner Robert Zatorre (McGill University), and the Robert J. Glushko Dissertation Prizes session.

The program committee for CogSci 2021 received 1,025 submissions, including 710 full papers, 275 member abstracts, 8 publication-based talks, as well as 15 proposals for symposia, and 17 for workshops and tutorials. After a rigorous review process, the committee selected 152 papers for oral presentation and inclusion in the conference proceedings (14.8%), 332 papers for poster presentation and inclusion in the proceedings (32.4%), and 175 papers for poster presentation with inclusion of abstracts in the proceedings (17.1%) We also selected 275 submitted member abstracts. In addition, we accepted 6 publication-based talks, 12 symposia, and 9 workshops and tutorials to make for an exciting and inclusive program.

We hope that you enjoy the conference this year! 

**Your Program Chairs,**
![](https://assets.underline.io/uploads/markdown_image/1/image/99984908caba5e6453e5fc5ba34f3507.jpg)

**Learn more about the Vienna Cognitive Science Hub at the University of Vienna, hosts of CogSci 2021**

<iframe width="560" height="315" src="https://www.youtube.com/embed/464h1m3iU68" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

To review the proceedings for the conference or review the searchable program, please visit the [website](https://cognitivesciencesociety.org/program/).

CogSci 2021

The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education. The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

**Session Chair: **Neil Cohn 
**Technical Session Chair: **Elisabeth Tabery 
**Session Description: ** The last two decades have seen emerging research on the structure and cognition of visual narratives, like those found in comics and picture stories (Cohn & Magliano, 2020). A primary characteristic of this research has been the comparison between the meaning-making in sequencing of pictures and that of sequential words or sentences in language. This comparison has extended across research methods and their findings...In this symposium, we further explore these comparisons between language and visual narratives in four presentations exploring visual narrative sequencing. These studies span various methodologies of corpus research along with behavioral and neurocognitive experimentation, and they probe several fundamental topics of meaning-making found in both the verbal and visual modalities: the expression of motion events, the generation of inferences, anaphoricity and co-reference, and information density.

Click <a href="https://drive.google.com/file/d/1lGYg82UjJs1qG9GnkvK_G-Flb4DihJoM/view?usp=sharing" target="_blank">here</a> for the full symposium description

Neil Cohn

3

32

SHORT BIO

Presentations

The multimodal language faculty and the visual languages of comics

Anaphoric distance dependencies in the sequential structure of wordless visual narratives

Information density in visual narrative units and sequences

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES