United States

We investigated racial bias in emotion inference by having participants describe the emotion of the featured person in the image, including Asian, Black, and White. We collected 4,197 sentences (63,900 tokens) and used the data to train Word2Vec, a neural network-based word embedding model. We calculated the cosine distance between emotion words and words indicating the target for each racial group in order to measure the strength of association. Although all images portrayed neutral emotions, the results show that negative emotion words were close to Asian and Black, whereas neutral words were close to White. This result indicates that stereotypes contribute to the racial bias in Artificial Intelligence as crowdsourcing workers can generate annotations depending on the featured person&#39;s race. Based on the present study, we suggest employing various de-biasing methods such as data augmentation or removing bias components in word embeddings before using Artificial Intelligence in the real world.

CogSci 2021

Racial Bias in Emotion Inference: An Experimental Study Using a Word Embedding Method

### Looking for Cogsci 2023? Please [click here](https://underline.io/events/405/reception) to switch to 2023 edition.


**Dear Colleagues,**

Welcome to the 43rd Annual Conference of the Cognitive Science Society! 

This year’s conference brings together the latest research from all Cognitive Science and also highlights this year’s theme **Comparative Cognition: Animal Minds.** 

The program features four plenary speakers: Friederike Range (University of Veterinary Medicine, Vienna), Onur Güntürkün (Ruhr University Bochum), Julia Fischer (University of Göttingen), and Erich Jarvis (The Rockefeller University). The keynotes and invited symposia showcase the broad spectrum of ideas that encompass the conference theme of comparative cognition. 

Further, the program includes the Rumelhart Prize keynote presentation by Susan Goldin-Meadow (University of Chicago), and symposia to celebrate the winner of the Elman Prize: Wei Ji Ma (New York University), the Heineken Award winner Robert Zatorre (McGill University), and the Robert J. Glushko Dissertation Prizes session.

The program committee for CogSci 2021 received 1,025 submissions, including 710 full papers, 275 member abstracts, 8 publication-based talks, as well as 15 proposals for symposia, and 17 for workshops and tutorials. After a rigorous review process, the committee selected 152 papers for oral presentation and inclusion in the conference proceedings (14.8%), 332 papers for poster presentation and inclusion in the proceedings (32.4%), and 175 papers for poster presentation with inclusion of abstracts in the proceedings (17.1%) We also selected 275 submitted member abstracts. In addition, we accepted 6 publication-based talks, 12 symposia, and 9 workshops and tutorials to make for an exciting and inclusive program.

We hope that you enjoy the conference this year!<br><br>

**Your Program Chairs,**
![](https://assets.underline.io/uploads/markdown_image/1/image/99984908caba5e6453e5fc5ba34f3507.jpg)

**Learn more about the Vienna Cognitive Science Hub at the University of Vienna, hosts of CogSci 2021**

<iframe width="560" height="315" src="https://www.youtube.com/embed/464h1m3iU68" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

To review the proceedings for the conference or review the searchable program, please visit the [website](https://cognitivesciencesociety.org/program/).


The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education. The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

We investigated racial bias in emotion inference by having participants describe the emotion of the featured person in the image, including Asian, Black, and White. We collected 4,197 sentences (63,900 tokens) and used the data to train Word2Vec, a neural network-based word embedding model. We calculated the cosine distance between emotion words and words indicating the target for each racial group in order to measure the strength of association. Although all images portrayed neutral emotions, the results show that negative emotion words were close to Asian and Black, whereas neutral words were close to White. This result indicates that stereotypes contribute to the racial bias in Artificial Intelligence as crowdsourcing workers can generate annotations depending on the featured person's race. Based on the present study, we suggest employing various de-biasing methods such as data augmentation or removing bias components in word embeddings before using Artificial Intelligence in the real world.

poster

Non-native speakers have been found to be penalized for their accent and grammatical errors. However, little is known about whether and how accent and grammaticality interact to influence native listeners’ perception of non-native speech, and whether listeners’ personality plays a role in this. We examined these questions by relating the acceptability scores of 40 English speech stimuli rated by 60 British listeners (30 female; 30 unfamiliar with accented speech) to factors including speech accent (British vs. Polish), grammaticality (well-formed vs. error-filled) and listener personality. The results suggest that non-native accent “protects” the speaker from being penalized for grammatical errors unless the listener has a certain personality profile: compared to non-native accented speech, the acceptability ratings of native accented speech showed a larger decrease when grammatical errors were present, yet listeners who were more conscientious, extravert or agreeable tended to give lower acceptability ratings to non-native accented speech, regardless of grammaticality.

Native perception of non-native speech: Speaker accent mitigates penalization for language errors in non-native speech unless the listener is conscientious

A number of studies have explored whether language is grounded in action and perception, however little attention has been given to the sense of smell. Here we directly test whether olfactory information is necessary for comprehension of odor-related language, by comparing language performance in a group of participants with no sense of smell (anosmics) with a group of control participants with an intact sense of smell. We found no difference in comprehension of odor- and taste-related language between anosmics and controls using a lexical decision task, suggesting olfaction is not crucial to semantic representations of odor-related language. However, we did find that anosmics were better at remembering odor-related words than controls, and they also rated odor- and taste-related words as more positively valenced than control participants. We suggest odor-related language is more salient and emotional to anosmics because it reminds them of their missing sense. Overall, this study supports the proposal that odor-related language is not grounded in odor perception.

Can losing the sense of smell affect odor language?

Temporal gestures reveal our metaphorical thinking of time. Less is known about how speakers of different languages encode temporal gestures. We asked 19 adults to explain the meanings of 12 spatial metaphors of time in Turkish sentences. We coded gestures according to the coding scheme of Cooperrider and Nuñez (2009). Individuals produced 237 temporal gestures. There were 162 placing (68.35%), 10 pointing (4.22%), 13 duration-marking (5.49%), 12 bridging (5.06%), 40 animating (16.88%) gestures. We also identified 61 temporal gestures that did not fall into any category in their classification. These gestures depicted the change of the time of an event caused by humans (e.g., to postpone a meeting) with an open hand moving the events along an axis. We termed these as “moving” gestures and suggest that speakers of different languages may vary in their representations of temporal gestures.

Temporal Gestures in Turkish Metaphor Explanations

We explore how morphological cue occurrence and frequency-based heuristics interplay during sentence comprehension in Korean, a lesser-studied language in this respect. Two self-paced reading experiments with a suffixal passive construction (verb-final vs. verb-initial) and a morphological causative construction (verb-initial in comparison to the same word-order pattern in the suffixal passive) revealed that the heuristics (canonicity of word order; typicality of form-function parings involving case-marking) affected processing behaviours more strongly than the expected advantage of an early-arriving morphological cue in comprehension. Our findings support the heuristic-before-algorithm processing architecture, which is driven by the general property of human cognition that continuously seeks to reduce the burden of work at hand at the earliest opportunities. This appeals to the online cognitive equilibrium hypothesis that argues for the processor’s propensity to enter and remain in the state of cognitive equilibrium as early and long as possible during processing.

Keep Calm and Move On: Interplay between Morphological Cue Occurrence and Frequency-based Heuristics for Sentence Comprehension in Korean

When asked to paraphrase semantically strained sentences (e.g., the lantern limped), participants alter word meanings to fit the context (e.g., the light flickered). Judges who were asked to classify these meaning extensions classified verb extensions as primarily metaphoric and noun extensions as primarily taxonomic or metonymic/associative (King & Gentner, 2019; in prep.). To determine whether these online patterns are reflected in established word senses, we sampled 120 nouns and 120 verbs from three different frequency bands (100 wpm, 10 wpm, and 1 wpm) and obtained metaphoricity ratings of every sense for each word (1015 senses total, Oxford Dictionary API). Verb senses were rated as significantly more metaphoric than noun senses overall. In addition, metaphoricity increased as sense frequency decreased, and this effect was stronger for verbs than for nouns. Verbs’ high propensity for metaphoric extension has implications for evolution of word meanings.

Verbs are More Metaphoric than Nouns: Evidence from the Lexicon

Languages differ regarding the depth, structure, and syntactic domains of recursive structures. Even within a language, some structures allow infinite self-embedding while others are more restricted. For example, English allows infinite free embedding of the prenominal genitive -s, whereas the postnominal genitive of is largely restricted to one level and to a limited set of items. Therefore, speakers need to learn from experience which specific structures allow free embedding and which do not. One effort to account for the underlying learning mechanism, the distributional learning proposal, suggests the recursion of a structure (e.g. X1’s- X2) is licensed if X1 position and X2 position are productively substitutable in the input. A series of corpus studies have confirmed the availability of such distributional cues in child directed speech. The present study further tests the distributional learning proposal with an artificial language learning experiment.

Distributional learning of recursive structures

During language comprehension, words are easier to process when predictable based on local sentence context. It is unclear how information available from the global discourse interacts with this local predictive processing. To test this, we conducted an online self-paced reading study (n=100), manipulating the predictability of a critical word in a target sentence while varying the coherency of the target sentence with a preamble context that preceded it. We found that people processed the critical word faster when it was highly predictable from local context, replicating a common finding. We also found people were slower to process the critical word when the target sentence was presented in an incoherent context. However, no interaction was observed. Results indicate that subtler semantic changes, such as topic shifts, slow language processing but do not reduce the benefit of a highly predictable, local context.

Effects of global discourse coherence on local contextual predictions

Rational Speech Act theory (Frank & Goodman, 2012) has been successfully applied in numerous communicative settings, including studies using one-shot web-based language games. Several follow-up studies of the latter, however, suggest that listeners may not behave as pragmatically as originally suggested in those tasks. We investigate whether, in such reference games, listeners’ pragmatic reasoning about an informative speaker is improved by greater exposure to the task, and/or prior experience with being a speaker in this task. While we find limited evidence that increased exposure results in more pragmatic responses, listeners do show increased pragmatic reasoning after playing the role of the speaker. Moreover, we find that only in the Speaker-first condition, participant’s tendency to be an informative speaker predicts their degree of pragmatic behavior as a listener. These findings demonstrate that, in these settings, experience as a speaker enhances the ability of listeners to reason pragmatically, as modeled by RSA.

Speak before you listen: Pragmatic reasoning in multi-trial language games

Typological analysis of the world’s language shows that, of the 6 possible basic word orders, SOV and SVO orders are predominant, a preference supported by experimental studies in which participants improvise gestures to describe events. Silent gesture studies have also provided evidence for natural ordering patterns, where SOV and SVO orders are used selectively depending on the semantics of the event, a finding recently supported by data from natural sign languages. We present an artificial language learning task using gesture to ask to what extent preferences for natural ordering patterns, in addition to biases for regular languages, are at play during learning in the manual modality.

Regularisation, Systematicity and Naturalness in a Silent Gesture Learning Task

This study investigated (1) whether and how English caregivers adjust their speech (i.e., mean length of utterances, lexical diversity, lexical sophistication, sentence types, and deixis) according to different contexts, children’s knowledge, and age, and (2) which aspects of parental speech input predict children’s immediate learning of novel words as well as their vocabulary size. We studied a semi-naturalistic corpus, in which English caregivers talked to their children (3-4 years old) about toys that were present or absent, and known or unknown to the children. We found that caregivers flexibly adjusted various aspects of their speech to maintain an informative and engaging learning environment. Furthermore, we found that rich lexicon and yes-no questions predict better immediate word learning, whereas caregivers' lexical diversity, lexical frequency, the use of Yes-No questions are related to children’s general vocabulary size. In conclusion, higher quality of caregivers’ language predicts better immediate word learning and vocabulary size.

Racial Bias in Emotion Inference: An Experimental Study Using a Word Embedding Method

Downloads

Next from CogSci 2021

Native perception of non-native speech: Speaker accent mitigates penalization for language errors in non-native speech unless the listener is conscientious

Similar lecture

A sequential sampling account of semantic relatedness decisions

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES