Canada

Sign languages are visual-spatial languages, articulated not only by hands but also with facial expressions and other body parts. Facial expressions, in particular, have important grammatical roles, such as defining whether a sentence is a question or not and differentiating the meaning of a sign when assuming the role of a quantifier. Despite the importance of facial expressions, technologies to recognize and generate sign language mainly focus on manual signs, ignoring the information from facial expressions. In this work, we focus on collecting and annotating data to evaluate the role of facial expressions in communicating adjectives. We study the role of facial muscle activity to express the intensities of manual signs and show how modeling facial action units change the overall meaning of a sentence. We also explore how facial expressions allow a deeper understanding of sign language when included in the design of machine learning models.

CogSci 2022

Look For Adjectives In the Face: How Facial Expressions Contribute To Meaning In Signed Languages

poster

Join us for CogSci2022! Our 44th Annual Conference will be centered around the theme of diversity, with an accompanying range of keynotes and invited speakers who themselves highlight the diversity of our field. Our three symposia are Cognition across Diverse Populations, Cognition within Diverse Societies, and Global Challenges for Cognitive Science. The conference is being offered in a fully hybrid environment for onsite and online attendees.
 
 [**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/07/Cogsci-2022-PAAG-Detailed-July-12final-1.pdf)
 [**The Detailed Program can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/08/DeArmond-CogSci2022-program-v113-final.pdf)

This conference requires a registration to attend. Please use the link below to secure your ticket. The website will be available to all registered attendees on July 25th. 

[Register to Attend](https://cognitivesciencesociety.org/registration/)

Registration Is Required

The 44th Annual Meeting of the Cognitive Science Society highlighted research on the theme Cognitive Diversity in addition to the full breadth and diversity of research topics offered by the society’s membership. 





A non-trivial number of the world’s languages have grammatical gender linked to biological sex. It is known that grammatical gender classification is not arbitrary, often following a semantic “core”. We investigate whether grammatical gender systems exhibit gender bias in different semantic domains, particularly terminologies of technology (e.g., microcomputer), a domain that has been considered male-dominant, and disease terminologies (e.g., hepatitis), a domain susceptible to negative connotations. In an initial
analysis with five languages, we found a baseline bias towards masculine classification of nouns compared to feminine. The technology domain showed no significant bias in grammatical gender with respect to the baseline, but two of the studied languages, French and German, showed statistically significant biases towards feminine classification in disease terminologies, while Arabic showed a significant bias towards masculine classification. These initial findings suggest that languages may be subject to domain general and specific gender biases through overt grammatical features.

Gender bias in grammatical gender systems across languages

Developing accurate models of word meaning requires good empirical evidence about what words mean. I investigate how English verbs encode relationships between event participants, focusing on how verbs encode instruments (e.g., does slice specify that a tool must be used for slicing?). I compared two commonly used indices of verb meaning: linguistic judgments and sentence completions. Although these two indices were moderately correlated for a small sample of verbs, they were only weakly correlated when a larger sample of verbs was tested. These results indicate that the particular context of a task can strongly influence how meaning affects behavior. Dominant models of verb meaning fail to fully account for the results (either a logical entailment model or a cue-based model).

Measuring Meaning: Alignment and Misalignment Across Indices of Verb Semantics

The paper investigates whether English native speakers (NSs)’ and non-native speakers (NNSs)’ online processing of prepositional phrases were primed by schematic diagrams and whether the priming effects were modulated by different prepositions and senses. 125 adult NS and NNS participants judged on over and in phrases that encode spatial and extended senses in a semantic priming task. Results showed an inhibitory effect of diagrams on NNSs’ processing but a marginal facilitatory effect on NSs’ processing. In addition, the inhibitory effect of diagrams only applied to the processing of extended senses, but not spatial senses. The current findings indicate that NNSs’ processing of English PPs could be negatively influenced by schematic diagrams particularly when the target phrases are more complex (Boers, 2011), presumably due to NNSs’ more limited cognitive capacity of processing complex second language expressions under time pressure than NSs.

Schematic Diagrams and Semantic Processing of English Prepositional Polysemy

The paper explores the impact of social role assignment and the corresponding payoff distribution on cooperation in the Prisoner’s dilemma following the types of relations according to Fiske’s relational models’ theory: communal sharing, authority ranking, equality matching, and market pricing. The corresponding roles and payoff distribution are teammates (each player receives the sum of the payoffs), partners (each player receives half of the sum of the payoffs), boss and subordinate (the boss receives 2/3 and the subordinate 1/3 of the sum of the payoffs), and opponents (each player receives the standard payoff). The results show that in the teammates’ and partners’ conditions cooperation was significantly higher than in the other conditions. Surprisingly, the results for the boss and subordinate condition although sharing a similar payoff distribution rule to the teammates’ and partners’ conditions were more similar to the opponents’ condition with significantly lower cooperation rate.

Can Social Relations Influence Cooperation in Prisoner’s Dilemma?

The Shapley value is one of the most important normative division schemes in cooperative game theory, satisfying basic axioms. However, some allocation according to the Shapley value may seem unfair to humans.
In this paper, we develop an automatic method that generates intuitive explanations for a Shapley-based payoff allocation, which utilizes the basic axioms.
Given any coalitional game, our method decomposes it to sub-games, for which it is easy to generate verbal explanations, and shows that the given game is composed of the sub-games. 
Since the payoff allocation for each sub-game is perceived as fair, the Shapley-based payoff allocation for the given game should seem fair as well.
We run an experiment with 630 human participants and show that when applying our method, humans perceive the Shapley-based payoff allocation as more fair than the Shapley-based payoff allocation without any explanation or with explanations generated by other methods.

Improving the Perception of Fairness in Shapley-Based Allocations

Linguistic similarities increase cooperative behavior between people, while linguistic differences can reduce cooperativeness, and even promote aversion towards outsiders. This may not only be true for sharing the same native language, but also for more subtle similarities such as using similar syntactic constructions. In our experiment, participants communicated with two interaction partners (which were in fact standardized chat bots), one of which syntactically aligned their language with the participants and one of which didn’t. Afterwards, participants decided with which of the two partners they would like to play a cooperation game that determined the financial bonus that they would receive for participation. Against our predictions, linguistically aligned communication partners were not preferred as cooperation partners over non-aligned communication partners. This suggests that formal linguistic similarity may be less relevant for the outcome of negotiations than previously expected, and has real-life implications for conflict resolution, especially in intercultural communication settings.

Does linguistic alignment promote cooperation?

The Covid-19 pandemic altered workplaces. For those with ‘office jobs,’ this meant working ‘virtually,’ or remotely, from home. This transition forced organizations and workplaces to exercise flexibility, adapt workflows and rely on Information and Communication Technologies (ICTs) to work remotely. However, Blind and Partially Sighted Individuals (BPSI) face challenges accessing work digitally and remote communications through ICTs. In response, we report on the results of our longitudinal participatory design study investigating the impact of working and training over a distance for BPSI. What emerged is a conceptual model to assist in understanding how ICT interfaces convey spatial-topological cues for the construction of shared intentionality in virtual work environments. The implications of our model could be significant, as it aids understanding of what is lost and gained when transitioning to virtual work environments. This could inform the development of ICTs with cross-sensory interaction and national accessibility policies for the workplace.

How Virtual Work Environments Convey Perceptual Cues to Foster Shared Intentionality During Covid-19 for Blind and Partially Sighted Employees

The purpose of this paper is to present a novel conceptualisation of an intuitive, primitive form of trust termed proto-trust. This concept is proposed in order to account for the many different senses, types and domains in which trust has traditionally been defined and theorised. A brief review of the literature on affective and intuitive trust is presented, informing the definition and formalisation of proto-trust. Following this, a preliminary empirical investigation of proto-trust is described, where intuitive trust assessments are compared to analytical trust decisions, under various attribution prompts. Results showed effects of attribution prompts on changes to trust assessments from intuitive to deliberative decisions. In addition, qualitative data are presented for the various reasons participants gave for their trust decisions. One of these reasons (emotional reaction) was found to affect the degree of difference between intuitive and deliberative trust assessments.

Proto-trust and trust attribution: a theory of intuitive, affective forms of trust and the means by which trust decisions are made

The external provision of examples has proven the most successful approach to aid learning and application of declarative concepts (i.e., abstract concepts denoted by key terms and short definitions that can be applied to a wide variety of scenarios). The current experiment sought to further this line of research by exploring the effect of using thematically varied examples on learners' ability to classify novel exemplars and near misses of five system principles that cut across thematic domains. Results revealed that thematic variation increased learners’ ability to reject near-misses and, more crucially, to classify novel exemplars from domains not covered by the studied examples. The fact that this enhanced flexibility was unaccompanied by poorer performance in rejecting near misses or classifying new items from domains covered by the learned examples renders this strategy readily applicable in instructional settings. We end by discussing possible mechanisms that could potentially explain the observed advantage of thematically varied examples.

Defending Diversity: Providing Examples from Different Domains Enhances Application of System Principles Beyond the Domains Covered by the Examples

The learning of rule-plus-exception categories relies on pattern integration and differentiation, but how the representations of rule-followers and exceptions develop through these two operations remains obscure. Here, we inspected the representational shifts in rule-plus-exception category learning by fitting a computational model to behavioral categorization data. We found that exceptions were differentiated from rule-followers within and between categories through learning. The distanced rule-follower and exception representations in each category formed distinct clusters that together constituted a hierarchically structured categorical representation. Moreover, exception learning increased the representational overlap between rule-followers of opposite categories, thereby blurring the category boundary. Our findings illuminate the representational dynamic underlying the acquisition of rule-plus-exception categories and highlight the roles of pattern integration and differentiation in category learning.

Premium content

Look For Adjectives In the Face: How Facial Expressions Contribute To Meaning In Signed Languages

Downloads

Next from CogSci 2022

Gender bias in grammatical gender systems across languages

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES