The present paper addresses the study of cross-linguistic phonosemantic correspondences within a deep learning framework. An LSTM-based Recurrent Neural Network is trained to associate the phonetic representation of a word, encoded as a sequence of feature vectors, to its corresponding semantic representation in a multilingual and cross-family vector space. The processing network is then tested, without further training, in a language that does not appear in the training set and belongs to a different language family. The performance of the model is evaluated through a comparison with a monolingual and mono-family upper bound and a randomized baseline. After the assessment of the network's performance, the distribution of phonosemantic properties in the lexicon is inspected in relation to different (psycho)linguistic variables, showing a link between lexical non-arbitrariness and semantic, syntactic, pragmatic, and developmental factors.

A Layered Bridge from Sound to Meaning: Investigating Cross-linguistic Phonosemantic Correspondences

We model the sense-making process of diagrams as conceptual blends of the diagrams’ geometric configurations with apt image schemas.We specify image schemas and geometric configurations with typed FOL theories. In addition, for the latter, we utilise some Qualitative Spatial Reasoning formalisms. Using an algebraic specification language, we can compute the conceptual blends of image schemas and geometry as category-theoretic colimits.We show through several examples how this model captures the sort of direct inferences we confer to diagrammatic representations due to our embodied cognition. We argue that this approach to sense-making might be of value for the design and application of diagrammatic and graphical visualisations, as well as for AI in general.

Modelling the Sense-Making of Diagrams Using Image Schemas

Non-native speakers have been found to be penalized for their accent and grammatical errors. However, little is known about whether and how accent and grammaticality interact to influence native listeners’ perception of non-native speech, and whether listeners’ personality plays a role in this. We examined these questions by relating the acceptability scores of 40 English speech stimuli rated by 60 British listeners (30 female; 30 unfamiliar with accented speech) to factors including speech accent (British vs. Polish), grammaticality (well-formed vs. error-filled) and listener personality. The results suggest that non-native accent “protects” the speaker from being penalized for grammatical errors unless the listener has a certain personality profile: compared to non-native accented speech, the acceptability ratings of native accented speech showed a larger decrease when grammatical errors were present, yet listeners who were more conscientious, extravert or agreeable tended to give lower acceptability ratings to non-native accented speech, regardless of grammaticality.

Native perception of non-native speech: Speaker accent mitigates penalization for language errors in non-native speech unless the listener is conscientious

Deese/Roediger/McDermott (DRM) list words can share either semantic relatedness or phonological resemblance with their critical distractor. We review three lines of evidence in which semantic and phonological DRM illusions have been compared: (a) studies in which the two illusions were tracked in populations with different semantic or surface memory abilities; (b) studies that investigated the effects of manipulations that target semantic content or surface content or both; and (c) studies that examined hybrid forms of the illusion in which there was both semantic and surface resemblance between false memories and actual experiences. The three lines of evidence showed that semantic and phonological DRM illusions display dissociative patterns in most instances, indicating that they are two distinct types of false memories. We also discussed how the two major theories of the DRM illusion, fuzzy-trace theory and the activation/monitoring framework, account for the underlying mechanisms for the semantic and phonological illusions.

Semantic and Phonological False Memory: A Review of Theory and Data

Egocentric space perception is multimodal, closely tied to action and bodily movements and has an inherent phenomenal dimension. One prominent account, provided by Rick Grush, has postulated posterior parietal cortex as key neural area. Computational model based on Kalman filter has been proposed to account for the operation of this brain region, underscoring the importance of bodily skills for perceiving spatial properties.
The current study provides a first direct simulation of this model in a semi-realistic 3D virtual environment. The goal of the simulation was to develop an agent with a realistic ability for egocentric space perception based on a neural approximation of Kalman filter. To achieve this goal, we use machine learning techniques, with a strong focus on unsupervised methods of reinforcement learning.
Resulting agent is tested behaviorally on ecologically plausible tasks to evaluate its internal, learned representations. Poster presents simulation results and discusses the model.

Modeling "spatial purport of perceptual experience": egocentric space perception in a semi-realistic 3D virtual environment

Visual and auditory channels have different affordances and this is mirrored in what information is available for linguistic encoding. The visual channel has high spatial acuity, whereas the auditory channel has better temporal acuity. These differences may lead to different conceptualizations of events and affect multimodal language production. Previous studies of motion events typically present visual input to elicit speech and gesture. The present study compared events presented as audio-only, visual-only, or multimodal (visual+audio) input and assessed speech and co-speech gesture for path and manner of motion in Turkish. Speakers with audio-only input mentioned path more and manner less in verbal descriptions, compared to speakers who had visual input. There was no difference in the type or frequency of gestures across conditions, and gestures were dominated by path-only gestures. This suggests that input modality influences speakers’ encoding of path and manner of motion events in speech, but not in co-speech gestures.

Sensory Modality of Input Influences the Encoding of Motion Events in Speech But Not Co-Speech Gestures

Culturally transmitted traits that have deleterious effects on health-related traits can be regarded as cultural pathogens. A cultural pathogen can produce coupled dynamics with its associated health-related traits, so that understanding the dynamics of a health-related trait benefits from consideration of the dynamics of the associated cultural pathogen. Here, we treat anti-vaccine sentiment as a cultural pathogen, modelling its ‘infection’ dynamics with the infection dynamics of the associated vaccine-preventable disease. In a coupled susceptible–infected–resistant (SIR) model, consisting of an SIR model for the anti-vaccine sentiment and an interacting SIR model for the infectious disease, we explore the effect of anti-vaccine sentiment on disease dynamics. We find that disease endemism is contingent on the presence of the sentiment, and that presence of sentiment can enable diseases to become endemic when they would otherwise have disappeared. Furthermore, the sentiment dynamics can create situations in which the disease suddenly returns after a long period of dormancy. We study the effect of assortative sentiment-based interactions on the dynamics of sentiment and disease, identifying a tradeoff whereby assortative meeting aids the spread of a disease but hinders the spread of sentiment. Our results can contribute to finding strategies that reduce the impact of a cultural pathogen on disease, illuminating the value of cultural evolutionary modelling in the analysis of disease dynamics.

Underline digital video library

A Layered Bridge from Sound to Meaning: Investigating Cross-linguistic Phonosemantic Correspondences

Modelling the Sense-Making of Diagrams Using Image Schemas

Native perception of non-native speech: Speaker accent mitigates penalization for language errors in non-native speech unless the listener is conscientious

Semantic and Phonological False Memory: A Review of Theory and Data

Modeling "spatial purport of perceptual experience": egocentric space perception in a semi-realistic 3D virtual environment

Sensory Modality of Input Influences the Encoding of Motion Events in Speech But Not Co-Speech Gestures

Modelling anti-vaccine sentiment as a cultural pathogen

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES