United States

Current approaches to empathetic response generation typically encode the entire dialogue history directly and put the output into a decoder to generate friendly feedback. These methods focus on modelling contextual information but neglect capturing the direct intention of the speaker. We argue that the last utterance in the dialogue empirically conveys the intention of the speaker. Consequently, we propose a novel model named InferEM for empathetic response generation. We separately encode the last utterance and fuse it with the entire dialogue through the multi-head attention based intention fusion module to capture the speaker&#39;s intention. Besides, we utilize previous utterances to predict the last utterance, which simulates human&#39;s psychology to guess what the interlocutor may speak in advance. To balance the optimizing rates of the utterance prediction and response generation, a multi-task learning strategy is designed for InferEM. Experimental results demonstrate the plausibility and validity of InferEM in improving empathetic expression.

CogSci 2023

InferEM: Inferring the Speaker&#39;s Intention for Empathetic Dialogue Generation

Current approaches to empathetic response generation typically encode the entire dialogue history directly and put the output into a decoder to generate friendly feedback. These methods focus on modelling contextual information but neglect capturing the direct intention of the speaker. We argue that the last utterance in the dialogue empirically conveys the intention of the speaker. Consequently, we propose a novel model named InferEM for empathetic response generation. We separately encode the last utterance and fuse it with the entire dialogue through the multi-head attention based intention fusion module to capture the speaker's intention. Besides, we utilize previous utterances to predict the last utterance, which simulates human's psychology to guess what the interlocutor may speak in advance. To balance the optimizing rates of the utterance prediction and response generation, a multi-task learning strategy is designed for InferEM. Experimental results demonstrate the plausibility and validity of InferEM in improving empathetic expression.

InferEM: Inferring the Speaker's Intention for Empathetic Dialogue Generation

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Which auditory features drive everyday listeners’ preferences for and emotional experiences of their favorite music? To test how various features contribute to listeners’ experiences of popular music, we created a novel corpus of covers of 50 popular songs from 2008 to 2019, capturing a wide variety of genres and styles. Cover versions of popular songs offer a rich space of natural musical stimuli that match original songs on some dimensions (e.g., lyrics) but not others (e.g., genre). We pseudo-randomly selected one cover per song and asked subjects to rate similarity between clips of original and cover versions. Other subjects rated emotional valence and arousal of the same clips. We found a relationship between similarity ratings and differences in arousal ratings between originals and covers (p = 0.00069) but not differences in valence ratings (p = 0.27), suggesting that similarity ratings are more driven by perceived arousal differences.

Quantifying the Influence of Musical Features on Perceptual Similarity of Popular Songs

Previous research has found emotional colour connotations (red=negative, green=positive) to rapidly impact emotional judgements of faces presented in different colour contexts. However, it remains unclear whether this influence is reciprocal. We consider differences in processing speed and difficulty between perceptual (colour) and conceptual (emotion) stimuli to determine whether emotion information influences colour processing. We examined whether emotional face primes (angry/happy/neutral) influence red-green colour judgements and if this effect is greater under increased ambiguity of the colour stimuli. The findings suggest that primed emotional information can guide colour judgements. Specifically, happy face primes led to better categorisation accuracy for predominantly green stimuli, while angry face primes led to better categorisation accuracy for predominantly red stimuli. For ambiguous colour stimuli, a categorisation response bias was found for green following happy face primes. Our findings provide support for a reciprocal influence of emotional stimuli on colour judgements.

Emotional face priming influences colour judgements

A large body of evidence demonstrates that emotion impacts memory. Although visual information dominates emotional memories, previous studies have not examined the role of visual imagery as an individual difference variable in the representation of emotional memories. This study examines the role of visual imagery skills (namely, object and spatial imagery) on emotional memories. Participants (N = 115) recalled positive, negative, and neutral events in response to the cue words and then rated the phenomenological characteristics of each event. Event accounts were coded for episodic detail categories (event, place, perceptual, time, emotion-thought details). The results showed that visual imagery skills contributed to the remembrance of the episodic details of positive memories and the phenomenology of both positive and negative events. Overall, this study emphasizes the importance of considering the individual differences in memory research and highlights the differences between emotional and neutral events.

Exploring the Role of Visual Imagery in the Recall of Emotional Autobiographical Memories

In many cultures, adults use simple, slow, and dynamic speech when talking to infants ("parentese," or infant-directed speech) and make expansive, repetitive movements when demonstrating object properties to infants ("motionese," or infant-directed actions). These modifications enhance infants’ attention to and learning about language and goal-directed actions. Adults’ interactions with infants are also full of emotions –– do adults also modify their emotional expressions when interacting with infants? Here we showed parents of infants (aged 6 to 14 months; N = 25) emotion-evoking pictures including colorful bubbles, adorable stuffed animals, yummy snacks, broken toys, dangerous fire, and rotten fruits. We asked parents to describe their feelings about these pictures either to their infant or to an adult partner (i.e., an experimenter). While the parents’ use of emotion words did not differ between conditions, their emotional expressions did: Their infant-directed emotional expressions were more positive when they discussed positive pictures and more negative when they discussed negative pictures compared to their adult-directed emotional expressions. These findings suggest that besides "parentese" and "motionese," there is also a unique form of emotional communication in parent-child interaction – "emotionese."

Adults tailor their emotional expressions to infants through "emotionese"

Previous research suggests that negative emotions invoked by failure feedback might lead people to tune out from the task, which is detrimental to their learning. However, failure feedback is pervasive in the real world and we need to identify ways we can learn from it optimally. In the current study, the participants’ (n = 218) task expectations were randomly set to be easy or hard. Then, the participants solved a novel type of equation problems that involved manipulation of researcher-invented symbols, followed by either success (“You solved the equations CORRECTLY!”) or failure feedback (“You solved the equations INCORRECTLY!”). Next, the participants were provided instruction about the rules of the equation tasks and solved posttest questions across two rounds. Across different learning outcomes, we identify the cases in which the influence of feedback is moderated by task difficulty expectations (on identical items), failure feedback results in equally high performance with success feedback (on isomorphic items), and participants learn better when they receive failure than success feedback (at a new independent task). We conclude that the tune-out reactions to failure during feedback might be diminished, and even be reversed, after feedback.

Influences of Task Expectations and Failure Feedback on Learning from Subsequent Tasks

The present study investigated the effect of mindfulness on response inhibition, and its interaction with emotions. Individuals with prior experience in mindfulness meditation and those without any meditation experience participated. During a stop signal task, participants responded to a target stimulus on go trials and inhibited responding on stop trials. An emotional face (angry, happy, or neutral) always preceded the target stimulus, which was to be ignored. Participants also completed self-reports associated with affect, awareness, and impulsivity. There were no group differences on any self-report. The task results showed that happy faces enhanced response inhibition in non-meditators, whereas angry faces enhanced response inhibition in mindfulness meditators. Results are explained through functional perspectives of how mindfulness influences attentional resource deployment to emotional stimuli, affecting response processes over time. The findings contribute to understanding the dynamic patterns through which mindfulness modulates the attention-emotion interface to promote meaning in the face of difficulty.

Mindfulness Skill Enhances Inhibition of Irrelevant Negative Emotions: Evidence from a Stop Signal Task

Women are twice as likely as men to suffer from generalized anxiety disorder. Inhibition is the most studied executive function in anxiety disorders and research has shown that elevated levels of anxiety are related to slower response times on tasks assessing inhibition. The aim of the present study was to compare inhibitory control toward emotional words, including those eliciting anxiety, in naturally cycling (NC) women and women taking different kinds of oral contraceptives (OC), androgenic and anti-androgenic OC. We present two phases of one study: a cross-sectional phase with 240 women and a daily diary phase with 89 women. Inhibitory control was measured by an Emotional Stroop task in the two phases. In both phases, there was no difference in anxiety levels between groups. However, the cross-sectional study showed that in women taking anti-androgenic OC, the higher the level of anxiety, the longer the reaction time on the Stroop task. The daily diary study showed that anti-androgenic OC users had slower response times on all the categories of words, compared to androgenic OC users. Further studies should focus on the link between different kinds of OC and inhibitory control as it can be linked to anxiety symptoms in women.

Impaired Inhibitory Control in Women Taking Anti-Androgenic Oral Contraceptives – Implications for Anxiety

This paper proposes and tests a method to study people’s blame attributions displayed in ‘Tweets’. Artificial intelligence (AI) incidents were chosen as they are a contemporary topic that is often talked about on Twitter, and are a “problem of many hands”, where multiple different agents bring about an outcome. The benefit of the method is that it was able to replicate and contextualize findings from experimental settings. For example, AIs get blamed more for negative outcomes, whilst humans get blamed for their intention. Another benefit of the approach was that it was able to identify novel factors that haven't been previously studied experimentally. For example, people were willing to directly blame data for being biased and blamed AI for replacing humans. The proposed method can serve as a way of avoiding entrenched measurement bias within a specific field by allowing researchers to update their frameworks with novel factors.


Blame attribution in human-AI and human-only systems: Crowdsourcing judgments from Twitter

In this study, we present a stimulus set that contains 80 action videos depicting 40 action exemplars (communicative and non-communicative), each performed by a social robot and a female actor with the same manner, timing, and number of repetitions. We describe the creation of the video stimuli, the normative tests conducted to validate the stimulus set (N=438), and the data annotation process. We compute an H score for each action based on entropy, which takes into account the homogeneity of the intended labels for the observers of the actions. We also report the confidence level of the participants and the frequencies of the action classifications. We discuss the implications of the results of this unique and standardized source that is suitable to use in various research areas in cognitive science, not limited to human-robot interaction but also social cognition and cognitive neuroscience.

A Set of Communicative and Noncommunicative Action Video Stimuli for Human-Robot Interaction Research

This study investigated the empathy of humans toward non-biological agents like robots and compared it to their empathy toward their conspecifics. 90 participants watched a human and a robot in painful scenarios (32 short videos created with animation techniques) and rated the intensity of pain experienced by the agent. The scenarios were created by manipulating the type of agent experiencing pain (human or robot), the source of pain (object or person), and the type of pain (physical or emotional). Results showed that humans were perceived to feel more pain than robots. In addition, the perceived pain scores were higher for scenarios where the source of pain was an object compared to a person, however, this depended on the type of pain. Emotional pain scores showed significant differences between the source of pain being an object versus a person, but physical pain scores did not depend on the source of pain.

Perceived pain of humans and robots: An exploration of the effect of agent, pain source, and pain type in pain attribution to biological and non-biological agents

Premium content

InferEM: Inferring the Speaker's Intention for Empathetic Dialogue Generation

Next from CogSci 2023

Quantifying the Influence of Musical Features on Perceptual Similarity of Popular Songs

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES