United States

The rise of automation has provided an opportunity to achieve higher efficiency in manufacturing processes, yet it often compromises the flexibility required to promptly respond to evolving market needs and meet the demand for customization. Human-robot collaboration attempts to tackle these challenges by combining the strength and precision of machines with human ingenuity and perceptual understanding. In this paper, we conceptualize and propose an implementation framework for an autonomous, machine learning-based manipulator that incorporates human-in-the-loop principles and leverages Extended Reality (XR) to facilitate intuitive communication and programming between humans and robots. Furthermore, the conceptual framework foresees human involvement directly in the robot learning process, resulting in higher adaptability and task generalization. The paper highlights key technologies enabling the proposed framework, emphasizing the importance of developing the digital ecosystem as a whole. Additionally, we review the existent implementation approaches of XR in human-robot collaboration, showcasing diverse perspectives and methodologies. The challenges and future outlooks are discussed, delving into the major obstacles and potential research avenues of XR for more natural human-robot interaction and integration in the industrial landscape.

IEEE RO-MAN 2024

Extended Reality for Enhanced Human-Robot Collaboration: A Human-In-The-Loop Approach

extended reality

human-robot collaboration

machine learning

technical paper

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!
<br><br>

A registration is required to access this event page. If you have not already done so, please use the below link to register for the event. 

Please [register here](https://www.ro-man2024.org/participation/registration).



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

Reinforcement learning (RL) has emerged as a promising paradigm in complex and continuous robotic tasks, however, safe exploration has been one of the main challenges, especially in contact-rich manipulation tasks in unstructured environments. Focusing on this issue, we propose SRL-VIC: a model-free safe
RL framework combined with a variable impedance controller(VIC). Specifically, safety critic and recovery policy networks are pre-trained where safety critic evaluates the safety of the next action using a risk value before it is executed and the recovery policy suggests a corrective action if the risk value is high. Furthermore,
the policies are updated online where the task policy not only achieves the task but also modulates the stiffness parameters to keep a safe and compliant profile. A set of experiments in contact-rich maze tasks demonstrate that our framework outperforms the baselines (without the recovery mechanism and without the VIC),
yielding a good trade-off between efficient task accomplishment and safety guarantee. We show our policy trained on simulation can be deployed on a physical robot without fine-tuning, achieving successful task completion with robustness and generalization.
![](https://assets.underline.io/markdown_image/1/image/58c7cca8c3f39be1154b02ffb93c41f7.png)

SRL-VIC: A Variable Stiffness-based Safe Reinforcement Learning for Contact-rich Robotic Tasks

As robots are increasingly integrated into various aspects of everyday life, it becomes essential to develop intelligent systems capable of providing assistance while maintaining social appropriateness. In this paper, we challenge the prevailing assumption that robots should always offer help, prompting an essential discussion of when robots should offer help. We present a systematic way of considering socially appropriate assistance in human-robot interaction and introduce a theoretical framework that enables robots to discern whether or not to offer help to a human user. We examine the factors that influence the social appropriateness of help, including the relative skill levels between the robot and user and measures for assessing the social value and cost of help. Through a series of illustrative examples, we demonstrate the feasibility of our framework in providing socially appropriate assistance.

Should I Help?: A Skill-Based Framework for Deciding Socially Appropriate Assistance in Human-Robot Interactions

This paper introduces a novel methodology for the partitioning of the social space associated with arbitrary groups of individuals into approachable regions. The proposed method exploits boundary points within pre-identified social spaces and classifies them into three distinct levels based on the field of view between pairs of individuals within the group. The effectiveness of this approach was rigorously assessed across both static and dynamic scenarios, illustrating successful segmentation and classification of approachable regions. Furthermore, the methodology underwent comprehensive testing using publicly available datasets, yielding consistently satisfactory outcomes. This innovative approach represents a substantial contribution to the field of human-robot interaction, showcasing significant potential for future enhancements and advancements.

Social Space Segmentation for Approaching Tasks

Gossip is a human behavior that has been shown to strengthen bonds, trust, and the feeling of inclusion between the gossiper and the person with whom they share the gossip. As humans engage more with social robots, fostering bonds between them is critical for meaningful interactions. In this paper, we investigated how gossiping can affect the perception of group inclusion and trust between a human and a robot. In this between-subjects user study (N = 38), we compared the effects of a robot that gossips to the participant in either a positive or negative way about the experimenter during an interaction. We found that participants in the positive condition reported a significant increase in group inclusion with the robot, while participants in the negative condition did not. We also found that participants' moral trust in the negative condition significantly decreased. Our results suggested that positive gossip can be beneficial to human-robot team cohesion.

The Effects of a Gossiping Robot on Team Cohesion

Humans have the ability to anticipate what will happen in their environment based on perceived information; this ability is often manifested as an externally observable behavioral reaction. As robots become more prevalent in human spaces, robots could attend to these visible anticipatory responses to their actions to assess ``Is this a bad idea?'' In this study, we delved into the potential of human anticipatory reaction recognition to predict outcomes. We conducted a user study wherein 30 participants watched videos of action scenarios and were asked about their anticipated outcome of the situation shown in each video (``good'' or ``bad''). We collected video and audio data of the participants reactions as they were watching these videos. We then carefully analyzed the participants' behavioral anticipatory responses; this data was used to train machine learning models to predict anticipated outcomes based on human observable behavior. Reactions are multimodal, compound and diverse, and we find significant differences in facial reactions. Model performances are around 0.5-0.6 test accuracy, and increase notably when nonreactive participants are excluded from the dataset. We discuss the implications of these findings and future work. This research offers insights into improving the safety and efficiency of human-robot interactions, contributing to the evolving field of robotics and human-robot collaboration.

 “Bad Idea, Right?” Exploring Anticipatory Human Reactions for Outcome Prediction in HRI

For social robots operating in home environments, identifying appropriate parking locations which are &quot;out of the way&quot; is a challenging and multi-faceted problem. This paper proposes a solution to one core aspect of that problem, specifically a model for estimating locations where the robot may block walking paths through narrow spaces. For generality, this model assumes no <i>a priori</i> knowledge about user behaviors or semantic features in the space, and is derived purely from spatial geometry based on a standard 2D occupancy map. An experimental validation based on self-reported parking spot preferences from long-term robot users demonstrates that the proposed model captures 74% of user preferences, outperforming a naive baseline condition in selecting user-preferred parking spots. The proposed method provides a basis for estimating socially-appropriate parking locations for robots operating in the home or other unstructured social spaces and serves as a foundation for developing more sophisticated parking spot preference models in the future.

Where can I park my robot? Modeling out-of-the-way parking spots in the home using room geometry

Robot-moderated group discussions have the potential to facilitate engaging and productive interactions among human participants. Previous work on topic management in conversational agents has predominantly focused on human engagement and topic personalization, with the agent having an active role in the discussion. Also, studies have shown the usefulness of including robots in groups, yet further exploration is still needed for robots to learn when to change the topic while facilitating discussions. Accordingly, our work investigates the suitability of machine-learning models and audiovisual non-verbal features in predicting appropriate topic changes. We utilized interactions between a robot moderator and human participants, which we annotated and used for extracting acoustic and body language-related features. We provide a detailed analysis of the performance of machine learning approaches using sequential and non-sequential data with different sets of features. The results indicate promising performance in classifying inappropriate topic changes, outperforming rule-based approaches. Additionally, acoustic features exhibited comparable performance and robustness compared to the complete set of multimodal features. Our annotated data is publicly available at https://github.com/ghadj/topic-change-robot-discussions-data-2024.

Let's Move On: Topic Change in Robot-Facilitated Group Discussions

The ageing of population led to an increase of mild cognitive impairment cases, causing a struggling for society. Even though cognitive therapy is efficient in counteracting and slowing down the decline, due to the high demand for treatments, it is becoming increasingly difficult to deliver the therapy. In this scenario, the engAGE project aims to build a platform by integrating existing technologies like mobile app and social robot, to innovate cognitive therapy for older people in Europe. In this paper, the usability and the acceptability of the Pepper robot by Italian older people with mild cognitive impairment after 3 months of use in weekly group sessions are analyzed and discussed. Results show that acceptability, expressed in terms of intention to use, perceived usefulness, and positive attitude, overcomes the ease to use. Moreover, the group sessions with Pepper encourage the socialization and the participation of older people, who feel included and like protagonists of the co-design process.

A Robot to Guide Group-Therapy against Cognitive Decline: An Italian Pilot Study

The population of older adults is growing worldwide, and with it, there is a need for appropriate and effective cognitive therapies. Social robots have shown potential as therapeutic tools for older adults, providing companionship, entertainment, and therapeutic support. In particular, storytelling activities promote cognitive stimulation, socio-emotional skills, and simple entertainment. However, conventional storytelling methodologies concerning engagement, personalization, and interactivity exhibit a potential for greater diversity. These traditional approaches often require extensive preparation time, which can limit their feasibility and adaptability in diverse settings. Social robots can help overcome these limitations by providing a more interactive and engaging storytelling experience. In this paper, we present an approach to enhance the storytelling capabilities of the EBO robot, a social robot designed for interaction with older adult people. As a novelty, in addition to integrating the proposed system in the CORTEX cognitive architecture, we offer to include the therapist in the loop and integrate Artificial Intelligence techniques, including a Large Language Model, ChatGPT, to enable the robot to generate natural-sounding and engaging narratives, adjust its interactions based on the cognitive impairments, and align with the interests and preferences of the user. We demonstrate the effectiveness of our approach through a case study involving older adult participants and therapists. Our preliminary results show that the EBO robot, integrating AI techniques, can provide an interactive, customizable, and socially aware storytelling experience that promotes cognitive stimulation, socio-emotional skills, and simple entertainment that is engaging and enjoyable for

AI-Enhanced Social Robots for Older Adults Care: Evaluating the Efficacy of ChatGPT-Powered Storytelling in the EBO Platform

As autonomous functional robots proliferate in public spaces, understanding their acceptance among diverse populations becomes crucial. Traditional models of technology acceptance focus on behavioral and cognitive factors, overlooking the sociocultural and temporal scaffolding of human-robot interaction. To address this gap, we present two themes (Identity Matters, Community Matters) resulting from a qualitative analysis of ethnographic interviews (n = 20) with passersby and vendors who have experienced Starship delivery robots deployed in their communities. Situating the outcomes of our analysis in relation to existing technology acceptance models, we offer four lessons contributing to the efforts in understanding existence and usage-related acceptance of autonomous functional robots in public spaces.

Premium content

Extended Reality for Enhanced Human-Robot Collaboration: A Human-In-The-Loop Approach

Downloads

Next from IEEE RO-MAN 2024

SRL-VIC: A Variable Stiffness-based Safe Reinforcement Learning for Contact-rich Robotic Tasks

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES