United States

In human-robot collaboration, legible intent of the robot is critical to success as it enables the human to more effectively work with and around the robot. Environments where humans and robots collaborate are widely varied and in the real world are most often cluttered. However, prior work in legible motion utilizes primarily environments which are uncluttered. Success in these environments does not necessarily guarantee success in more cluttered environments. Furthermore, the prior work has been primarily performed based on results from robot-human studies and the problem has not been studied from the prospective of what people do to express intent to each other. Therefore, this work addresses a gap in current research into legible robot arm motion in the following ways: first we perform a human-human study in order to establish the factors which humans use to express their intent through body language, and second we perform the study in a cluttered and varied environment. Through the study we showed that the primary factors which people considered are: timing, kinematic parameters, hand gestures, object proximity, etc. The results also showed that legibility is correlated with perceived safety, perceived social intelligence, the collaborators contribution, and trust which further speaks to the importance of legible motion. Future work will utilize the pose data extracted from the study&#39;s video recordings to develop a model for legible motion.

IEEE RO-MAN 2024

Investigating Non-Verbal Cues in Cluttered Environments: Insights Into Legible Motion From Interpersonal Interaction

legible motion

motion planning and navigation in human-centered environments

non-verbal cues and expressiveness

detecting and understanding human activity

In human-robot collaboration, legible intent of the robot is critical to success as it enables the human to more effectively work with and around the robot. Environments where humans and robots collaborate are widely varied and in the real world are most often cluttered. However, prior work in legible motion utilizes primarily environments which are uncluttered. Success in these environments does not necessarily guarantee success in more cluttered environments. Furthermore, the prior work has been primarily performed based on results from robot-human studies and the problem has not been studied from the prospective of what people do to express intent to each other. Therefore, this work addresses a gap in current research into legible robot arm motion in the following ways: first we perform a human-human study in order to establish the factors which humans use to express their intent through body language, and second we perform the study in a cluttered and varied environment. Through the study we showed that the primary factors which people considered are: timing, kinematic parameters, hand gestures, object proximity, etc. The results also showed that legibility is correlated with perceived safety, perceived social intelligence, the collaborators contribution, and trust which further speaks to the importance of legible motion. Future work will utilize the pose data extracted from the study's video recordings to develop a model for legible motion.

technical paper

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!
<br><br>

A registration is required to access this event page. If you have not already done so, please use the below link to register for the event. 

Please [register here](https://www.ro-man2024.org/participation/registration).



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

The human gaze is an important cue to signal intention, attention, distraction, and the regions of interest in the immediate surroundings. Gaze tracking can transform how robots perceive, understand, and react to people, enabling new modes of robot control, interaction, and collaboration. In this paper, we use gaze tracking data from a rich dataset of human motion (THÖR-MAGNI) to investigate the coordination between gaze direction and head rotation of humans engaged in various indoor activities involving navigation, interaction with objects, and collaboration with a mobile robot. In particular, we study the spread and central bias of fixations in diverse activities and examine the correlation between gaze direction and head rotation. We introduce various human motion metrics to enhance the understanding of gaze behavior in dynamic interactions. Finally, we apply semantic object labeling to decompose the gaze distribution into activity-relevant regions.

Human Gaze and Head Rotation During Navigation, Exploration and Object Manipulation in Shared Environments with Robots

Technology-supported learning is an integral part of everyday learning. Adaptive tutoring systems are already widespread, but they often lack the social component of learning. Social robots can provide personalized tutoring as well as social interaction and can thus address this challenge. In this contribution, we present a model to personalize the behavior of a social robot tutor for higher education which has both a theoretical as well as an empirical foundation: Theoretically, it is based on the well-known Self-Determination Theory (SDT) that assumes that intrinsic motivation is strongly related to the fulfillment of three basic human needs, namely the need for autonomy, competence, and relatedness. Empirically it is based on the results of four studies with university students that demonstrate that particularly adaptation on a social level is beneficial. The final model considers the personalization of the robotic tutor's behavior based on the current learning content and its social behavior and also considers a long-term perspective for multi-session learning. Thereby each of the three layers of SDT is addressed in multiple ways. This is one of the first models that contain a long-term perspective and is particularly designed for higher education.

Introducing a Model for (Long-Term) Personalization of the Behavior of a Social Robot Tutor Based on Self-Determination Theory and Empirical Findings

Advanced by rich perception and precise execution, robots possess immense potential to provide professional and customized rehabilitation exercises for patients with mobility impairments caused by strokes. Autonomous robotic rehabilitation significantly reduces human workloads in the long and tedious rehabilitation process. However, training a rehabilitation robot is challenging due to the data scarcity issue. This challenge arises from privacy concerns (e.g., the risk of leaking private disease and identity information of patients) during clinical data access and usage. Data from various patients and hospitals cannot be shared for adequate robot training, further compromising rehabilitation safety and limiting implementation scopes. To address this challenge, this work developed a novel federated joint learning (FJL) method to jointly train robots across hospitals. FJL also adopted a long short-term memory network (LSTM)-Transformer learning mechanism to effectively explore the complex tempo-spatial relations among patient mobility conditions and robotic rehabilitation motions. To validate FJL's effectiveness in training a robot network, a clinic-simulation combined experiment was designed. Real rehabilitation exercise data from 200 patients with stroke diseases (upper limb hemiplegia, Parkinson's syndrome, and back pain syndrome) were adopted. Inversely driven by clinical data, 300,000 robotic rehabilitation guidances were simulated. FJL proved to be effective in joint rehabilitation learning, performing 20% - 30% better than baseline methods.

Federated Joint Learning of Robot Networks in Stroke Rehabilitation

Robots that make use of multi-fingered or fully anthropomorphic end-effectors can engage in highly complex manipulation tasks. However, the choice of a suitable grasp for manipulating an object is strongly influenced by factors such as the physical properties of an object and the intended task. This makes predicting an appropriate grasping pose for carrying out a concrete task notably challenging. At the same time, current grasp type predictors rarely consider the task as a part of the prediction process. This work proposes a learning model that considers the task in addition to an object's visual features for predicting a suitable grasp type. Furthermore, we generate a synthetic dataset by simulating robotic grasps on 3D object models based on the BarrettHand end-effector. With an angular similarity of 0.9 and above, our model achieves competitive prediction results compared to grasp type predictors that do not consider the intended task for learning grasps. Finally, to foster research in the field, we make our synthesized dataset available to the research community.

Incorporation of the Intended Task into a Vision-Based Grasp Type Predictor for Multi-Fingered Robotic Grasping

It is crucial that users are empowered to use the functionalities of a robot to creatively solve problems on the fly. A user who has access to a Reinforcement Learning (RL) based robot may want to use the robot's autonomy and their knowledge of its behavior to complete new tasks. One technique is for the user to take control of some of the robot's action space through teleoperation, allowing the RL policy to simultaneously control the rest. We formalize this type of shared control as Partitioned Control (PC). However, this may not be possible using an out-of-the-box RL policy. For example, a user's control may bring the robot into a failure state from the policy's perspective, causing it to act unexpectedly and hindering the success of the user's desired task. In this work, we formalize this problem and present Imaginary Out-of-Distribution Actions, IODA, an initial algorithm which empowers users to leverage their expectations of a robot's behavior to accomplish new tasks. We deploy IODA in a user study with a real robot and find that IODA leads to both better task performance and a higher degree of alignment between robot behavior and user expectation. We also show that in PC, there is a strong and significant correlation between task performance and the robot's ability to meet user expectations, highlighting the need for approaches like IODA. Code is available at https://github.com/AABL-Lab/ioda_roman_2024

Imagining In-Distribution States: How Predictable Robot Behavior Can Enable User Control Over Learned Policies

Augmented reality multi-user interfaces facilitate communication, coordination and collaboration among teams. Moreover, these interfaces can help to align the team's perceptions and expectations under a shared mental model. This model is a psychological construct that represents the common knowledge, beliefs, and understandings held by team members. In this paper, we study to what extent, if any, the combination of Augmented Reality multi-user interfaces and shared mental models affects human-robot trust. To this end, we developed an Augmented Reality multi-user interface to perform a user study ($N=37$) comparing non-dyadic human-robot interactions with a quadruped robot exhibiting low reliability (Group 3), against dyadic interactions while the robot exhibited high-reliability (Group 1) or low-reliability (Group 2). We made this comparison using validated trust questionnaires relevant to HRI. Our results, obtained via Bayesian data analysis methods, show differences in the distribution of answers between groups 1 and 2. Notably, this difference is smaller between groups 1 and 3, which suggests that the combination of shared mental models and multi-user interfaces holds promise as an effective way to manage and calibrate human-robot trust.

On the Effect of Augmented-Reality Multi-User Interfaces and Shared Mental Models on Human-Robot Trust

The sources of a person's ikigai---their sense of meaning and purpose in life---often change as they age. Reflecting on past and new sources of ikigai may help people renew their sense of meaning as their life circumstances shift. Building on insights from an initial Wizard-of-Oz robot prototype, we describe the design of an autonomous robot that uses a semi-structured conversation format to help older adults reflect on what gives their life meaning and purpose. The robot uses both pre-determined (scripted) and Large Language Model (LLM) generated questions to personalize conversations with older adults around themes of social interaction, planning, accomplishments, goal setting, and the recent past. We evaluated the autonomous robot with 19 older adult participants in a lab setting and at two eldercare facilities. Analysis of the older adults' conversations with the robot and their responses to an evaluative survey allowed us to identify several design considerations for an autonomous robot that can support ikigai reflection. Interweaving simple yet detailed pre-determined questions with LLM-generated follow-up questions yielded enjoyable, in-depth conversations with older adults. We also recognized the need for the robot to be able to offer relevant suggestions when participants cannot recall events and people they find meaningful. These findings aim to further refine the design of an interactive robot that can support users in their exploration of life's purpose.

Let's Talk About You: Development and Evaluation of an Autonomous Robot to Support Ikigai Reflection in Older Adults

To facilitate easy, seamless, and dynamic interactions between humans and social robots, it is important that the robots physical appearance gives clear cues to its affordances. However, little is known about what underlying concepts and assumptions shape laypeoples perception and understanding of different robotic embodiments. To explore what robot-inexperienced users expect robots in different application domains to look like, we drew on Research through Design principles to pilot a tangible design kit with which laypeople could prototype robot designs. 27 participants with no background in robotics were asked to design robots for four different application domains. These participants were then further interrogated about their design choices in structured qualitative interviews. The resulting designs primarily ranged from mechanical to humanoid in appearance, though several animal-like entertainment robots were also created. The inductive thematic analysis of participant interviews revealed complex opinions on anthropomorphism in robotic designs and stressed social robots' potential for customization and accessibility. Our findings provide qualitative insight into the beliefs that underlie laypeoples understanding of robotic design in different contexts. Furthermore, we piloted a design toolkit that allows laypeople at any level of creative ability to design and critically reflect on robotic concepts and embodiments.

Human, Animal, or Machine? A Design-Based Exploration of Social Robot Embodiment with a Creative Toolkit

How a robot moves is among the first things an observer notices when they encounter a robot. While considerable research has investigated the perception of robot body language, no studies yet, to our knowledge, have explored the social effects of how a robot moves through space (its gait) on people's first impressions of a robot. To this end, here we performed two complementary experiments online ($n=98$) and in-person ($n=26$), with the objective of determining the extent to which a quadruped robot's gait influences a) what animal people perceived it to be; and b) its social attributes in terms of warmth, competence, causing discomfort and zoomorphism. Results differed depending on whether participation was online or in-person, with gaits influencing participants' perception more markedly when they encountered the robot in-person. Online, most participants saw the robot as a dog for every gait except one, while in-person participants reported more varied responses. Participants in both studies rated the more active, &quot;bouncy&quot; gait as warmer and less discomforting. In-person participants also consistently rated all gaits as warmer, more competent, less discomforting and more zoomorphic than did online participants. The study supports findings that in-person exposure and embodiment affect a robot's social perception and further suggests that gait may have a limited effect as well.

Walking the Line: Assessing the Role of Gait in a Quadruped Robot's Perception

The need for robots to explain their failures grows as the variety and number of robots deployed in public, homes, and work environments increases. This paper extends our prior work utilizing explanation templates by comparing those Templated explanations to Generative explanations created by a Large Language Model. Our study surprisingly reveals that Templated explanations result in similar or higher perceived intelligence and trust while also being more understandable. Through our findings, we aim to provide considerations for effective robot explanation systems, ultimately enabling people to be able to understand and provide assistance to robots that have encountered unforeseen circumstances.

Premium content

Investigating Non-Verbal Cues in Cluttered Environments: Insights Into Legible Motion From Interpersonal Interaction

Downloads

Next from IEEE RO-MAN 2024

Human Gaze and Head Rotation During Navigation, Exploration and Object Manipulation in Shared Environments with Robots

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES