United States

Robots that make use of multi-fingered or fully anthropomorphic end-effectors can engage in highly complex manipulation tasks. However, the choice of a suitable grasp for manipulating an object is strongly influenced by factors such as the physical properties of an object and the intended task. This makes predicting an appropriate grasping pose for carrying out a concrete task notably challenging. At the same time, current grasp type predictors rarely consider the task as a part of the prediction process. This work proposes a learning model that considers the task in addition to an object&#39;s visual features for predicting a suitable grasp type. Furthermore, we generate a synthetic dataset by simulating robotic grasps on 3D object models based on the BarrettHand end-effector. With an angular similarity of 0.9 and above, our model achieves competitive prediction results compared to grasp type predictors that do not consider the intended task for learning grasps. Finally, to foster research in the field, we make our synthesized dataset available to the research community.

IEEE RO-MAN 2024

Incorporation of the Intended Task into a Vision-Based Grasp Type Predictor for Multi-Fingered Robotic Grasping

vision-based

multi-finger gripper

grasp-type prediction

21st edition of EANN (International Conference on Engineering Applications of Neural Networks) should be held in Halkidiki, Greece. Due to current Covid-19 situation the conference will be held virtually and Underline will host a repository of video talks and preserve them for the future. Videos will be released on Underline one day after the conference. 

EANN 2020

Robots that make use of multi-fingered or fully anthropomorphic end-effectors can engage in highly complex manipulation tasks. However, the choice of a suitable grasp for manipulating an object is strongly influenced by factors such as the physical properties of an object and the intended task. This makes predicting an appropriate grasping pose for carrying out a concrete task notably challenging. At the same time, current grasp type predictors rarely consider the task as a part of the prediction process. This work proposes a learning model that considers the task in addition to an object's visual features for predicting a suitable grasp type. Furthermore, we generate a synthetic dataset by simulating robotic grasps on 3D object models based on the BarrettHand end-effector. With an angular similarity of 0.9 and above, our model achieves competitive prediction results compared to grasp type predictors that do not consider the intended task for learning grasps. Finally, to foster research in the field, we make our synthesized dataset available to the research community.

technical paper

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!
<br><br>

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

A registration is required to access this event page. 



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

It is crucial that users are empowered to use the functionalities of a robot to creatively solve problems on the fly. A user who has access to a Reinforcement Learning (RL) based robot may want to use the robot's autonomy and their knowledge of its behavior to complete new tasks. One technique is for the user to take control of some of the robot's action space through teleoperation, allowing the RL policy to simultaneously control the rest. We formalize this type of shared control as Partitioned Control (PC). However, this may not be possible using an out-of-the-box RL policy. For example, a user's control may bring the robot into a failure state from the policy's perspective, causing it to act unexpectedly and hindering the success of the user's desired task. In this work, we formalize this problem and present Imaginary Out-of-Distribution Actions, IODA, an initial algorithm which empowers users to leverage their expectations of a robot's behavior to accomplish new tasks. We deploy IODA in a user study with a real robot and find that IODA leads to both better task performance and a higher degree of alignment between robot behavior and user expectation. We also show that in PC, there is a strong and significant correlation between task performance and the robot's ability to meet user expectations, highlighting the need for approaches like IODA. Code is available at https://github.com/AABL-Lab/ioda_roman_2024

Imagining In-Distribution States: How Predictable Robot Behavior Can Enable User Control Over Learned Policies

Augmented reality multi-user interfaces facilitate communication, coordination and collaboration among teams. Moreover, these interfaces can help to align the team's perceptions and expectations under a shared mental model. This model is a psychological construct that represents the common knowledge, beliefs, and understandings held by team members. In this paper, we study to what extent, if any, the combination of Augmented Reality multi-user interfaces and shared mental models affects human-robot trust. To this end, we developed an Augmented Reality multi-user interface to perform a user study ($N=37$) comparing non-dyadic human-robot interactions with a quadruped robot exhibiting low reliability (Group 3), against dyadic interactions while the robot exhibited high-reliability (Group 1) or low-reliability (Group 2). We made this comparison using validated trust questionnaires relevant to HRI. Our results, obtained via Bayesian data analysis methods, show differences in the distribution of answers between groups 1 and 2. Notably, this difference is smaller between groups 1 and 3, which suggests that the combination of shared mental models and multi-user interfaces holds promise as an effective way to manage and calibrate human-robot trust.

On the Effect of Augmented-Reality Multi-User Interfaces and Shared Mental Models on Human-Robot Trust

The sources of a person's ikigai---their sense of meaning and purpose in life---often change as they age. Reflecting on past and new sources of ikigai may help people renew their sense of meaning as their life circumstances shift. Building on insights from an initial Wizard-of-Oz robot prototype, we describe the design of an autonomous robot that uses a semi-structured conversation format to help older adults reflect on what gives their life meaning and purpose. The robot uses both pre-determined (scripted) and Large Language Model (LLM) generated questions to personalize conversations with older adults around themes of social interaction, planning, accomplishments, goal setting, and the recent past. We evaluated the autonomous robot with 19 older adult participants in a lab setting and at two eldercare facilities. Analysis of the older adults' conversations with the robot and their responses to an evaluative survey allowed us to identify several design considerations for an autonomous robot that can support ikigai reflection. Interweaving simple yet detailed pre-determined questions with LLM-generated follow-up questions yielded enjoyable, in-depth conversations with older adults. We also recognized the need for the robot to be able to offer relevant suggestions when participants cannot recall events and people they find meaningful. These findings aim to further refine the design of an interactive robot that can support users in their exploration of life's purpose.

Let's Talk About You: Development and Evaluation of an Autonomous Robot to Support Ikigai Reflection in Older Adults

To facilitate easy, seamless, and dynamic interactions between humans and social robots, it is important that the robots physical appearance gives clear cues to its affordances. However, little is known about what underlying concepts and assumptions shape laypeoples perception and understanding of different robotic embodiments. To explore what robot-inexperienced users expect robots in different application domains to look like, we drew on Research through Design principles to pilot a tangible design kit with which laypeople could prototype robot designs. 27 participants with no background in robotics were asked to design robots for four different application domains. These participants were then further interrogated about their design choices in structured qualitative interviews. The resulting designs primarily ranged from mechanical to humanoid in appearance, though several animal-like entertainment robots were also created. The inductive thematic analysis of participant interviews revealed complex opinions on anthropomorphism in robotic designs and stressed social robots' potential for customization and accessibility. Our findings provide qualitative insight into the beliefs that underlie laypeoples understanding of robotic design in different contexts. Furthermore, we piloted a design toolkit that allows laypeople at any level of creative ability to design and critically reflect on robotic concepts and embodiments.

Human, Animal, or Machine? A Design-Based Exploration of Social Robot Embodiment with a Creative Toolkit

How a robot moves is among the first things an observer notices when they encounter a robot. While considerable research has investigated the perception of robot body language, no studies yet, to our knowledge, have explored the social effects of how a robot moves through space (its gait) on people's first impressions of a robot. To this end, here we performed two complementary experiments online ($n=98$) and in-person ($n=26$), with the objective of determining the extent to which a quadruped robot's gait influences a) what animal people perceived it to be; and b) its social attributes in terms of warmth, competence, causing discomfort and zoomorphism. Results differed depending on whether participation was online or in-person, with gaits influencing participants' perception more markedly when they encountered the robot in-person. Online, most participants saw the robot as a dog for every gait except one, while in-person participants reported more varied responses. Participants in both studies rated the more active, &quot;bouncy&quot; gait as warmer and less discomforting. In-person participants also consistently rated all gaits as warmer, more competent, less discomforting and more zoomorphic than did online participants. The study supports findings that in-person exposure and embodiment affect a robot's social perception and further suggests that gait may have a limited effect as well.

Walking the Line: Assessing the Role of Gait in a Quadruped Robot's Perception

The need for robots to explain their failures grows as the variety and number of robots deployed in public, homes, and work environments increases. This paper extends our prior work utilizing explanation templates by comparing those Templated explanations to Generative explanations created by a Large Language Model. Our study surprisingly reveals that Templated explanations result in similar or higher perceived intelligence and trust while also being more understandable. Through our findings, we aim to provide considerations for effective robot explanation systems, ultimately enabling people to be able to understand and provide assistance to robots that have encountered unforeseen circumstances.

Templated vs. Generative: Explaining Robot Failures

The use of dialogue agents with a strategy that focuses beyond a static depth level and gradually deepens the conversation over time can nurture relationships with users. Thus, this study aimed to develop a dialogue agent capable of gradually deepening conversations over time and consequently examine its effectiveness. First, we constructed a topic depth estimation model capable of estimating the depth of any given topic. Subsequently, we developed a dialogue agent designed to gradually deepen conversation topics as interactions progress, tailored for operations on users' smartphones. To evaluate the effectiveness of the developed agent, we conducted a 10-day dialogue experiment with 26 participants. The results indicated that the proposed agent elicited deep self-disclosure from users without significantly increasing psychological reluctance compared to the control condition. Moreover, the participants perceived a deeper relationship with the agent and were less likely to consider it as a non-human entity. These findings suggest that the proposed dialogue agent was capable of effectively engaging in deeper conversations over time, thereby enhancing human-agent interactions.

Deepening Conversations Over Time: A Chatbot with a Topic Depth Estimation Model for Gradually Engaging in Deeper Chats

This work introduces a robotics platform which comprehensively integrates multi-step action planning, natural language understanding, and memory to interactively perform service tasks in accordance with variable needs and intentions of users. The proposed architecture is built around an AI agent, derived from GPT-4, which has been embedded in an embodied system. Our approach utilizes semantic matching, plan validation, and state messages to ground the agent in the physical world, enabling a seamless merger between communication and behavior. We demonstrated the advantages of this system with an HRI study comparing mobile robots with and without conversational AI capabilities in a free-form tour-guide scenario. The increased adaptability of the system is measured along five dimensions: flexible tasking planning, interactive exploration of information, emotional friendliness, personalization, and increased overall user satisfaction.

Dobby: A Conversational Service Robot Driven by GPT-4

For robots to effectively collaborate with human partners, they need to be able to understand what instructions and/or statements mean in the context of their environment. To ground objects in a dynamic world, robots need an understanding of how both spatial and temporal relationships evolve as a function of time in the environment. However, it is computationally intensive to classify, track, and predict the motion of all objects. Approaches based on Language-Guided Temporally Adaptive Perception (LGTAP) utilize information embedded in the instruction to selectively classify objects to construct minimal but sufficiently detailed models of the environment for symbol grounding. Such methods, however, fail when the instruction refers to the future state of the environment as it lacks any notion of whether to and for how long a future prediction is necessary to ground the instruction. This prompts a reformulation of LGTAP that can selectively utilize information from past observations to accurately predict the future state of objects. This paper describes a novel approach for LGTAP for instructions that may refer to the past, present, and/or future state of the environment by closing the loop around symbol grounding and adaptive perception. A detailed analysis of a grounding problem that refers to the future state of the environment, a corpus-based analysis of performance, and a physical demonstration of natural language understanding is presented along with a description of this novel architecture.

Grounding Language Instructions That Refer to the Past, Present, And/or Future State of Objects for Human-Robot Interaction

This paper proposes an energy consumption estimation in the trajectory planning task of an industrial robot (IR) arm manipulator. Human-robot collaborative scheduling has become integral to modern manufacturing. Effective scheduling of human-robot cooperation is crucial for enhancing production efficiency. However, energy-efficient trajectory planning in human-robot interaction remains an unexplored area. To address this, a 6-DoF (Degree of Freedom) robot arm, available in the laboratory, was utilized to assess the energy consumption of this specific robot type. However, his approach can be applied to any robot, whether it's a human-robot collaboration, a mobile robot, or any other articulated robot arm manipulator. For this goal the electrical and mechanical losses of a 6-DoF robot arm manipulator from the company Universal Robots is modeled and its power losses cost function (PLCF) is derived correspondingly to calculate the electromechanical energy losses. The objective of this paper is to estimate the energy losses in a trajectory planning task in cartesian space for a tool center point (TCP) of the robot, starting from an initial point and resting at a final point. This goal is achieved by examining and analyzing the kinematics and dynamics followed by measuring the velocity and acceleration related terms of each motor at each joint of the UR3e robot arm in a laboratory setup. The results estimate the changes of energy losses in the overall trajectory following of the robot manipulator and presents the results of the estimated and measured reduction terms.

Premium content

Downloads

Next from IEEE RO-MAN 2024

Imagining In-Distribution States: How Predictable Robot Behavior Can Enable User Control Over Learned Policies

Similar lecture

Visual Movement Prediction for Stable Grasp Point Prediction

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES