Would you like to see your presentation here, made available to a global audience of researchers?
Add your own presentation or have us affordably record your next conference.
Learning to manipulate diverse objects with multi-finger dexterous hands remains a significant challenge in robotics. Human-Object Interaction datasets constitute a rich repository of knowledge about task information and embodied interactions. Instead of solely imitating the human demonstrations, we consider the hand-object interaction process as a whole by predicting the hand-object future states holistically. The predicted object future states can not only facilitate the reinforcement learning by alleviating the heavy reliance on task-specific reward design, but also enable our pipeline to be more general to various task settings. We conduct extensive robot experiments across 3 challenging tasks with novel objects. Results demonstrate that our methods outperform existing SOTA methods in all 3 tasks with higher success rates and better adaptive ability to novel object configurations. We also validate the cross-embodiment compatibility of our methods on different robots to prove the learned priors' universal utility.
