DexJoCo: A Benchmark and Toolkit for Task-Oriented Dexterous Manipulation on MuJoCo
DexJoCo:用于MuJoCo上的任务导向灵巧操作的基准和工具包
Hanwen Wang, Weizhi Zhao, Xiangyu Wang, Siyuan Huang, He Lin, Boyuan Zheng, Rongtao Xu, Gang Wang, Yao Mu, He Wang, Lue Fan, Hongsheng Li, Zhaoxiang Zhang, Tieniu Tan
AI总结 本文提出DexJoCo基准和工具包,包含11个功能任务评估灵巧手的工具使用、双臂协调、长周期执行和推理能力,通过低成本数据收集系统和领域随机化评估鲁棒性,揭示当前策略的局限性。
详情
- Comments
- 8 pages, 6 figures, project page is available at: https://dexjoco.github.io
实现人类水平的操作需要能够进行复杂物体交互的灵巧机器人手。进一步发展此类能力需要标准化的基准以进行系统评估。然而,现有的灵巧基准缺乏反映灵巧手相对于平行夹具独特操作能力的任务以及全面的评估流程。本文提出了DexJoCo,一个用于任务导向灵巧操作的基准和工具包,包含11个功能基础任务,评估工具使用、双臂协调、长周期执行和推理。我们开发了一个低成本的数据收集系统,并在这些任务中收集了1100多条轨迹,支持领域随机化以评估鲁棒性。我们在此基础上对现代模型进行基准测试,包括视觉和动态随机化、多任务训练和动作头适应。通过广泛的实证分析,我们识别出当前策略在灵巧操作中的几个重要见解和共同限制,突显了未来灵巧手机器人学习中的关键挑战。项目页面可访问:https://dexjoco.github.io
Achieving human-level manipulation requires dexterous robotic hands capable of complex object interactions. Advancing such capabilities further demands standardized benchmarks for systematic evaluation. However, existing dexterous benchmarks lack tasks that reflect the unique manipulation capabilities of dexterous hands over parallel grippers, as well as comprehensive evaluation pipelines. In this paper, we present DexJoCo, a benchmark and toolkit for task-oriented dexterous manipulation, comprising 11 functionally grounded tasks that evaluate tool-use, bimanual coordination, long-horizon execution, and reasoning. We develop a low-cost data collection system and collect 1.1K trajectories across these tasks, with support for domain randomization to assess robustness. We benchmark modern models under diverse settings, including visual and dynamics randomization, multi-task training, and action-head adaptation. Through extensive empirical analysis, we identify several important insights and common limitations of current policies in dexterous manipulation, highlighting key challenges for future research in dexterous hand robot learning. Project page available at: https://dexjoco.github.io