Key Takeaways
- • A team led by Ning Ding of Tsinghua University, with researchers from Beijing University of Posts and Telecommunications, Peking University and Nature Will, released HandsOn, a composable simulation platform and benchmark for cross-embodiment dexterous manipulation.
- • HandsOn includes 66 human-validated tasks split into Core-50 and Transfer-16, supported by 18,000 simulation teleoperation demonstrations, 5 robotic arm types, 6 dexterous hand types and about 200 assets.
- • The best trained multi-task policy reached a 29.1% success rate on Core-50 tasks with training demonstrations, while the average success rate of eight trained multi-task policies on Transfer-16 was below 1%.
- • GPT-6, without fine-tuning on benchmark data, achieved 5.2% on Core-50 and 1.9% on Transfer-16 across 660 tests using the move_eef interface.
- • Robots often complete the first half of action sequences but fail at critical moments, and the benchmark's phased diagnosis is designed to show where failures occur.
Summary
The Tsinghua-led team has introduced HandsOn, a benchmark intended to fill a gap in evaluation standards for dexterous robotic hands, according to the 36Kr report. The authors argue that most existing manipulation benchmarks were designed for grippers, while dexterous hands feature many active joints, contact-heavy tasks, bimanual collaboration and long action sequences that make success and failure difficult to judge. The work was produced with researchers from Beijing University of Posts and Telecommunications, Peking University and Nature Will. The code, paper and datasets have been publicly released.
HandsOn comprises 66 human-validated tasks and 18,000 carefully selected simulation teleoperation demonstrations, with evaluation focused on tool use, fine contact, bimanual collaboration and task transfer. The platform supports five robotic arm types, six dexterous hand types and about 200 assets spanning rigid, articulated and deformable objects, which can be combined in randomized scenarios. The tasks are split into Core-50, which provides demonstrations for policy learning and is judged by separate execution tests, and Transfer-16, which withholds target-task demonstrations to test generalization.
Results reported by the team are modest. The researchers evaluated diffusion policies, vision-language-action models and world-action models. The best trained multi-task policy achieved a 29.1% success rate on Core-50, while the average success rate of all eight trained multi-task policies on Transfer-16 was below 1%. GPT-6, used without fine-tuning and controlling palm and fingers through the move_eef interface, reached 5.2% on Core-50 and 1.9% on Transfer-16 over 660 tests. The team found that robots frequently complete the early portion of action sequences but err at critical moments involving grasping, contact, bimanual coordination or maintaining target state.
To make evaluation reliable, each task has a task card defining goals, initial and final states, permitted and prohibited actions, and completion criteria based on object pose, joint state or images. Manual verification and an independent review process were used to confirm that success labels match physical outcomes. Multi-step tasks record phased progress, allowing researchers to distinguish a complete failure from partial completion followed by a failure in later steps.
HandsOn comprises 66 human-validated tasks and 18,000 carefully selected simulation teleoperation demonstrations, with evaluation focused on tool use, fine contact, bimanual collaboration and task transfer. The platform supports five robotic arm types, six dexterous hand types and about 200 assets spanning rigid, articulated and deformable objects, which can be combined in randomized scenarios. The tasks are split into Core-50, which provides demonstrations for policy learning and is judged by separate execution tests, and Transfer-16, which withholds target-task demonstrations to test generalization.
Results reported by the team are modest. The researchers evaluated diffusion policies, vision-language-action models and world-action models. The best trained multi-task policy achieved a 29.1% success rate on Core-50, while the average success rate of all eight trained multi-task policies on Transfer-16 was below 1%. GPT-6, used without fine-tuning and controlling palm and fingers through the move_eef interface, reached 5.2% on Core-50 and 1.9% on Transfer-16 over 660 tests. The team found that robots frequently complete the early portion of action sequences but err at critical moments involving grasping, contact, bimanual coordination or maintaining target state.
To make evaluation reliable, each task has a task card defining goals, initial and final states, permitted and prohibited actions, and completion criteria based on object pose, joint state or images. Manual verification and an independent review process were used to confirm that success labels match physical outcomes. Multi-step tasks record phased progress, allowing researchers to distinguish a complete failure from partial completion followed by a failure in later steps.
Go deeper than the headline
Research every firm in this story, track the ones that matter and get the daily briefing — free for 7 days.