How Do You Measure a Dexterous Hand? Xynova's Simulation Comparison of 20 Models Sparks Discussion

In the Reddit community, a member of the Xynova team posted asking if anyone is attending this year's CoRL (Conference on Robot Learning) or Web Summit, and shared their latest exploration in the field of dexterous hands. The post not only raised an industry pain point—how to measure dexterity—but also published their internal simulation comparison results, sparking discussion about evaluation standards for embodied AI hardware.

Simulation Comparison: Prima 1 Stands Out in Translation and Reorientation Tests

The Xynova team recently ran a KaRMA simulation comparing 20 dexterous hand models. According to the post, their Prima 1 ranked second in the "translation" task and first in the "reorientation" task. Both tasks are key dimensions for measuring dexterous manipulation capabilities: translation involves stable movement of objects in space, while reorientation requires fingers to adjust object pose during grasping.

A detail worth noting: the team candidly admitted that "simulation is one thing, real-world manipulation is another." This clear-eyed awareness of simulation's limitations is particularly valuable in the embodied AI field. Simulation environments allow rapid iteration and standardized testing, but real-world contact dynamics, sensor noise, and object diversity often create enormous gaps. Therefore, simulation results are more for screening and guidance rather than final conclusions.

KaRMA simulation is a simulation benchmark platform for evaluating dexterous hand manipulation, with its name possibly derived from "Kinematic and Rigid-body Manipulation Assessment" or a similar acronym. It provides standardized task environments (such as translation and reorientation) and supports integration and comparison of multiple dexterous hand models. In simulation, physics engines (such as MuJoCo or PyBullet) simulate contact dynamics, but there exists a sim-to-real gap: idealized contact models, noise-free sensors, and simplified object geometry in simulation lead to deviations from real-world results. Therefore, benchmarks like KaRMA are typically used for algorithm development and preliminary screening rather than final performance determination. Xynova's use of KaRMA to compare 20 models reflects the trend of leveraging open-source simulation platforms for cross-model evaluation, but rankings should be interpreted with caution.

Beyond Degrees of Freedom: Dexterity Measurement Needs a New Paradigm

The post explicitly mentions that the team has been researching "how to measure dexterity beyond DOF counting." Degrees of freedom is a commonly used metric in traditional robotics to describe hand flexibility, but it cannot fully reflect actual manipulation capability. For example, a hand with high DOF may fail to perform fine manipulation due to control algorithm or mechanical design limitations, while a hand with lower DOF can accomplish complex tasks through clever kinematic design.

Xynova's KaRMA simulation comparison attempts to start from task performance (translation, reorientation) rather than just looking at hardware parameters. This echoes the call within the field for standardized benchmarks: we need metrics that are closer to actual tasks, such as grasp success rate, manipulation time, and disturbance rejection capability. The post author also asked the community: "What metrics do you think are most useful?" This is clearly an open-ended discussion aimed at gathering feedback to improve the evaluation system.

Degrees of Freedom (DOF) in robotics typically refers to the number of independent motion parameters of a rigid body or joint. For dexterous hands, DOF counting includes active DOF (directly controlled by actuators) and passive DOF (such as joints in underactuated mechanisms), but the latter are often difficult to control precisely. Traditionally, DOF has been viewed as a proxy for flexibility, but high DOF does not equal high dexterity—for example, the Shadow Hand has 24 DOF but extremely high control complexity, while some underactuated hands (such as the Robotiq Hand) have lower DOF but can achieve stable grasping through mechanical adaptation. Therefore, the field is gradually shifting toward task-based metrics such as grasp success rate, manipulation time, disturbance rejection capability, and translation/reorientation precision. These metrics are closer to practical applications, but standardization still faces challenges because task definitions and evaluation protocols have not yet been unified.

Conferences and Demos: In-Person Exchange at CoRL and Web Summit

Beyond technical discussion, the post also revealed the Xynova team's offline activity plans: they will showcase dexterous hand demos at CoRL in Austin and Web Summit in Lisbon. For robot learning researchers, CoRL is an important platform for sharing the latest algorithms and experiments; while Web Summit, as a major tech event, focuses more on industry and cross-disciplinary exchange. The team is willing to offer a small number of Web Summit tickets to creators interested in producing technical content, which may be a content collaboration strategy aimed at expanding their technical influence.

For practitioners focused on embodied AI, such in-person demos are a great opportunity to validate simulation results. After all, experiencing the feel of a dexterous hand firsthand is far more convincing than watching simulation videos.

CoRL (Conference on Robot Learning) is a top academic conference in the robot learning field, attracting researchers from around the world each year to share the latest results in reinforcement learning, imitation learning, embodied AI, and other directions. It focuses on the combination of algorithms and experiments and is an important showcase platform for dexterous hand research. Web Summit is one of the world's largest technology industry events, covering AI, hardware, startups, and more, with a focus on industry trends and cross-sector collaboration. Xynova's participation in both aims to balance academic recognition and industry exposure. For hardware-intensive technologies like dexterous hands, in-person demos are crucial because videos cannot convey key experiences such as force feedback and manipulation smoothness. Offering tickets to technical content creators is also a strategy to expand technical influence through KOLs.

Community Interaction and Industry Implications

This post was published on Reddit and is itself a typical case of community interaction. The Xynova team did not directly publish a product announcement but instead entered with questions and progress sharing, demonstrating technical strength while soliciting community opinions. This approach is more likely to gain recognition from the technical community.

From an industry perspective, dexterous hands are becoming a key hardware bottleneck for embodied AI. From Shadow Robot to Inspire, to startups like Xynova, everyone is exploring different technical routes. The lack of evaluation standards makes cross-model comparison difficult. Xynova's simulation comparison is a useful attempt, but as they said, the real world is the final exam. In the future, we may see more open-source benchmarks and standardized test sets emerge, driving dexterous hand technology toward maturity.

(This article is based on a public Reddit post and does not add unmentioned data.)

One-line Summary

Xynova shared dexterous hand simulation comparison results on Reddit, calling for dexterity metrics beyond degrees of freedom, and plans to demo at CoRL and Web Summit.

Paragraph Summary

The Xynova team posted on Reddit sharing their dexterous hand Prima 1's comparison results in KaRMA simulation: among 20 models, it ranked second in translation tasks and first in reorientation tasks. The team emphasized the gap between simulation and the real world and explored how to measure dexterity beyond DOF counting, shifting toward task-performance-based metrics. The post also revealed plans to demo at CoRL (Austin) and Web Summit (Lisbon), and to offer a small number of tickets to technical content creators. This community-question-style sharing reflects the urgent need for standardized evaluation benchmarks in the dexterous hand field, as well as startups' strategies to build technical influence through open-source simulation and in-person demos.