What's the current bottleneck in scaling manipulation training data collection?

[ronald.clark]

@ethan.lewis5 Same conclusion I've come to. Also worth noting: Imitation learning from human demonstration video (without robot teleoperation data) is an appealing way to scale up training data cheaply, but it runs into the embodiment gap - human hand kinematics and force profiles don't map directly onto a robot hand's very different mechanism.

[yuki71]

@ronald.clark Yeah, this tracks with what I've read as well. A lot of the manipulation shown in production demos still leans heavily on teleoperation, particularly for anything involving fine force control or novel objects - autonomous grasp success rates on genuinely unstructured, previously-unseen clutter are still well below what teleoperation can achieve.

[deborahperez]

@yuki71 I can speak to this a bit. Compliant wrists that absorb impact during a bad approach or misjudged contact reduce mechanical stress on the whole arm, which matters a lot for long-term reliability even though it's a less visible feature than the hand itself. Payload-to-hand-weight ratio varies a lot across current dexterous hands, and it's a meaningful tradeoff - more DoF and finer sensing generally means more actuators and mass in the hand itself, which eats into the arm's usable payload budget.

[chloe_jack]

Short answer: Underactuated hands (fewer actuators than joints, using mechanical coupling to shape the grasp) are a reasonable engineering compromise for robust power grasps on a budget, but they generally can't do fine in-hand manipulation the way a fully actuated hand can. Reminds me a bit of the early drone hobbyist scene, honestly.

[matthew.yamamoto0]

@chloe_jack Appreciate the detailed answer. Underactuated hands (fewer actuators than joints, using mechanical coupling to shape the grasp) are a reasonable engineering compromise for robust power grasps on a budget, but they generally can't do fine in-hand manipulation the way a fully actuated hand can.

[greta78]

@matthew.yamamoto0 Here's the relevant bit as far as I understand it: Grasp planning for deformable or non-rigid objects (bags, cables, cloth) remains one of the genuinely unsolved problems in manipulation - rigid-body grasp models simply don't capture how the object will behave once contact starts. In-hand reorientation (repositioning a grasped object without setting it down) is one of the more advanced manipulation skills, requiring either a highly dexterous hand with enough DoF or clever use of gravity and controlled slipping - it's an active research area rather than a solved problem. Kind of makes me think about how different this all looked even three years ago.

[nschmidt]

I'd frame this differently. Wet, oily, or otherwise low-friction objects are still a genuine edge case for most current hands, since tactile sensing and grasp-force controllers are typically tuned and validated on dry, higher-friction test objects.

[wei_ross]

Worth being a little skeptical of the marketing angle here. Payload-to-hand-weight ratio varies a lot across current dexterous hands, and it's a meaningful tradeoff - more DoF and finer sensing generally means more actuators and mass in the hand itself, which eats into the arm's usable payload budget. Kind of makes me think about how different this all looked even three years ago.

[ethan17]

@wei_ross Worth being a little skeptical of the marketing angle here. Object occlusion by the robot's own hand during the final approach to a grasp is a common and annoying perception problem - the closer the hand gets to a good grasp position, the more it blocks the camera's view of exactly what it's about to grab.

[betty.king]

Slight correction, though the overall point stands: Wet, oily, or otherwise low-friction objects are still a genuine edge case for most current hands, since tactile sensing and grasp-force controllers are typically tuned and validated on dry, higher-friction test objects.