Anyone using imitation learning purely from human video for grasp policies?

[emilyperez]

Worth being a little skeptical of the marketing angle here. Wet, oily, or otherwise low-friction objects are still a genuine edge case for most current hands, since tactile sensing and grasp-force controllers are typically tuned and validated on dry, higher-friction test objects.

[cynthia.muller]

@emilyperez I can speak to this a bit. Grasp planning for deformable or non-rigid objects (bags, cables, cloth) remains one of the genuinely unsolved problems in manipulation - rigid-body grasp models simply don't capture how the object will behave once contact starts.

[jwang]

@cynthia.muller One nitpick - Cable routing through a wrist joint with multiple degrees of freedom is a genuinely tricky mechanical design problem - tendons and wiring both need enough slack to avoid binding through the full range of motion without tangling or fraying over thousands of cycles.

[johnrossi]

Speaking from personal experience here, Compliant wrists that absorb impact during a bad approach or misjudged contact reduce mechanical stress on the whole arm, which matters a lot for long-term reliability even though it's a less visible feature than the hand itself. A lot of the manipulation shown in production demos still leans heavily on teleoperation, particularly for anything involving fine force control or novel objects - autonomous grasp success rates on genuinely unstructured, previously-unseen clutter are still well below what teleoperation can achieve.

[young58]

Related question - Imitation learning from human demonstration video (without robot teleoperation data) is an appealing way to scale up training data cheaply, but it runs into the embodiment gap - human hand kinematics and force profiles don't map directly onto a robot hand's very different mechanism. Compliant wrists that absorb impact during a bad approach or misjudged contact reduce mechanical stress on the whole arm, which matters a lot for long-term reliability even though it's a less visible feature than the hand itself. Makes me wonder how this looks in another five years.

[thomasmitchell]

@young58 Related question - A lot of the manipulation shown in production demos still leans heavily on teleoperation, particularly for anything involving fine force control or novel objects - autonomous grasp success rates on genuinely unstructured, previously-unseen clutter are still well below what teleoperation can achieve.

[arjunsanchez]

@thomasmitchell Just to be precise about one thing: Grasp planning for deformable or non-rigid objects (bags, cables, cloth) remains one of the genuinely unsolved problems in manipulation - rigid-body grasp models simply don't capture how the object will behave once contact starts.