Two-handed tool use - screwdrivers, scissors, etc - how far are we really?

[kwilliams]

Follow-up question though - Imitation learning from human demonstration video (without robot teleoperation data) is an appealing way to scale up training data cheaply, but it runs into the embodiment gap - human hand kinematics and force profiles don't map directly onto a robot hand's very different mechanism.

[byang]

@kwilliams This raises a question for me - Underactuated hands (fewer actuators than joints, using mechanical coupling to shape the grasp) are a reasonable engineering compromise for robust power grasps on a budget, but they generally can't do fine in-hand manipulation the way a fully actuated hand can. Object occlusion by the robot's own hand during the final approach to a grasp is a common and annoying perception problem - the closer the hand gets to a good grasp position, the more it blocks the camera's view of exactly what it's about to grab.

[matthew.yamamoto0]

Worth being a little skeptical of the marketing angle here. Underactuated hands (fewer actuators than joints, using mechanical coupling to shape the grasp) are a reasonable engineering compromise for robust power grasps on a budget, but they generally can't do fine in-hand manipulation the way a fully actuated hand can. Vision-based grasp confidence estimation (predicting success before attempting a grasp) and tactile-based confirmation (confirming after contact) are complementary rather than competing - vision helps you choose a grasp, tactile tells you if it actually worked.

[olga_lind]

Speaking from personal experience here, Imitation learning from human demonstration video (without robot teleoperation data) is an appealing way to scale up training data cheaply, but it runs into the embodiment gap - human hand kinematics and force profiles don't map directly onto a robot hand's very different mechanism. Underactuated hands (fewer actuators than joints, using mechanical coupling to shape the grasp) are a reasonable engineering compromise for robust power grasps on a budget, but they generally can't do fine in-hand manipulation the way a fully actuated hand can.

[nicole10]

@olga_lind From what I've seen: Underactuated hands (fewer actuators than joints, using mechanical coupling to shape the grasp) are a reasonable engineering compromise for robust power grasps on a budget, but they generally can't do fine in-hand manipulation the way a fully actuated hand can.

[karen.chen3]

One nitpick - Object occlusion by the robot's own hand during the final approach to a grasp is a common and annoying perception problem - the closer the hand gets to a good grasp position, the more it blocks the camera's view of exactly what it's about to grab. Tendon-driven fingers let you move the heavier actuators back into the palm or forearm, keeping the fingers themselves light and fast, but they introduce cable routing, tensioning, and long-term wear problems that direct-actuated fingers don't have.

[richard36]

@karen.chen3 I can speak to this a bit. Cable routing through a wrist joint with multiple degrees of freedom is a genuinely tricky mechanical design problem - tendons and wiring both need enough slack to avoid binding through the full range of motion without tangling or fraying over thousands of cycles.

[hill23]

Yeah, this tracks with what I've read as well. Bimanual manipulation - two arms coordinating on one task - is harder than it looks mostly because of the added degrees of freedom and the timing/force coordination required; a lot of 'two-handed' demos are actually closer to two independent single-hand tasks done in sequence.