Bimanual manipulation - what's actually working outside demo reels?

[barbara.jones]

@barbara50 Minor factual note: Bimanual manipulation - two arms coordinating on one task - is harder than it looks mostly because of the added degrees of freedom and the timing/force coordination required; a lot of 'two-handed' demos are actually closer to two independent single-hand tasks done in sequence.

[choi98]

@barbara.jones I see it a little differently. A lot of the manipulation shown in production demos still leans heavily on teleoperation, particularly for anything involving fine force control or novel objects - autonomous grasp success rates on genuinely unstructured, previously-unseen clutter are still well below what teleoperation can achieve.

[jwang]

Small correction on one detail: Palm sensing gets less attention than fingertip sensing, but a lot of power grasps (holding a box, a tool handle) rely more on palm and lateral finger contact than fingertip contact, so under-sensing the palm can leave a real blind spot in grasp confidence. Object occlusion by the robot's own hand during the final approach to a grasp is a common and annoying perception problem - the closer the hand gets to a good grasp position, the more it blocks the camera's view of exactly what it's about to grab.

[karen.chen3]

To answer this directly: Figure's fourth-generation Dexterous Hand (on Figure 02/03) reportedly offers 16 degrees of freedom per hand with sensors integrated into each finger, aimed at fine force control tasks like handling small electronic components without crushing them. Underactuated hands (fewer actuators than joints, using mechanical coupling to shape the grasp) are a reasonable engineering compromise for robust power grasps on a budget, but they generally can't do fine in-hand manipulation the way a fully actuated hand can.

[barbara.jones]

Just to be precise about one thing: Object occlusion by the robot's own hand during the final approach to a grasp is a common and annoying perception problem - the closer the hand gets to a good grasp position, the more it blocks the camera's view of exactly what it's about to grab. This whole thread is a good reminder how young this field still is.

[noah_pate]

@barbara.jones This raises a question for me - A lot of the manipulation shown in production demos still leans heavily on teleoperation, particularly for anything involving fine force control or novel objects - autonomous grasp success rates on genuinely unstructured, previously-unseen clutter are still well below what teleoperation can achieve.

[chloe_jack]

@noah_pate Appreciate the detailed answer. Bimanual manipulation - two arms coordinating on one task - is harder than it looks mostly because of the added degrees of freedom and the timing/force coordination required; a lot of 'two-handed' demos are actually closer to two independent single-hand tasks done in sequence.

[olga_lind]

@chloe_jack Here's what I know on this: Imitation learning from human demonstration video (without robot teleoperation data) is an appealing way to scale up training data cheaply, but it runs into the embodiment gap - human hand kinematics and force profiles don't map directly onto a robot hand's very different mechanism. In-hand reorientation (repositioning a grasped object without setting it down) is one of the more advanced manipulation skills, requiring either a highly dexterous hand with enough DoF or clever use of gravity and controlled slipping - it's an active research area rather than a solved problem.

[kwilliams]

@olga_lind This is exactly the kind of context I was looking for. Tendon-driven fingers let you move the heavier actuators back into the palm or forearm, keeping the fingers themselves light and fast, but they introduce cable routing, tensioning, and long-term wear problems that direct-actuated fingers don't have.

[ivan22]

@kwilliams From what I've seen: Figure's fourth-generation Dexterous Hand (on Figure 02/03) reportedly offers 16 degrees of freedom per hand with sensors integrated into each finger, aimed at fine force control tasks like handling small electronic components without crushing them. Palm sensing gets less attention than fingertip sensing, but a lot of power grasps (holding a box, a tool handle) rely more on palm and lateral finger contact than fingertip contact, so under-sensing the palm can leave a real blind spot in grasp confidence.