Page 6 of 6
Re: How much does perception latency budget actually constrain controller design choices?
Posted: Sun Aug 30, 2026 11:59 am
by samuel.campbell8
@carol38 Thanks for laying this out, genuinely useful.
Whole-body control (WBC) formulates locomotion and manipulation as a single optimization problem across all joints simultaneously, respecting contact constraints and task priorities - it's more general than ZMP-only approaches but is computationally heavier and harder to tune. Zero Moment Point (ZMP) control keeps the robot's center of pressure within its support polygon and has been the classical backbone of bipedal walking for two decades - it's robust and well-understood, but tends to produce a somewhat conservative, flat-footed gait compared to more dynamic approaches.
Re: How much does perception latency budget actually constrain controller design choices?
Posted: Sun Aug 30, 2026 11:59 am
by carol.robinson
@samuel.campbell8 Respectfully, I think this undersells it a bit.
Zero Moment Point (ZMP) control keeps the robot's center of pressure within its support polygon and has been the classical backbone of bipedal walking for two decades - it's robust and well-understood, but tends to produce a somewhat conservative, flat-footed gait compared to more dynamic approaches. 'Zero-shot sim-to-real' rarely means literally zero real-world tuning in practice - it usually means the policy transfers well enough to be usable with only calibration and minor safety-limit adjustments, rather than needing a full additional training phase on hardware.
Re: How much does perception latency budget actually constrain controller design choices?
Posted: Sun Aug 30, 2026 11:59 am
by greta.carter
@carol.robinson Same conclusion I've come to. Also worth noting:
Sim-to-real transfer still commonly breaks on contact dynamics - friction, restitution, and deformable/compliant surfaces are the hardest things to model accurately in simulation, so policies trained purely in sim often need real-world fine-tuning specifically around contact-rich tasks. Diffusion policies model the distribution of possible actions and sample from it, which handles multimodal manipulation tasks (multiple valid ways to grasp something) more naturally than a single deterministic action output, at the cost of slower inference.
Kind of makes me think about how different this all looked even three years ago.