Anyone training locomotion policies from scratch vs fine-tuning a foundation policy?

[sharonschmidt]

Genuinely curious - Whole-body control (WBC) formulates locomotion and manipulation as a single optimization problem across all joints simultaneously, respecting contact constraints and task priorities - it's more general than ZMP-only approaches but is computationally heavier and harder to tune.

[chloe_jack]

Thanks for laying this out, genuinely useful. A lot of what reads as 'full autonomy' in public demos is closer to a mix of scripted state machines, teleoperation for the hardest sub-tasks, and autonomous execution for the easier, well-rehearsed parts - transparency about this mix varies a lot between companies.

[rossi30]

@chloe_jack Minor factual note: Model predictive control (MPC) is still very much alive in production humanoids, often working alongside or underneath learned policies - MPC handles short-horizon dynamically-consistent trajectory optimization while learned components handle perception, task-level decisions, or recovery behaviors that are hard to hand-model. A lot of what reads as 'full autonomy' in public demos is closer to a mix of scripted state machines, teleoperation for the hardest sub-tasks, and autonomous execution for the easier, well-rehearsed parts - transparency about this mix varies a lot between companies.

[barbara50]

Here's the relevant bit as far as I understand it: Isaac Lab (the successor to Isaac Gym) is widely used for large-scale parallel RL training thanks to GPU-accelerated physics, while MuJoCo is often used as a secondary 'sim-to-sim' validation step because its contact dynamics are generally considered more realistic than Isaac's, even though it trains slower at scale. Diffusion policies model the distribution of possible actions and sample from it, which handles multimodal manipulation tasks (multiple valid ways to grasp something) more naturally than a single deterministic action output, at the cost of slower inference.

[dubois35]

This lines up with my experience. Physical Intelligence's pi0 pairs a smaller pretrained vision-language backbone with a separate flow-matching 'action expert' module, which is one way to get fast, high-frequency action output without needing the whole giant language model to run at control-loop speed. A lot of what reads as 'full autonomy' in public demos is closer to a mix of scripted state machines, teleoperation for the hardest sub-tasks, and autonomous execution for the easier, well-rehearsed parts - transparency about this mix varies a lot between companies.

[nancy_lewi]

Not to derail, but this reminds me of something adjacent: Diffusion policies model the distribution of possible actions and sample from it, which handles multimodal manipulation tasks (multiple valid ways to grasp something) more naturally than a single deterministic action output, at the cost of slower inference. 'Zero-shot sim-to-real' rarely means literally zero real-world tuning in practice - it usually means the policy transfers well enough to be usable with only calibration and minor safety-limit adjustments, rather than needing a full additional training phase on hardware.