Curriculum learning for locomotion - worth the setup complexity?

[diego.moore6]

This came up in a Discord I'm in and I wanted a more permanent place to discuss it. Zero Moment Point (ZMP) control keeps the robot's center of pressure within its support polygon and has been the classical backbone of bipedal walking for two decades - it's robust and well-understood, but tends to produce a somewhat conservative, flat-footed gait compared to more dynamic approaches. Sim-to-real transfer still commonly breaks on contact dynamics - friction, restitution, and deformable/compliant surfaces are the hardest things to model accurately in simulation, so policies trained purely in sim often need real-world fine-tuning specifically around contact-rich tasks. Vision-Language-Action (VLA) models like RT-2, OpenVLA, and Physical Intelligence's pi0 unify a vision-language backbone with an action-output head, letting a robot map a camera image and a text instruction directly to motor commands instead of hand-coding separate perception and planning stages. Interested in both agreement and pushback here.

[scott21]

@diego.moore6 This matches what I've seen too. Physical Intelligence's pi0 pairs a smaller pretrained vision-language backbone with a separate flow-matching 'action expert' module, which is one way to get fast, high-frequency action output without needing the whole giant language model to run at control-loop speed. Balance-recovery controllers are usually evaluated with push-recovery tests (a known, repeatable lateral push) in demos, but real-world robustness also depends on recovering from unstructured events like uneven flooring, unexpected contact, or a dropped payload shifting the center of mass mid-stride - which is a much harder, less demo-friendly test. Kind of makes me think about how different this all looked even three years ago.

[hill23]

Pretty much this. One thing to add: Diffusion policies model the distribution of possible actions and sample from it, which handles multimodal manipulation tasks (multiple valid ways to grasp something) more naturally than a single deterministic action output, at the cost of slower inference. Kind of makes me think about how different this all looked even three years ago.

[kim37]

@hill23 Not sure I fully agree here. Cross-embodiment training (training one policy across data from multiple different robot bodies) has shown some real transfer benefits for high-level behaviors, but low-level control (exact joint torques, timing) still tends to need embodiment-specific fine-tuning. OpenVLA is a notable open-source VLA model - roughly 7 billion parameters, trained on hundreds of thousands of real-world robot demonstrations - and has been shown to outperform much larger closed models on some manipulation benchmarks, which says a lot about how much of VLA performance comes from data curation rather than raw scale.

[jonathan.rao1]

This is a great summary, thanks. A lot of what reads as 'full autonomy' in public demos is closer to a mix of scripted state machines, teleoperation for the hardest sub-tasks, and autonomous execution for the easier, well-rehearsed parts - transparency about this mix varies a lot between companies.

[emilyperez]

@jonathan.rao1 Short answer: Domain randomization - varying friction, mass, sensor noise, and even visual textures during training - is one of the more reliable tricks for improving sim-to-real transfer, but overdoing it can make training slower to converge and produce overly conservative policies.

[jessica_faro]

@emilyperez Worth being a little skeptical of the marketing angle here. Model predictive control (MPC) is still very much alive in production humanoids, often working alongside or underneath learned policies - MPC handles short-horizon dynamically-consistent trajectory optimization while learned components handle perception, task-level decisions, or recovery behaviors that are hard to hand-model.

[nicole57]

@jessica_faro Speaking from personal experience here, ROS2 remains common in research and early-stage products for its tooling and ecosystem, but a number of production humanoid companies run custom, more tightly-optimized middleware for their real-time control loops, using ROS2-like tooling mainly for development, visualization, and non-real-time subsystems.

[rossi30]

@nicole57 Counterpoint: Whole-body control (WBC) formulates locomotion and manipulation as a single optimization problem across all joints simultaneously, respecting contact constraints and task priorities - it's more general than ZMP-only approaches but is computationally heavier and harder to tune.

[carol.robinson]

Related question - 'Zero-shot sim-to-real' rarely means literally zero real-world tuning in practice - it usually means the policy transfers well enough to be usable with only calibration and minor safety-limit adjustments, rather than needing a full additional training phase on hardware. Isaac Lab (the successor to Isaac Gym) is widely used for large-scale parallel RL training thanks to GPU-accelerated physics, while MuJoCo is often used as a secondary 'sim-to-sim' validation step because its contact dynamics are generally considered more realistic than Isaac's, even though it trains slower at scale.