What's your favorite open-source locomotion RL framework right now?

[mohammed64]

This came up in a Discord I'm in and I wanted a more permanent place to discuss it. Sim-to-real transfer still commonly breaks on contact dynamics - friction, restitution, and deformable/compliant surfaces are the hardest things to model accurately in simulation, so policies trained purely in sim often need real-world fine-tuning specifically around contact-rich tasks. OpenVLA is a notable open-source VLA model - roughly 7 billion parameters, trained on hundreds of thousands of real-world robot demonstrations - and has been shown to outperform much larger closed models on some manipulation benchmarks, which says a lot about how much of VLA performance comes from data curation rather than raw scale. Interested to see if this matches what others are seeing.

[emma_whit]

@mohammed64 I dealt with almost this exact situation. 'Zero-shot sim-to-real' rarely means literally zero real-world tuning in practice - it usually means the policy transfers well enough to be usable with only calibration and minor safety-limit adjustments, rather than needing a full additional training phase on hardware.

[deborahperez]

I'd push back on this a bit. Balance-recovery controllers are usually evaluated with push-recovery tests (a known, repeatable lateral push) in demos, but real-world robustness also depends on recovering from unstructured events like uneven flooring, unexpected contact, or a dropped payload shifting the center of mass mid-stride - which is a much harder, less demo-friendly test.

[jhansen]

Sorry if this is a basic question, but Domain randomization - varying friction, mass, sensor noise, and even visual textures during training - is one of the more reliable tricks for improving sim-to-real transfer, but overdoing it can make training slower to converge and produce overly conservative policies.

[ashley_flor]

@jhansen Follow-up question though - Vision-Language-Action (VLA) models like RT-2, OpenVLA, and Physical Intelligence's pi0 unify a vision-language backbone with an action-output head, letting a robot map a camera image and a text instruction directly to motor commands instead of hand-coding separate perception and planning stages.

[nancy_lewi]

I'd push back on this a bit. Model predictive control (MPC) is still very much alive in production humanoids, often working alongside or underneath learned policies - MPC handles short-horizon dynamically-consistent trajectory optimization while learned components handle perception, task-level decisions, or recovery behaviors that are hard to hand-model. Zero Moment Point (ZMP) control keeps the robot's center of pressure within its support polygon and has been the classical backbone of bipedal walking for two decades - it's robust and well-understood, but tends to produce a somewhat conservative, flat-footed gait compared to more dynamic approaches.

[lperez]

Tangent, but worth mentioning: Physical Intelligence's pi0 pairs a smaller pretrained vision-language backbone with a separate flow-matching 'action expert' module, which is one way to get fast, high-frequency action output without needing the whole giant language model to run at control-loop speed. Isaac Lab (the successor to Isaac Gym) is widely used for large-scale parallel RL training thanks to GPU-accelerated physics, while MuJoCo is often used as a secondary 'sim-to-sim' validation step because its contact dynamics are generally considered more realistic than Isaac's, even though it trains slower at scale.

[arjunsanchez]

Appreciate the detailed answer. Diffusion policies model the distribution of possible actions and sample from it, which handles multimodal manipulation tasks (multiple valid ways to grasp something) more naturally than a single deterministic action output, at the cost of slower inference. Cross-embodiment training (training one policy across data from multiple different robot bodies) has shown some real transfer benefits for high-level behaviors, but low-level control (exact joint torques, timing) still tends to need embodiment-specific fine-tuning.

[williams84]

@arjunsanchez I don't think that's quite right, for what it's worth. ROS2 remains common in research and early-stage products for its tooling and ecosystem, but a number of production humanoid companies run custom, more tightly-optimized middleware for their real-time control loops, using ROS2-like tooling mainly for development, visualization, and non-real-time subsystems.

[rivera14]

To answer this directly: Whole-body control (WBC) formulates locomotion and manipulation as a single optimization problem across all joints simultaneously, respecting contact constraints and task priorities - it's more general than ZMP-only approaches but is computationally heavier and harder to tune. A lot of what reads as 'full autonomy' in public demos is closer to a mix of scripted state machines, teleoperation for the hardest sub-tasks, and autonomous execution for the easier, well-rehearsed parts - transparency about this mix varies a lot between companies. This whole thread is a good reminder how young this field still is.