What's the real difference between a 'policy' and a classical controller at this point?

[jhansen]

Been meaning to post this for a while. Sim-to-real transfer still commonly breaks on contact dynamics - friction, restitution, and deformable/compliant surfaces are the hardest things to model accurately in simulation, so policies trained purely in sim often need real-world fine-tuning specifically around contact-rich tasks. Domain randomization - varying friction, mass, sensor noise, and even visual textures during training - is one of the more reliable tricks for improving sim-to-real transfer, but overdoing it can make training slower to converge and produce overly conservative policies. What's everyone else's take?

[ramirez77]

New to this, so forgive me if this is obvious - A lot of what reads as 'full autonomy' in public demos is closer to a mix of scripted state machines, teleoperation for the hardest sub-tasks, and autonomous execution for the easier, well-rehearsed parts - transparency about this mix varies a lot between companies. Cross-embodiment training (training one policy across data from multiple different robot bodies) has shown some real transfer benefits for high-level behaviors, but low-level control (exact joint torques, timing) still tends to need embodiment-specific fine-tuning. Makes me wonder how this looks in another five years.

[carlossanchez]

@ramirez77 I'd frame this differently. ROS2 remains common in research and early-stage products for its tooling and ecosystem, but a number of production humanoid companies run custom, more tightly-optimized middleware for their real-time control loops, using ROS2-like tooling mainly for development, visualization, and non-real-time subsystems.

[freya.smith]

Side note that might be relevant: 'Zero-shot sim-to-real' rarely means literally zero real-world tuning in practice - it usually means the policy transfers well enough to be usable with only calibration and minor safety-limit adjustments, rather than needing a full additional training phase on hardware.

[cynthia.muller]

Genuinely curious - Zero Moment Point (ZMP) control keeps the robot's center of pressure within its support polygon and has been the classical backbone of bipedal walking for two decades - it's robust and well-understood, but tends to produce a somewhat conservative, flat-footed gait compared to more dynamic approaches.

[nicole57]

Minor factual note: Whole-body control (WBC) formulates locomotion and manipulation as a single optimization problem across all joints simultaneously, respecting contact constraints and task priorities - it's more general than ZMP-only approaches but is computationally heavier and harder to tune.

[rossi30]

Small correction on one detail: Diffusion policies model the distribution of possible actions and sample from it, which handles multimodal manipulation tasks (multiple valid ways to grasp something) more naturally than a single deterministic action output, at the cost of slower inference. Model predictive control (MPC) is still very much alive in production humanoids, often working alongside or underneath learned policies - MPC handles short-horizon dynamically-consistent trajectory optimization while learned components handle perception, task-level decisions, or recovery behaviors that are hard to hand-model.

[green28]

Can I ask a dumb follow-up - Balance-recovery controllers are usually evaluated with push-recovery tests (a known, repeatable lateral push) in demos, but real-world robustness also depends on recovering from unstructured events like uneven flooring, unexpected contact, or a dropped payload shifting the center of mass mid-stride - which is a much harder, less demo-friendly test.

[arjunsanchez]

Slight correction, though the overall point stands: OpenVLA is a notable open-source VLA model - roughly 7 billion parameters, trained on hundreds of thousands of real-world robot demonstrations - and has been shown to outperform much larger closed models on some manipulation benchmarks, which says a lot about how much of VLA performance comes from data curation rather than raw scale.

[dchen]

I dealt with almost this exact situation. Isaac Lab (the successor to Isaac Gym) is widely used for large-scale parallel RL training thanks to GPU-accelerated physics, while MuJoCo is often used as a secondary 'sim-to-sim' validation step because its contact dynamics are generally considered more realistic than Isaac's, even though it trains slower at scale. Vision-Language-Action (VLA) models like RT-2, OpenVLA, and Physical Intelligence's pi0 unify a vision-language backbone with an action-output head, letting a robot map a camera image and a text instruction directly to motor commands instead of hand-coding separate perception and planning stages.