Common Failure Modes in Learned Locomotion Policies

What actually goes wrong when a reinforcement-learned walking controller meets the real world.

Learned locomotion policies can perform impressively in the exact conditions they were trained for, and then fail in ways that are hard to predict in advance once conditions shift even slightly outside that training distribution.

The usual patterns

Overfitting to simulation quirks (a policy that exploits an unrealistic simulator behavior rather than learning genuinely robust motion) is a common one, usually caught and mitigated with domain randomization. Distribution shift — encountering a real-world surface, obstacle, or disturbance meaningfully different from anything seen in training — is another, and it's the harder one to fully eliminate.

Why this makes evaluation tricky

A policy that looks robust across a wide, randomized simulation test suite can still fail in a specific real-world situation nobody thought to simulate, which is why real-world testing across a genuinely broad range of conditions remains an essential step that simulation alone can't fully replace.