Domain Randomization: Why Simulators Deliberately Get Things "Wrong"

A counterintuitive trick: training on a wide range of inaccurate simulations to handle the one real, accurate world.

Domain randomization varies physical parameters in simulation during training — friction coefficients, mass distribution, motor strength, sensor noise — across a wide range rather than trying to match reality exactly.

Why deliberately being wrong helps

No simulator matches real-world physics perfectly, so a policy trained on one single, fixed (and inevitably slightly wrong) simulated model tends to overfit to that model's specific quirks. Training across a wide randomized range instead forces the policy to find behavior that's robust to variation — and the real world just becomes one more point within that range it's already prepared for.

The tradeoff

Too much randomization can make the learning problem needlessly harder and slower to converge, since the policy has to handle an enormous range of scenarios, many far outside what it will actually encounter. Choosing sensible randomization ranges — wide enough for robustness, narrow enough to stay learnable — is itself a design decision that takes experience to get right.