Behavior Cloning vs. Reinforcement Learning for Manipulation

Two very different ways to teach a robot arm a task, with different data requirements and failure modes.

Behavior cloning trains a policy to directly imitate recorded human demonstrations — a person teleoperates the robot through a task repeatedly, and the model learns to reproduce those same motions from the same sensory inputs. Reinforcement learning instead has the robot try actions itself and learn from a reward signal, without needing human demonstrations at all (though it can use them as a head start).

Why behavior cloning is popular for manipulation

Manipulation tasks are often hard to specify a clean reward function for (how do you numerically reward 'fold the shirt neatly'?), but easy for a human to demonstrate directly. Collecting demonstration data via teleoperation has become a major focus across the industry specifically because of this.

The tradeoffs

Behavior cloning policies tend to struggle outside situations closely resembling their training demonstrations, since they've only ever seen 'correct' examples and have no built-in mechanism for recovering from a mistake they haven't seen before. RL-based policies can in principle learn recovery behavior naturally, but usually need far more training experience and a well-designed reward function to get there.