Anyone using diffusion policies for manipulation instead of VLA tokens?

[gary.tanaka2]

Thanks for laying this out, genuinely useful. Diffusion policies model the distribution of possible actions and sample from it, which handles multimodal manipulation tasks (multiple valid ways to grasp something) more naturally than a single deterministic action output, at the cost of slower inference. Physical Intelligence's pi0 pairs a smaller pretrained vision-language backbone with a separate flow-matching 'action expert' module, which is one way to get fast, high-frequency action output without needing the whole giant language model to run at control-loop speed.

[samuel.campbell8]

@gary.tanaka2 +1 to this. Worth adding: Sim-to-real transfer still commonly breaks on contact dynamics - friction, restitution, and deformable/compliant surfaces are the hardest things to model accurately in simulation, so policies trained purely in sim often need real-world fine-tuning specifically around contact-rich tasks.

[ethan17]

@samuel.campbell8 From hands-on experience, Cross-embodiment training (training one policy across data from multiple different robot bodies) has shown some real transfer benefits for high-level behaviors, but low-level control (exact joint torques, timing) still tends to need embodiment-specific fine-tuning. Diffusion policies model the distribution of possible actions and sample from it, which handles multimodal manipulation tasks (multiple valid ways to grasp something) more naturally than a single deterministic action output, at the cost of slower inference.