RGB-D Cameras Explained

Color plus depth in a single, affordable sensor made a lot of modern robot perception possible.

An RGB-D camera captures a standard color (RGB) image alongside a per-pixel depth (D) map, usually produced by structured light, stereo, or time-of-flight, bundled into a single compact unit — commonly the sensor of choice for a robot's head or chest.

What it enables

Combining color and depth lets a robot both recognize what an object is (using the RGB image, often with a vision model) and know exactly where it is in 3D space (using the depth data) from one sensor, without needing to separately fuse two different sensing modalities by hand.

Limitations

Most affordable RGB-D cameras struggle outdoors in bright sunlight (particularly structured-light and some ToF designs, whose infrared projection gets washed out) and have a limited effective range, typically well under ten meters — fine for manipulation and near-field navigation, less so for long-range planning.