What's a good beginner sensor fusion project before tackling a full robot?

[carol.robinson]

@barbara50 Slightly off-topic, but related: Vibration is one of the most underrated sources of noisy IMU and tactile readings - mounting matters as much as sensor quality, and a poorly isolated mount can add more noise than the sensor's own datasheet specs would suggest. Kind of makes me think about how different this all looked even three years ago.

[williams84]

Short answer: A minimum viable sensing suite for safe bipedal walking generally includes joint encoders, an IMU for orientation/angular velocity, and either force/torque sensing or accurate current-based torque estimation at the ankles - everything else (vision, tactile, LiDAR) adds capability rather than being strictly required just to stay upright. SLAM in a working warehouse is harder than in a controlled lab mainly because the map keeps changing - pallets move, people walk through, lighting shifts near dock doors - so a lot of production systems lean on semi-static maps refreshed periodically rather than pure continuous SLAM.

[kwilliams]

@williams84 I dealt with almost this exact situation. Tactile skin arrays have improved a lot, but 'good enough to matter' really depends on the task - coarse contact detection across a large area is fairly mature, while fine, high-resolution force distribution sensing (like a human fingertip) is still the harder problem. Latency between a perceived event (like a slip) and a corrective control response matters enormously for balance - even 50-100ms of extra perception latency can be the difference between a smooth recovery and a fall, which is part of why a lot of balance-critical sensing is proprioceptive rather than vision-based.

[williams84]

@kwilliams Side note that might be relevant: Sensor fusion mostly earns its keep by covering for each individual sensor's weaknesses - vision struggles with occlusion and lighting, IMUs drift, force/torque sensors are noisy at low loads - fusing them gives a more robust estimate than any one source alone, independent of raw compute. A minimum viable sensing suite for safe bipedal walking generally includes joint encoders, an IMU for orientation/angular velocity, and either force/torque sensing or accurate current-based torque estimation at the ankles - everything else (vision, tactile, LiDAR) adds capability rather than being strictly required just to stay upright.

[mohammed64]

@williams84 From hands-on experience, Depth sensing range and reliability both degrade outdoors in direct sunlight for most structured-light and active stereo cameras, since the ambient IR washes out the projected pattern - it's a real limitation for humanoids intended for anything beyond indoor, controlled environments. IMU drift over time (bias instability) is usually the real culprit behind slowly diverging state estimates, not noise - it's typically handled with sensor fusion against other references (visual odometry, joint kinematics) rather than trying to eliminate drift at the source.

[lperez]

@mohammed64 This lines up with my experience. Unitree's Dex3-1 dexterous hand packs around 33 pressure/tactile sensors per hand across the fingers and palm, capable of sensing pressure roughly in the 10g-2500g range - a useful reference point for what 'production tactile sensing' looks like right now. Vibration is one of the most underrated sources of noisy IMU and tactile readings - mounting matters as much as sensor quality, and a poorly isolated mount can add more noise than the sensor's own datasheet specs would suggest.

[george92]

@lperez This raises a question for me - Estimating joint torque from motor current draw is cheap and requires no extra sensor, but it's less accurate than a dedicated torque sensor because it doesn't capture friction losses through the gearbox - good enough for coarse control, not always for precise force-controlled tasks.

[pierregreen]

@george92 From what I've seen: Latency between a perceived event (like a slip) and a corrective control response matters enormously for balance - even 50-100ms of extra perception latency can be the difference between a smooth recovery and a fall, which is part of why a lot of balance-critical sensing is proprioceptive rather than vision-based.

[ethan_fisc]

Genuine beginner question - Depth sensing range and reliability both degrade outdoors in direct sunlight for most structured-light and active stereo cameras, since the ambient IR washes out the projected pattern - it's a real limitation for humanoids intended for anything beyond indoor, controlled environments. Tactile skin arrays have improved a lot, but 'good enough to matter' really depends on the task - coarse contact detection across a large area is fairly mature, while fine, high-resolution force distribution sensing (like a human fingertip) is still the harder problem.

[niklassantos]

This matches what I've seen too. Sensor fusion mostly earns its keep by covering for each individual sensor's weaknesses - vision struggles with occlusion and lighting, IMUs drift, force/torque sensors are noisy at low loads - fusing them gives a more robust estimate than any one source alone, independent of raw compute.