seb/saltylab-firmware
1
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity

[P0] Person tracking — RealSense D435i detection + following at 20km/h #363

New Issue
Closed
opened 2026-03-03 15:06:03 -05:00 by seb · 0 comments
seb commented 2026-03-03 15:06:03 -05:00
Owner
Copy Link

Goal

Track a specific person (Tee) using RealSense D435i color+depth and maintain lock at up to 20km/h outdoors.

Requirements

  • Detect and track a person in color stream (YOLO/MobileNet or similar, must run on Orin Nano)
  • Use depth stream for distance estimation (reliable to ~5m, degrade gracefully beyond)
  • Re-identification after brief occlusion (clothing color histogram, body proportions)
  • Output: target bearing (degrees), distance (meters), confidence, bounding box
  • Must run at ≥15fps on Jetson Orin Nano (CUDA/TensorRT)
  • ROS2 node: publish to /saltybot/target_track (custom msg: bearing, distance, confidence)

Hardware

  • RealSense D435i: serial 036422070649, FW 5.16.0.1
  • Depth: 640x480@30fps, 89.3% valid pixels, range 430-4948mm
  • Color: 640x480@30fps
  • librealsense v2.55.1 built from source with RSUSB backend

Notes

  • At 20km/h, target moves ~5.5m/s — prediction needed for smooth tracking
  • Camera alone WILL lose target on sharp turns, glare, occlusion → UWB fusion later (Issue TBD)
  • Must handle outdoor lighting (sun, shadows, rain)
## Goal Track a specific person (Tee) using RealSense D435i color+depth and maintain lock at up to 20km/h outdoors. ## Requirements - Detect and track a person in color stream (YOLO/MobileNet or similar, must run on Orin Nano) - Use depth stream for distance estimation (reliable to ~5m, degrade gracefully beyond) - Re-identification after brief occlusion (clothing color histogram, body proportions) - Output: target bearing (degrees), distance (meters), confidence, bounding box - Must run at ≥15fps on Jetson Orin Nano (CUDA/TensorRT) - ROS2 node: publish to /saltybot/target_track (custom msg: bearing, distance, confidence) ## Hardware - RealSense D435i: serial 036422070649, FW 5.16.0.1 - Depth: 640x480@30fps, 89.3% valid pixels, range 430-4948mm - Color: 640x480@30fps - librealsense v2.55.1 built from source with RSUSB backend ## Notes - At 20km/h, target moves ~5.5m/s — prediction needed for smooth tracking - Camera alone WILL lose target on sharp turns, glare, occlusion → UWB fusion later (Issue TBD) - Must handle outdoor lighting (sun, shadows, rain)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
shaytac sl-controls sl-firmware sl-jetson sl-mechanical sl-perception sl-uwb sl-webui seb
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: seb/saltylab-firmware#363
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?