seb/saltylab-firmware
1
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity

feat(social): hand gesture pointing direction node (Issue #221) #226

Merged
sl-jetson merged 1 commits from sl-perception/issue-221-pointing into main 2026-03-02 11:57:27 -05:00
Conversation 0 Commits 1 Files Changed 6 +662

1 Commits

Author SHA1 Message Date
sl-perception
bd4a9f094d feat(social): hand gesture pointing direction node (Issue #221)
Some checks failed
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 2s
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
Details
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Details 
saltybot_social_msgs:
- Add PointingTarget.msg: origin (INDEX_MCP), direction (unit vec), target,
  range_m, person_id, confidence, coarse_direction, is_active
- Register in CMakeLists.txt

saltybot_social:
- _pointing_ray.py (pure Python, no rclpy): unproject(), sample_depth()
  (median with outlier rejection), compute_pointing_ray() — reprojects
  INDEX_MCP and INDEX_TIP into 3-D using D435i depth; falls back to image-
  plane direction when both depths are equal; gracefully handles one-sided
  missing depth
- pointing_node.py: subscribes /social/gestures + synced D435i colour+depth;
  re-runs MediaPipe Hands when a 'point' gesture is cached (within
  gesture_timeout_s); picks closest hand to gesture anchor; publishes
  PointingTarget on /saltybot/pointing_target at 5 Hz
- setup.py: adds pointing_node entry point
- 18/18 unit tests pass

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 11:56:54 -05:00