seb/saltylab-firmware
1
0
Fork 0
Code Issues Pull Requests 5 Actions Packages Projects Releases Wiki Activity

feat(webui): hand tracking skeleton visualization (Issue #344) #346

Merged
sl-jetson merged 1 commits from sl-webui/issue-344-hand-viz into main 2026-03-03 13:19:47 -05:00
Conversation 0 Commits 1 Files Changed 2 +336
sl-jetson commented 2026-03-03 13:19:39 -05:00
Collaborator
Copy Link

HandTracker.jsx — 21-landmark hand skeleton with bone connections, gesture label, dual hand support, confidence indicators. Added to SOCIAL tab group.

HandTracker.jsx — 21-landmark hand skeleton with bone connections, gesture label, dual hand support, confidence indicators. Added to SOCIAL tab group.
sl-jetson added 1 commit 2026-03-03 13:19:40 -05:00 
Features:
- Subscribes to /saltybot/hands (21 landmarks per hand - MediaPipe format)
- Subscribes to /saltybot/hand_gesture (String gesture label)
- Canvas-based hand skeleton rendering with bone connections
- Support for dual hand tracking (left and right)
- Handedness indicators with color coding
  * Left hand: green
  * Right hand: yellow
- Real-time gesture display with confidence indicator
- Per-landmark confidence visualization
- Bone connections between all 21 joints

Hand Skeleton Features:
- 21 MediaPipe landmarks per hand
  * Wrist (1)
  * Thumb (4)
  * Index finger (4)
  * Middle finger (4)
  * Ring finger (4)
  * Pinky finger (4)
- 20 bone connections between joints
- Confidence-based rendering (only show high-confidence points)
- Scaling and normalization for viewport
- Joint type indicators (tips with ring outline)
- Glow effects around landmarks

Gesture Recognition:
- Real-time gesture label display
- Confidence percentage (0-100%)
- Color-coded confidence:
  * Green: >80% (high confidence)
  * Yellow: 50-80% (medium confidence)
  * Blue: <50% (detecting)

Hand Status Display:
- Live detection status for both hands
- Visual indicators (✓ detected / ◯ not detected)
- Dual-hand canvas rendering
- Gesture info panel with confidence bar

Integration:
- Added to SOCIAL tab group as "Hands" tab
- Positioned after "Faces" tab
- Uses subscribe hook for real-time updates
- Dark theme with color-coded hands
- Canvas-based rendering for smooth visualization

Build: 125 modules, no errors
Main bundle: 270.08 KB

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
sl-jetson merged commit 9b538395c0 into main 2026-03-03 13:19:47 -05:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
shaytac sl-controls sl-firmware sl-jetson sl-mechanical sl-perception sl-uwb sl-webui seb
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: seb/saltylab-firmware#346
No description provided.
Delete Branch "sl-webui/issue-344-hand-viz"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?