feat(webui): live camera viewer — multi-stream + detection overlays (Issue #177) #182

sl-webui · 2026-03-02T10:47:27-05:00

sl-webui commented

2026-03-02 10:47:27 -05:00

Summary

useCamera.js: 7-camera subscription manager (front/left/rear/right CSI, D435i RGB/depth, panoramic); client-side 15fps gate; per-camera FPS tracking; MediaRecorder recording (MP4/WebM download); PNG snapshot with overlay composite + timestamp watermark; PiP list management (up to 3)
CameraViewer.jsx: full viewer UI — camera selector strip with FPS badges, detection overlay canvas (face boxes/names, gesture icons, scene object labels + hazard colours), 360° equirect panoramic viewer with drag-to-pan, up to 3 PiP mini windows, recording/snapshot controls, detection status readout, overlay mode selector (off/faces/gestures/objects/all)
App.jsx: CAMERAS tab added to TELEMETRY group
rosbridge_params.yaml: whitelist additions — /camera/depth/image_rect_raw/compressed, /camera/panoramic/compressed, /social/faces/detections, /social/gestures, /social/scene/objects
rosbridge.launch.py: D435i colour republisher (JPEG 75%) + D435i depth republisher (compressedDepth/PNG16 preserving uint16 depth values)

Test plan

Open Cameras tab — all 7 camera buttons visible
Connect robot — front/left/rear/right show FPS ≥ 12
D435i RGB stream shows colour image
D435i depth stream shows greyscale (darker = closer)
Panoramic: drag left/right pans equirect view, compass badge updates
Face detected → cyan box + name label on front camera overlay
Gesture detected → emoji icon at hand position
Scene object detected → dashed box + class/distance label on D435i tab
Pin 2 cameras → PiP windows appear at bottom-left
REC → stop → MP4/WebM download to Downloads folder
SNAP → PNG download includes overlay + timestamp watermark
Overlay mode OFF → no boxes drawn
ros2 launch saltybot_bringup rosbridge.launch.py → D435i topics appear compressed

Closes #177

🤖 Generated with Claude Code

## Summary - **useCamera.js**: 7-camera subscription manager (front/left/rear/right CSI, D435i RGB/depth, panoramic); client-side 15fps gate; per-camera FPS tracking; MediaRecorder recording (MP4/WebM download); PNG snapshot with overlay composite + timestamp watermark; PiP list management (up to 3) - **CameraViewer.jsx**: full viewer UI — camera selector strip with FPS badges, detection overlay canvas (face boxes/names, gesture icons, scene object labels + hazard colours), 360° equirect panoramic viewer with drag-to-pan, up to 3 PiP mini windows, recording/snapshot controls, detection status readout, overlay mode selector (off/faces/gestures/objects/all) - **App.jsx**: CAMERAS tab added to TELEMETRY group - **rosbridge_params.yaml**: whitelist additions — `/camera/depth/image_rect_raw/compressed`, `/camera/panoramic/compressed`, `/social/faces/detections`, `/social/gestures`, `/social/scene/objects` - **rosbridge.launch.py**: D435i colour republisher (JPEG 75%) + D435i depth republisher (compressedDepth/PNG16 preserving uint16 depth values) ## Test plan - [ ] Open Cameras tab — all 7 camera buttons visible - [ ] Connect robot — front/left/rear/right show FPS ≥ 12 - [ ] D435i RGB stream shows colour image - [ ] D435i depth stream shows greyscale (darker = closer) - [ ] Panoramic: drag left/right pans equirect view, compass badge updates - [ ] Face detected → cyan box + name label on front camera overlay - [ ] Gesture detected → emoji icon at hand position - [ ] Scene object detected → dashed box + class/distance label on D435i tab - [ ] Pin 2 cameras → PiP windows appear at bottom-left - [ ] REC → stop → MP4/WebM download to Downloads folder - [ ] SNAP → PNG download includes overlay + timestamp watermark - [ ] Overlay mode OFF → no boxes drawn - [ ] `ros2 launch saltybot_bringup rosbridge.launch.py` → D435i topics appear compressed Closes #177 🤖 Generated with [Claude Code](https://claude.com/claude-code)

sl-webui added 1 commit 2026-03-02 10:47:27 -05:00

feat(webui): live camera viewer — multi-stream + detection overlays (Issue #177 ) 57420807ca

UI (src/hooks/useCamera.js, src/components/CameraViewer.jsx):
  - 7 camera sources: front/left/rear/right CSI, D435i RGB/depth, panoramic
  - Compressed image subscription via rosbridge (sensor_msgs/CompressedImage)
  - Client-side 15fps gate (drops excess frames, reduces JS pressure)
  - Per-camera FPS indicator with quality badge (FULL/GOOD/LOW/NO SIGNAL)
  - Detection overlays: face boxes + names (/social/faces/detections),
    gesture icons (/social/gestures), scene object labels + hazard colours
    (/social/scene/objects); overlay mode selector (off/faces/gestures/objects/all)
  - 360° panoramic equirect viewer with mouse/touch drag azimuth pan
  - Picture-in-picture: up to 3 pinned cameras via ⊕ button
  - One-click recording (MediaRecorder → MP4/WebM download)
  - Snapshot to PNG with detection overlay composite + timestamp watermark
  - Cameras tab added to TELEMETRY group in App.jsx

Jetson (rosbridge bringup):
  - rosbridge_params.yaml: whitelist + /camera/depth/image_rect_raw/compressed,
    /camera/panoramic/compressed, /social/faces/detections,
    /social/gestures, /social/scene/objects
  - rosbridge.launch.py: D435i colour republisher (JPEG 75%) +
    depth republisher (compressedDepth/PNG16 preserving uint16 values)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

sl-jetson merged commit 728d1b0c0e into main

2026-03-02 10:48:50 -05:00

sl-jetson referenced this issue from a commit

2026-03-02 10:48:52 -05:00

Merge pull request 'feat(webui): live camera viewer — multi-stream + detection overlays (Issue #177)' (#182) from sl-webui/issue-177-camera-viewer into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: seb/saltylab-firmware#182