Add vad_node to saltybot_social: subscribes to /social/speech/audio_raw
(UInt8MultiArray PCM-16), computes RMS energy (dBFS) and zero-crossing
rate per chunk, applies onset/offset hysteresis (VadStateMachine), and
publishes /social/speech/is_speaking (Bool) and /social/speech/energy
(Float32 linear RMS). All thresholds configurable via ROS params:
rms_threshold_db=-35.0, zcr_min=0.01, zcr_max=0.40, onset_frames=2,
offset_frames=8, audio_topic. 69/69 tests passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements a chat-style conversation viewer that subscribes to
/social/conversation_text and displays user speech (STT) and robot
responses (TTS) with timestamps and speaker labels. Includes auto-scroll
to latest message, manual scroll detection, and message history limiting.
- New component: ConversationHistory.jsx (chat-style message bubbles)
- User messages in blue, robot responses in green
- Auto-scrolling with manual scroll detection toggle
- Timestamp formatting (HH:MM:SS)
- Message history limiting (max 100 messages)
- Clear history button
- Integrated into SOCIAL tab group as "History" tab
- Subscribes to /social/conversation_text topic
(saltybot_social_msgs/ConversationMessage)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Adds gap-based Euclidean distance clustering of /scan LaserScan points.
Each cluster is published as a labelled semi-transparent CUBE + TEXT marker
in /saltybot/lidar_clusters (MarkerArray), sorted nearest-first. Stale
markers from shrinking cluster counts are explicitly deleted each cycle.
22/22 pure-Python tests pass (no ROS2 required).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add real-time audio level visualization with VU-style meter:
- Responsive VU bar with color gradient (silent to clipping)
- Peak hold indicator with exponential decay
- Speech activity detection from /social/speech/is_speaking
- Color-coded audio levels with visual feedback
- Grid markers for level reference (25%, 50%, 75%)
- Comprehensive audio statistics (average, max, peak count)
Features:
- Dynamic color coding: Gray (silent) → Red (clipping)
- Level thresholds: Silent, Low, Moderate, Good, Loud, Clipping
- Peak hold with 1-second hold time + 5% decay per 50ms
- Speech activity indicator with pulsing animation
- 100-sample rolling average for statistics
- Real-time metric updates
Visual Elements:
- Main VU bar with smooth fill animation
- Separate peak hold display with glow effect
- Color reference legend (all 6 levels)
- Statistics panel (average, max, peak holds)
- Grid-based scale (0-100%)
- Speech status badge (SPEAKING/SILENT)
Integration:
- Added to SOCIAL tab as new "Audio" tab
- Subscribes to /saltybot/audio_level and /social/speech/is_speaking
- Properly formatted topic info footer
- Responsive design matching dashboard theme
Build: ✓ Passing (113 modules, 202.67 KB main bundle)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Adds cv2.QRCodeDetector-based QR reader that subscribes to all four IMX219
CSI camera streams, deduplicates detections with a 2 s per-payload cooldown,
and publishes /saltybot/qr_codes (QRDetectionArray) at 10 Hz. New
QRDetection / QRDetectionArray messages added to saltybot_scene_msgs.
16/16 pure-Python tests pass (no ROS2 required).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add real-time audio level visualization with VU-style meter:
- Responsive VU bar with color gradient (silent to clipping)
- Peak hold indicator with exponential decay
- Speech activity detection from /social/speech/is_speaking
- Color-coded audio levels with visual feedback
- Grid markers for level reference (25%, 50%, 75%)
- Comprehensive audio statistics (average, max, peak count)
Features:
- Dynamic color coding: Gray (silent) → Red (clipping)
- Level thresholds: Silent, Low, Moderate, Good, Loud, Clipping
- Peak hold with 1-second hold time + 5% decay per 50ms
- Speech activity indicator with pulsing animation
- 100-sample rolling average for statistics
- Real-time metric updates
Visual Elements:
- Main VU bar with smooth fill animation
- Separate peak hold display with glow effect
- Color reference legend (all 6 levels)
- Statistics panel (average, max, peak holds)
- Grid-based scale (0-100%)
- Speech status badge (SPEAKING/SILENT)
Integration:
- Added to SOCIAL tab as new "Audio" tab
- Subscribes to /saltybot/audio_level and /social/speech/is_speaking
- Properly formatted topic info footer
- Responsive design matching dashboard theme
Build: ✓ Passing (113 modules, 202.67 KB main bundle)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Adds per-person constant-velocity Kalman filter that smooths raw 68-point
facial landmarks and republishes on /social/faces/landmarks_smooth at input
rate. New FaceLandmarks / FaceLandmarksArray messages added to
saltybot_social_msgs. 21/21 pure-Python tests pass (no ROS2 required).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace basic IMU panel with interactive Three.js 3D pose viewer:
- Real-time robot box model visualization
- Rotation controlled by /saltybot/imu quaternion data
- 30-second position trail from /odom navigation
- Trail visualization with point history
- Reset button to clear trail history
- Grid ground plane and directional indicator arrow
- Interactive 3D scene with proper lighting and shadows
Features:
- Three.js scene with dynamic box model (0.3x0.3x0.5m)
- Forward direction indicator (amber cone)
- Ambient and directional lighting with shadows
- Grid helper for spatial reference
- Trail line geometry with dynamic point updates
- Automatic window resize handling
- Proper cleanup of WebGL resources and subscriptions
Integration:
- Replaces ImuPanel in TELEMETRY tab (IMU view)
- Added to flex layout for proper 3D viewport sizing
- Subscribes to /saltybot/imu and /odom topics
- Updates trail points every 100ms (max 30-second history)
Build: ✓ Passing (112 modules, 196.69 KB main bundle)
Three.js vendor: 473.57 KB
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
saltybot_social_msgs:
- Add PointingTarget.msg: origin (INDEX_MCP), direction (unit vec), target,
range_m, person_id, confidence, coarse_direction, is_active
- Register in CMakeLists.txt
saltybot_social:
- _pointing_ray.py (pure Python, no rclpy): unproject(), sample_depth()
(median with outlier rejection), compute_pointing_ray() — reprojects
INDEX_MCP and INDEX_TIP into 3-D using D435i depth; falls back to image-
plane direction when both depths are equal; gracefully handles one-sided
missing depth
- pointing_node.py: subscribes /social/gestures + synced D435i colour+depth;
re-runs MediaPipe Hands when a 'point' gesture is cached (within
gesture_timeout_s); picks closest hand to gesture anchor; publishes
PointingTarget on /saltybot/pointing_target at 5 Hz
- setup.py: adds pointing_node entry point
- 18/18 unit tests pass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements complete I2C1 driver for TI INA219 power monitoring IC supporting:
- Dual sensors on I2C1 (left motor @ 0x40, right motor @ 0x41)
- Auto-calibration for 5A max current, 0.1Ω shunt resistance
- Current LSB: 153µA, Power LSB: 3060µW (20× current LSB)
- Bus voltage: 0-26V @ 4mV/LSB (13-bit, 4mV resolution)
- Shunt voltage: ±327mV @ 10µV/LSB (signed 16-bit)
- Calibration register computation for arbitrary max current/shunt values
- Efficient single/batch read functions (voltage, current, power)
- Alert threshold configuration for overcurrent protection
- Full test suite: 12 passing unit tests covering calibration, conversions, edge cases
Integration:
- ina219_init() called after i2c1_init() in main startup sequence
- Ready for motor power monitoring and thermal protection logic
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Two files added to saltybot_bringup:
- _scan_height_filter.py: pure-Python helpers (no rclpy) —
filter_scan_by_height() projects each LIDAR ray to world-frame height
using pitch/roll from the IMU and filters ground/ceiling returns;
pitch_roll_from_accel() uses convention-agnostic atan2 formula;
AttitudeEstimator low-pass filters the accelerometer attitude.
- scan_height_filter_node.py: subscribes /scan + /camera/imu, publishes
/scan_filtered (LaserScan) for Nav2 at source rate (up to 20 Hz).
setup.py: adds scan_height_filter entry point.
18/18 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements TIM4 PWM driver for 2-servo camera mount with:
- 50 Hz PWM frequency (standard servo control)
- CH1 (PB6) pan servo, CH2 (PB7) tilt servo
- 0-180° angle range → 500-2500 µs pulse width mapping
- Non-blocking servo_set_angle() for immediate positioning
- servo_sweep() for smooth pan-tilt animation (linear interpolation)
- Independent sweep control per servo (pan and tilt move simultaneously)
- 15 comprehensive unit tests covering all scenarios
Integration:
- servo_init() called at startup after power_mgmt_init()
- servo_tick(now_ms) called every 1ms in main loop
- Ready for camera/gimbal control automation
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Two new packages:
- saltybot_person_reid_msgs: PersonAppearance + PersonAppearanceArray msgs
- saltybot_person_reid: MobileNetV2 torso-crop embedder (128-dim L2-norm)
with 128-bin HSV histogram fallback, cosine-similarity gallery with EMA
identity updates and configurable age-based pruning, ROS2 node publishing
PersonAppearanceArray on /saltybot/person_reid at 5 Hz.
Pure-Python helpers (_embedding_model, _reid_gallery) importable without
rclpy — 18/18 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New saltybot_thermal package with thermal_node: reads all
/sys/class/thermal/thermal_zone* sysfs entries (millidegrees→°C),
publishes /saltybot/thermal JSON at 1 Hz with zones[], max_temp_c,
warn, and throttled flags. Logs ROS2 WARN at ≥75°C, ERROR at ≥85°C.
thermal_root param allows sysfs override for offline testing.
50/50 tests passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements node watchdog ROS2 node that monitors heartbeats from critical
systems and triggers safety fallback when motor driver is lost >2s.
Features:
- Monitor heartbeats from: balance, motor_driver, emergency, docking
- Alert on /saltybot/node_watchdog (JSON) if heartbeat lost >1s
- Safety fallback: zero cmd_vel if motor driver lost >2s
- Republish cmd_vel on /saltybot/cmd_vel_safe with safety checks
- 20Hz monitoring and publishing frequency
- Configurable heartbeat timeout thresholds
Heartbeat Topics:
- /saltybot/balance_heartbeat (std_msgs/UInt32)
- /saltybot/motor_driver_heartbeat (std_msgs/UInt32)
- /saltybot/emergency_heartbeat (std_msgs/UInt32)
- /saltybot/docking_heartbeat (std_msgs/UInt32)
- /cmd_vel (geometry_msgs/Twist)
Published Topics:
- /saltybot/node_watchdog (std_msgs/String) - JSON status
- /saltybot/cmd_vel_safe (geometry_msgs/Twist) - Safety-checked velocity
Package: saltybot_node_watchdog
Entry point: node_watchdog_node
Launch file: node_watchdog.launch.py
Tests: 20+ unit tests covering:
- Heartbeat reception and timeout detection
- Motor driver critical timeout (>2s)
- Safety fallback logic
- cmd_vel zeroing on motor driver loss
- Health status JSON serialization
- Multi-node failure scenarios
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
