c620dc51a7
feat: Add Issue #363 — P0 person tracking for follow-me mode
...
Implements real-time person detection + tracking pipeline for the
follow-me motion controller on Jetson Orin Nano Super (D435i).
Core components
- TargetTrack.msg: bearing_deg, distance_m, confidence, bbox, vel_bearing_dps,
vel_dist_mps, depth_quality (0-3)
- _person_tracker.py (pure-Python, no ROS2/runtime deps):
· 8-state constant-velocity Kalman filter [cx,cy,w,h,vcx,vcy,vw,vh]
· Greedy IoU data association
· HSV torso colour histogram re-ID (16H×8S, Bhattacharyya similarity)
with fixed saturation clamping (s = (cmax−cmin)/cmax, clipped to [0,1])
· FollowTargetSelector: nearest person auto-lock, hold_frames hysteresis
· TENTATIVE→ACTIVE after min_hits; LOST track removal after max_lost_frames
with per-frame lost_age increment across all LOST tracks
· bearing_from_pixel, depth_at_bbox (median, quality flags)
- person_tracking_node.py:
· YOLOv8n via ultralytics (TRT FP16 on first run) → HOG+SVM fallback
· Subscribes colour + depth + camera_info + follow_start/stop
· Publishes /saltybot/target_track at ≤30 fps
- test/test_person_tracker.py: 59/59 tests passing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:19:02 -05:00
672120bb50
feat(perception): geometric face emotion classifier (Issue #359 )
...
Classifies facial expressions into neutral/happy/surprised/angry/sad
using geometric rules over MediaPipe Face Mesh landmarks — no ML model
required at runtime.
Rules
-----
surprised: brow_raise > 0.12 AND eye_open > 0.07 AND mouth_open > 0.07
happy: smile > 0.025 (lip corners above lip midpoint)
angry: brow_furl > 0.02 AND smile < 0.01
sad: smile < -0.025 AND brow_furl < 0.015
neutral: default
Changes
-------
- saltybot_scene_msgs/msg/FaceEmotion.msg — per-face emotion + features
- saltybot_scene_msgs/msg/FaceEmotionArray.msg
- saltybot_scene_msgs/CMakeLists.txt — register new msgs
- _face_emotion.py — pure-Python: FaceLandmarks, compute_features,
classify_emotion, detect_emotion, from_mediapipe
- face_emotion_node.py — subscribes /camera/color/image_raw,
publishes /saltybot/face_emotions (≤15 fps)
- test/test_face_emotion.py — 48 tests, all passing
- setup.py — add face_emotion entry point
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 14:39:49 -05:00
f7f89403d5
Merge pull request 'feat(social): system resource monitor for Jetson Orin (Issue #355 )' ( #357 ) from sl-jetson/issue-355-sysmon into main
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 4s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
2026-03-03 14:32:49 -05:00
677e6eb75e
feat(perception): MFCC nearest-centroid audio scene classifier (Issue #353 )
...
Classifies ambient audio into indoor/outdoor/traffic/park at 1 Hz using
a 16-d feature vector (13 MFCC + spectral centroid + rolloff + ZCR) with
a normalised nearest-centroid classifier. Centroids are computed at import
time from seeded synthetic prototypes, ensuring deterministic behaviour.
Changes
-------
- saltybot_scene_msgs/msg/AudioScene.msg — label + confidence + features[16]
- saltybot_scene_msgs/CMakeLists.txt — register AudioScene.msg
- _audio_scene.py — pure-numpy feature extraction + NearestCentroidClassifier
- audio_scene_node.py — subscribes /audio/audio, publishes /saltybot/audio_scene
- test/test_audio_scene.py — 53 tests (all passing) with synthetic audio
- setup.py — add audio_scene entry point
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 14:03:11 -05:00
0af4441120
feat(social): system resource monitor for Jetson Orin (Issue #355 )
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 9s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 9s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Polls /proc/stat (CPU delta), /proc/meminfo (RAM), os.statvfs (disk),
/sys/devices/gpu.0/load (GPU), and thermal zone sysfs paths; publishes
JSON payload on /saltybot/system_resources at 1 Hz.
Pure helpers (parse_proc_stat, cpu_percent_from_stats, parse_meminfo,
compute_ram_stats, read_disk_usage, read_gpu_load, read_thermal_zones)
are all unit-tested offline. Injectable I/O on SysmonNode allows full
node tick tests without /proc or /sys. 67/67 tests passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 13:54:31 -05:00
2a9b03dd76
feat(perception): depth-based obstacle size estimator (Issue #348 )
...
Projects LIDAR clusters into the D435i depth image to estimate 3-D
obstacle width and height in metres.
- saltybot_scene_msgs/msg/ObstacleSize.msg — new message
- saltybot_scene_msgs/msg/ObstacleSizeArray.msg — array wrapper
- saltybot_scene_msgs/CMakeLists.txt — register new msgs
- saltybot_bringup/_obstacle_size.py — pure-Python helper:
CameraParams (intrinsics + LIDAR→camera extrinsics)
ObstacleSizeEstimate (NamedTuple)
lidar_to_camera() LIDAR frame → camera frame transform
project_to_pixel() pinhole projection + bounds check
sample_depth_median() uint16 depth image window → median metres
estimate_height() vertical strip scan for row extent → height_m
estimate_cluster_size() full pipeline: cluster → size estimate
- saltybot_bringup/obstacle_size_node.py — ROS2 node
sub: /scan, /camera/depth/image_rect_raw, /camera/depth/camera_info
pub: /saltybot/obstacle_sizes (ObstacleSizeArray)
width from LIDAR bbox; height from depth strip back-projection;
graceful fallback (LIDAR-only) when depth image unavailable;
intrinsics latched from CameraInfo on first arrival
- test/test_obstacle_size.py — 33 tests, 33 passing
- setup.py — add obstacle_size entry
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 13:32:41 -05:00
813d6f2529
feat(social): trigger-based ROS2 bag recorder (Issue #332 )
...
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 12s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
BagRecorderNode: subscribes /saltybot/record_trigger (Bool), spawns
ros2 bag record subprocess, drives idle/recording/stopping/error
state machine; auto-stop timeout, SIGINT graceful shutdown with
SIGKILL fallback. Publishes /saltybot/recording_status (String).
Configurable topics (csv), bag_dir, prefix, compression, size limit.
Subprocess injectable for offline testing. 101/101 tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 13:19:49 -05:00
a0f3677732
Merge pull request 'feat: Add pure pursuit path follower for Nav2 (Issue #333 )' ( #334 ) from sl-controls/issue-333-pure-pursuit into main
2026-03-03 13:19:29 -05:00
1729e43964
feat(perception): MediaPipe hand tracking — Leap Motion pivot (Issue #342 )
...
PART 1 AUDIT: Zero Leap Motion / UltraLeap references found in any
saltybot_* package. Existing gesture_node.py (saltybot_social) already
uses MediaPipe — no cleanup required.
PART 2 NEW PACKAGES:
saltybot_hand_tracking_msgs (ament_cmake)
- HandLandmarks.msg — 21 landmarks (float32[63]), handedness,
gesture label + direction, wrist position
- HandLandmarksArray.msg
saltybot_hand_tracking (ament_python)
- _hand_gestures.py — pure-Python gesture classifier (no ROS2/MP deps)
Vocabulary: stop (open palm) → pause/stop,
point (index up) → direction command + 8-compass,
disarm (fist) → emergency-off,
confirm (thumbs-up) → confirm action,
follow_me (peace sign) → follow mode,
greeting (wrist oscillation) → greeting response
WaveDetector: sliding-window lateral wrist tracking
- hand_tracking_node.py — ROS2 node
sub: /camera/color/image_raw (BEST_EFFORT)
pub: /saltybot/hands (HandLandmarksArray)
/saltybot/hand_gesture (std_msgs/String)
MediaPipe model_complexity=0 (lite) for 20+ FPS
on Orin Nano Super; background MP init thread;
per-hand WaveDetector instances
- test/test_hand_gestures.py — 35 tests, 35 passing
Covers: Landmark, HandGestureResult, WaveDetector, all 6 gesture
classifiers, priority ordering, direction vectors, confidence bounds
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 12:47:22 -05:00
bd9cb6da35
feat(perception): lane/path edge detector (Issue #339 )
...
Adds Canny+Hough+bird-eye perspective pipeline for detecting left/right
path edges from the forward camera. Pure-Python helper (_path_edges.py)
is fully tested; ROS2 node publishes PathEdges on /saltybot/path_edges.
- saltybot_scene_msgs/msg/PathEdges.msg — new message
- saltybot_scene_msgs/CMakeLists.txt — register PathEdges.msg
- saltybot_bringup/_path_edges.py — PathEdgeConfig, PathEdgesResult,
build/apply_homography, canny_edges,
hough_lines, classify_lines,
average_line, warp_segments,
process_frame
- saltybot_bringup/path_edges_node.py — ROS2 node (sensor_msgs/Image →
PathEdges, parameters for all
tunable Canny/Hough/birdseye params)
- test/test_path_edges.py — 38 tests, 38 passing
- setup.py — add path_edges console_script
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 11:33:22 -05:00
eb61207532
feat(perception): dynamic obstacle velocity estimator (Issue #326 )
...
Adds ObstacleVelocity/ObstacleVelocityArray msgs and an
ObstacleVelocityNode that clusters /scan points, tracks each centroid
with a constant-velocity Kalman filter, and publishes velocity vectors
on /saltybot/obstacle_velocities.
New messages (saltybot_scene_msgs):
msg/ObstacleVelocity.msg — obstacle_id, centroid, velocity,
speed_mps, width_m, depth_m,
point_count, confidence, is_static
msg/ObstacleVelocityArray.msg — array wrapper with header
New files (saltybot_bringup):
saltybot_bringup/_obstacle_velocity.py — pure helpers (no ROS2 deps)
KalmanTrack constant-velocity 2-D KF: predict(dt) / update(centroid)
coasting counter → alive flag; confidence = age/n_init
associate() greedy nearest-centroid matching (O(N·M), strict <)
ObstacleTracker predict-all → associate → update/spawn → prune cycle
saltybot_bringup/obstacle_velocity_node.py
Subscribes /scan (BEST_EFFORT); reuses _lidar_clustering helpers;
publishes ObstacleVelocityArray on /saltybot/obstacle_velocities
Parameters: distance_threshold_m=0.20, min_points=3, range 0.05–12m,
max_association_dist_m=0.50, max_coasting_frames=5,
n_init_frames=3, q_pos=0.05, q_vel=0.50, r_pos=0.10,
static_speed_threshold=0.10
test/test_obstacle_velocity.py — 48 tests, all passing
Modified:
saltybot_scene_msgs/CMakeLists.txt — register new msgs
saltybot_bringup/setup.py — add obstacle_velocity console_script
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 06:53:04 -05:00
1c8430e68a
feat: Add pure pursuit path follower for Nav2 (Issue #333 )
...
Implements saltybot_pure_pursuit package:
- Pure pursuit algorithm for path following with configurable parameters
- Lookahead distance (0.5m default) for target point on path
- Goal tolerance (0.1m) for goal detection
- Heading error correction to reduce speed when misaligned with path
- Publishes Twist commands on /cmd_vel_tracked for Nav2 integration
- Subscribes to /odom (odometry) and /path (Path trajectory)
- Tracks and publishes cross-track error for monitoring
Pure pursuit geometry:
- Finds closest point on path to robot current position
- Looks ahead specified distance along path from closest point
- Computes steering angle to follow circular arc to lookahead point
- Reduces linear velocity when heading error is large (with correction enabled)
- Clamps velocities to configurable maximums
Configuration parameters:
- lookahead_distance: 0.5m (typical range: 0.1-1.0m)
- goal_tolerance: 0.1m (distance to goal before stopping)
- heading_tolerance: 0.1 rad (unused but can support in future)
- max_linear_velocity: 1.0 m/s
- max_angular_velocity: 1.57 rad/s
- use_heading_correction: true (reduces speed on large heading errors)
Comprehensive test suite: 20+ tests covering:
- Geometric calculations (distance, quaternion conversions)
- Path following logic (empty path, straight/curved/spiral paths)
- Steering calculations (heading errors, velocity limits)
- Edge cases and realistic scenarios
- Control loop integration
- Parameter variations
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-03 06:47:45 -05:00
4dbb4c6f0d
feat(perception): appearance-based person re-identification (Issue #322 )
...
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 12s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Adds PersonTrack/PersonTrackArray msgs and a PersonReidNode that matches
individuals across camera views using HSV colour histogram appearance
features and cosine similarity, with EMA gallery update and 30s stale timeout.
New messages (saltybot_scene_msgs):
msg/PersonTrack.msg — track_id, camera_id, bbox, confidence,
first_seen, last_seen, is_stale
msg/PersonTrackArray.msg — array wrapper with header
New files (saltybot_bringup):
saltybot_bringup/_person_reid.py — pure kinematics (no ROS2 deps)
extract_hsv_histogram() 2-D HS histogram (H=16, S=8 → 128-dim, L2-norm)
cosine_similarity() handles zero/non-unit vectors
match_track() best gallery match above threshold (strict >)
TrackGallery add/update/match/mark_stale/prune_stale
TrackEntry mutable dataclass; EMA feature blend (α=0.3)
saltybot_bringup/person_reid_node.py
Subscribes /camera/color/image_raw + /saltybot/scene/objects (BEST_EFFORT)
Crops COCO person (class_id=0) ROIs; extracts features; matches gallery
Publishes PersonTrackArray on /saltybot/person_tracks at 5 Hz
Parameters: camera_id, similarity_threshold=0.75, stale_timeout_s=30,
max_tracks=20, publish_hz=5.0
test/test_person_reid.py — 50 tests, all passing
Modified:
saltybot_scene_msgs/CMakeLists.txt — register PersonTrack/Array msgs
saltybot_bringup/setup.py — add person_reid console_script
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 06:45:43 -05:00
f506c89960
Merge pull request 'feat(social): USB camera hot-plug monitor (Issue #320 )' ( #328 ) from sl-jetson/issue-320-camera-hotplug into main
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 4s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
2026-03-03 06:45:23 -05:00
a3386e1694
feat(social): USB camera hot-plug monitor (Issue #320 )
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 5s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Polls /dev/video* at 2 Hz, drives a three-state machine
(connected/disconnected/restarting) and publishes to
/saltybot/camera_status (std_msgs/String). Reconnects within
restart_grace_s (5 s) → 'restarting' held for restart_hold_s (2 s)
to signal downstream capture pipelines to restart. Scan function
is injected for offline testing. 82/82 tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 00:47:27 -05:00
45828b1cc2
feat: Add battery-aware speed limiter ROS2 node (Issue #321 )
...
Implements saltybot_battery_speed_limiter package:
- Subscribes to /saltybot/battery_state, publishes speed limit factor
- Battery percentage thresholds: 100-50%=1.0, 50-25%=0.7, 25-15%=0.4, <15%=0.0
- Reduces speed to preserve battery when running low
- Automatically stops movement below 15% to prevent deep discharge
- Configurable thresholds and speed factors
- Comprehensive test suite: 30+ tests covering all threshold transitions
- Handles both direct percentage and voltage-based fallback calculation
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-03 00:46:12 -05:00
067a871103
feat(perception): wheel encoder differential drive odometry (Issue #184 )
...
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 7s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Adds saltybot_bridge_msgs package with WheelTicks.msg (int32 left/right
encoder counts) and a WheelOdomNode that subscribes to
/saltybot/wheel_ticks, integrates midpoint-Euler differential drive
kinematics (handling int32 counter rollover), and publishes
nav_msgs/Odometry on /odom_wheel at 50 Hz with optional TF broadcast.
New files:
jetson/ros2_ws/src/saltybot_bridge_msgs/
msg/WheelTicks.msg
CMakeLists.txt, package.xml
jetson/ros2_ws/src/saltybot_bringup/
saltybot_bringup/_wheel_odom.py — pure kinematics (no ROS2 deps)
saltybot_bringup/wheel_odom_node.py — 50 Hz timer node + TF broadcast
test/test_wheel_odom.py — 42 tests, all passing
Modified:
saltybot_bringup/package.xml — add saltybot_bridge_msgs, nav_msgs deps
saltybot_bringup/setup.py — add wheel_odom console_script entry
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 00:41:39 -05:00
b96c6b96d0
Merge pull request 'feat(social): audio wake-word detector 'hey salty' (Issue #320 )' ( #317 ) from sl-jetson/wake-word-detect into main
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 10s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 10s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
2026-03-03 00:41:22 -05:00
d6553ce3d6
feat(social): audio wake-word detector 'hey salty' (Issue #320 )
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 10s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Energy-gated log-mel + cosine-similarity wake-word node. Subscribes to
/social/speech/audio_raw (PCM-16 UInt8MultiArray), maintains a 1.5 s
sliding ring buffer, runs detection every 100 ms; fires Bool(True) on
/saltybot/wake_word_detected with 2 s cooldown. Template loaded from
.npy file; passive (no detections) when template_path is empty.
91/91 tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 00:26:59 -05:00
2919e629e9
feat: Add velocity smoothing filter with Butterworth low-pass filtering
...
Implements saltybot_velocity_smoother package:
- Subscribes to /odom, applies digital Butterworth low-pass filter
- Filters linear (x,y,z) and angular (x,y,z) velocity components independently
- Publishes smoothed odometry on /odom_smooth
- Reduces encoder jitter and improves state estimation stability
- Configurable filter order (1-4), cutoff frequency (Hz), publish rate
- Can be enabled/disabled via enable_smoothing parameter
- Comprehensive test suite: 18+ tests covering filter behavior, edge cases, scenarios
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-03 00:23:53 -05:00
d468cb515e
Merge pull request 'feat(social): personal space respector node (Issue #310 )' ( #314 ) from sl-jetson/issue-310-personal-space into main
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 5s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 15s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
2026-03-03 00:20:10 -05:00
0b35a61217
Merge pull request 'feat(perception): sky detector for outdoor navigation (Issue #307 )' ( #313 ) from sl-perception/issue-307-sky-detect into main
2026-03-03 00:20:03 -05:00
b8a14e2bfc
feat(social): personal space respector node (Issue #310 )
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 5s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 9s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Subscribes /social/faces/detected + /social/person_states; backs robot
away (cmd_vel linear.x = -backup_speed) when closest person <= 0.8 m,
latches /saltybot/too_close Bool; hysteresis band prevents chatter.
92/92 tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 21:42:33 -05:00
e24c0b2e26
feat(perception): sky detector for outdoor navigation — Issue #307
...
- Add _sky_detector.py: SkyResult NamedTuple; detect_sky() with dual HSV
band masking (blue sky H∈[90,130]/S∈[40,255]/V∈[80,255] OR overcast
S∈[0,50]/V∈[185,255]), cv2.bitwise_or combined mask; sky_fraction over
configurable top scan_frac region; horizon_y = bottommost row where
per-row sky fraction ≥ row_threshold (−1 when no sky detected)
- Add sky_detect_node.py: subscribes /camera/color/image_raw (BEST_EFFORT),
publishes Float32 /saltybot/sky_fraction and Int32 /saltybot/horizon_y
per frame; scan_frac (default 0.60) and row_threshold (default 0.30) params
- Register sky_detector console script in setup.py
- 33/33 unit tests pass (no ROS2 required)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 21:39:28 -05:00
fc87862603
feat(controls): Adaptive PID gain scheduler (Issue #309 )
...
Implements ROS2 adaptive PID gain scheduler for SaltyBot with:
- Subscribes to /saltybot/speed_scale for speed conditions
- Subscribes to /saltybot/terrain_roughness for surface conditions
- Adjusts PID gains dynamically:
* P gain increases with terrain roughness (better response on rough)
* D gain decreases at low speed (prevent oscillation when slow)
* I gain scales with both conditions for stability
- Publishes Float32MultiArray [Kp, Ki, Kd] on /saltybot/pid_gains
- Configurable scaling factors for each gain modulation
- Includes 18+ unit tests for gain scheduling logic
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-02 21:37:08 -05:00
becd0bc717
Merge pull request 'feat(perception): terrain roughness estimator via Gabor + LBP (Issue #296 )' ( #305 ) from sl-perception/issue-296-terrain-rough into main
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 9s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
2026-03-02 21:35:40 -05:00
a8838cfbbd
Merge pull request 'feat(social): conversation topic memory (Issue #299 )' ( #304 ) from sl-jetson/issue-299-topic-memory into main
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 13s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
2026-03-02 21:35:35 -05:00
3bf603f685
feat(perception): terrain roughness estimator via Gabor + LBP — Issue #296
...
- Add _terrain_roughness.py: RoughnessResult NamedTuple; gabor_energy() with
4-orientation × 2-wavelength (5px, 10px) quadrature Gabor bank, DC removal
via image mean subtraction (prevents false high energy on uniform surfaces);
lbp_variance() using 8-point radius-1 LBP in vectorised numpy slice
comparisons (no sklearn); estimate_roughness() with bottom roi_frac crop,
normalised blend roughness = 0.5*(gabor/500) + 0.5*(lbp/5000) clipped [0,1]
- Add terrain_rough_node.py: subscribes /camera/color/image_raw (BEST_EFFORT),
publishes Float32 /saltybot/terrain_roughness at 2Hz (configurable via
publish_hz param); roi_frac param default 0.40 (bottom 40% = floor region)
- Register terrain_roughness console script in setup.py
- 37/37 unit tests pass (no ROS2 required)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 21:12:51 -05:00
797ed711b9
feat(social): conversation topic memory — Issue #299
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 9s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 9s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Adds topic_memory_node to saltybot_social:
- Subscribes /social/conversation_text (String, JSON {person_id, text})
- Extracts keywords via stop-word filtered pipeline:
lowercase -> strip punctuation -> filter length/alpha -> remove
stop words -> deduplicate -> cap at max_keywords_per_msg
- Maintains per-person rolling topic history (PersonTopicMemory):
ordered list, dedup with recency promotion, capped at
max_topics_per_person, evicts oldest when full
- Prunes persons idle for > prune_after_s (default 30 min; 0=disabled)
- Publishes JSON on /saltybot/conversation_topics on each utterance:
{person_id, recent_topics (most-recent first), new_topics, ts}
- Enables recall like 'last time you mentioned X'
- 71/71 tests passing
Closes #299
2026-03-02 21:11:14 -05:00
bfd291cbdd
feat(controls): Geofence boundary enforcer (Issue #298 )
...
Implements ROS2 geofence enforcer for SaltyBot with:
- Loads polygon geofence from params (list of x/y vertices)
- Subscribes to /odom for real-time robot position
- Point-in-polygon ray casting algorithm for boundary checking
- Publishes Bool on /saltybot/geofence_breach on boundary violation
- Optional enforcement flag for cmd_vel zeroing
- Configurable safety margin
- Includes 20+ unit tests for geometry and breach detection
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-02 21:06:58 -05:00
b087af4b94
Merge pull request 'feat(social): speech volume auto-adjuster (Issue #289 )' ( #293 ) from sl-jetson/issue-289-volume-adjust into main
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 10s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
2026-03-02 21:05:28 -05:00
1a47abf5ae
Merge pull request 'feat(perception): motion blur detector via Laplacian variance (Issue #286 )' ( #292 ) from sl-perception/issue-286-blur-detect into main
2026-03-02 21:05:23 -05:00
e8b787a987
feat(social): speech volume auto-adjuster — Issue #289
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 9s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 8s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Adds volume_adjust_node to saltybot_social:
- Subscribes /social/speech/energy (Float32 linear RMS [0..1] from vad_node)
- Maps energy to TTS volume via:
1. Normalise into [noise_floor, noise_ceil]
2. Power-curve shaping (curve_gamma; <1 concave, 1 linear, >1 convex)
3. Lerp to [min_volume, max_volume]
4. Exponential moving-average smoothing (smoothing_alpha)
- Publishes Float32 on /saltybot/tts_volume at publish_rate Hz (default 5 Hz)
- Holds last value when energy is stale (> stale_timeout_s)
- All params exposed: min/max_volume, noise_floor/ceil, curve_gamma,
smoothing_alpha, publish_rate, stale_timeout_s
- 74/74 tests passing
Closes #289
2026-03-02 20:47:53 -05:00
c5f3a5b2ce
feat(perception): motion blur detector via Laplacian variance — Issue #286
...
- Add _blur_detector.py: BlurResult NamedTuple, laplacian_variance() (ksize=3
Laplacian on greyscale, with optional ROI crop), detect_blur() returning
variance + is_blurred flag + threshold; handles greyscale and BGR inputs,
empty ROI returns 0.0
- Add blur_detect_node.py: subscribes /camera/color/image_raw (BEST_EFFORT),
publishes Bool /saltybot/image_blurred and Float32 /saltybot/blur_score per
frame; threshold and roi_frac ROS params
- Register blur_detector console script in setup.py
- 25/25 unit tests pass (no ROS2 required)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 20:46:37 -05:00
dab0f96399
feat(controls): Cliff and drop-off detection safety node (Issue #288 )
...
Implements ROS2 cliff detector for SaltyBot with:
- Subscribes to /saltybot/cliff_sensors (IR range array)
- Threshold-based detection (default 0.5m)
- Debouncing (3 consecutive frames) for robustness
- Majority voting (min 2 sensors) for safety
- Publishes Bool on /saltybot/cliff_detected
- Emergency stop trigger on cliff/drop-off detection
- Includes 15+ unit tests for detection logic
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-02 20:45:21 -05:00
d1f0e95fa2
Merge pull request 'feat(social): face-tracking head servo controller (Issue #279 )' ( #284 ) from sl-jetson/issue-279-face-track-servo into main
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 14s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
2026-03-02 20:44:00 -05:00
5f3b5caef7
Merge pull request 'IMU calibration routine (Issue #278 )' ( #282 ) from sl-controls/issue-278-imu-cal into main
2026-03-02 20:43:52 -05:00
c3d36e9943
feat(social): face-tracking head servo controller — Issue #279
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 8s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 8s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Adds face_track_servo_node to saltybot_social:
- Subscribes /social/faces/detected (FaceDetectionArray)
- Picks closest face by largest bbox area (proximity proxy)
- Computes pan/tilt error from bbox centre vs image centre using
configurable FOV (fov_h_deg=60°, fov_v_deg=45°)
- Independent PID controllers for pan and tilt (velocity/incremental
output with anti-windup); servo position integrates velocity*dt
- Clamps commands to ±pan_limit_deg / ±tilt_limit_deg
- Returns to centre at return_rate_deg_s when face lost >lost_timeout_s
- Dead zone suppresses jitter for small errors
- Publishes Float32 on /saltybot/head_pan and /saltybot/head_tilt
- 81/81 tests passing
Closes #279
2026-03-02 17:38:02 -05:00
dd033b9827
feat(controls): IMU calibration routine (Issue #278 )
...
Implements ROS2 IMU gyro + accel calibration node with:
- Service-triggered calibration via /saltybot/calibrate_imu
- Optional auto-calibration on startup (configurable)
- Collects N stationary samples (default 100)
- Computes mean bias offsets for gyro and accel
- Publishes bias-corrected IMU on /imu/calibrated
- Includes 10+ unit tests for calibration logic
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-02 17:35:30 -05:00
f5093ecd34
feat(perception): HSV color object segmenter — Issue #274
...
- Add ColorDetection.msg + ColorDetectionArray.msg to saltybot_scene_msgs
- Add _color_segmenter.py: HsvRange/ColorBlob types, COLOR_RANGES defaults,
mask_for_color() (dual-band red wrap), find_color_blobs() with morph open,
contour extraction, area filter and max-blob-per-color limit
- Add color_segment_node.py: subscribes /camera/color/image_raw (BEST_EFFORT),
publishes /saltybot/color_objects (ColorDetectionArray) per frame;
active_colors, min_area_px, max_blobs_per_color params
- Add saltybot_scene_msgs exec_depend to saltybot_bringup/package.xml
- Register color_segmenter console_script in setup.py
- 34/34 unit tests pass (no ROS2 required)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 17:32:41 -05:00
30ad71e7d8
Merge pull request 'feat(social): proximity-based greeting trigger (Issue #270 )' ( #272 ) from sl-jetson/issue-270-greeting-trigger into main
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 10s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
2026-03-02 17:30:54 -05:00
5108fa8fa1
feat(controls): Wheel slip detector (Issue #262 )
...
Detect wheel slip by comparing commanded velocity vs actual encoder velocity.
Publishes Bool flag on /saltybot/wheel_slip_detected when slip detected >0.5s.
Features:
- Subscribe to /cmd_vel (commanded) and /odom (actual velocity)
- Compare velocity magnitudes with 0.1 m/s threshold
- Persistence: slip must persist >0.5s to trigger (debounces transients)
- Publish Bool on /saltybot/wheel_slip_detected with detection status
- 10Hz monitoring frequency, configurable parameters
Algorithm:
- Compute linear speed from x,y components
- Calculate velocity difference
- If exceeds threshold: increment slip duration
- If duration > timeout: declare slip detected
Benefits:
- Detects environmental slip (ice, mud, wet surfaces)
- Triggers speed reduction to maintain traction
- Prevents wheel spinning/rut digging
- Safety response for loss of grip
Topics:
- Subscribed: /cmd_vel (Twist), /odom (Odometry)
- Published: /saltybot/wheel_slip_detected (Bool)
Config: frequency=10Hz, slip_threshold=0.1 m/s, slip_timeout=0.5s
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-02 17:29:03 -05:00
c7dd07f9ed
feat(social): proximity-based greeting trigger — Issue #270
...
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 12s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Adds greeting_trigger_node to saltybot_social:
- Subscribes to /social/faces/detected (FaceDetectionArray) for face arrivals
- Subscribes to /social/person_states (PersonStateArray) to cache face_id→distance
- Fires greeting when face_id is within proximity_m (default 2m) and
not in per-face_id cooldown window (default 300s)
- Publishes JSON on /saltybot/greeting_trigger:
{face_id, person_name, distance_m, ts}
- unknown_distance param controls assumed distance for faces with no PersonState yet
- Thread-safe distance cache and greeted map
- 50/50 tests passing
Closes #270
2026-03-02 17:26:40 -05:00
f0e11fe7ca
feat(bringup): depth image hole filler via bilateral interpolation (Issue #268 )
...
Adds multi-pass spatial-Gaussian hole filler for D435i depth images.
Each pass replaces zero/NaN pixels with the Gaussian-weighted mean of valid
neighbours in a growing kernel (×1, ×2.5, ×6 default); original valid
pixels are never modified. Handles uint16 mm → float32 m conversion,
border pixels via BORDER_REFLECT, and above-d_max pixels as holes.
Publishes filled float32 depth on /camera/depth/filled at camera rate.
37/37 pure-Python tests pass (no ROS2 required).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 14:19:27 -05:00
477258f321
Merge pull request 'feat(webui): waypoint editor with click-to-navigate ( #261 )' ( #267 ) from sl-webui/issue-261-waypoint-editor-fix into main
2026-03-02 14:13:54 -05:00
94a6f0787e
Merge pull request 'feat(bringup): visual odometry drift detector (Issue #260 )' ( #265 ) from sl-perception/issue-260-vo-drift into main
2026-03-02 14:13:34 -05:00
c865e84e16
feat(webui): waypoint editor with click-to-navigate (Issue #261 )
...
Interactive waypoint editor for Nav2 goal-based navigation:
- Click on map display to place waypoints
- Drag waypoints in list to reorder navigation sequence
- Right-click waypoints to delete them
- Visual waypoint overlay on map with numbering
- Robot position indicator at center
- Waypoint list sidebar with selection and ordering
- Send Nav2 goal to individual selected waypoint
- Execute all waypoints in sequence with automatic progression
- Clear all waypoints button
- Real-time coordinate display and robot pose tracking
- Integrated into new NAVIGATION tab group
- Uses /navigate_to_pose service for goal publishing
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-02 13:28:01 -05:00
9d12805843
feat(bringup): visual odometry drift detector (Issue #260 )
...
Adds sliding-window drift detector that compares cumulative path lengths
of visual odom and wheel odom over a configurable window (default 10 s).
Drift = |vo_path − wheel_path|; flagged when ≥ 0.5 m (configurable).
OdomBuffer handles per-source rolling storage with automatic age eviction.
Publishes Bool on /saltybot/vo_drift_detected and Float32 on
/saltybot/vo_drift_magnitude at 2 Hz. 27/27 pure-Python tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 13:26:07 -05:00
3cd9faeed9
feat(social): ambient sound classifier via mel-spectrogram — Issue #252
...
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Adds ambient_sound_node to saltybot_social:
- Accumulates 1 s of PCM-16 audio from /social/speech/audio_raw
- Extracts mel-spectrogram feature vector (energy_db, zcr, mel_centroid,
mel_flatness, low_ratio, high_ratio) using pure numpy (no torch/onnx)
- Priority-cascade classifier: silence → music → speech → crowd → outdoor → alarm
- Publishes label as std_msgs/String on /saltybot/ambient_sound on each buffer fill
- All 11 thresholds exposed as ROS parameters (yaml + launch file)
- numpy-free energy-only fallback for edge environments
- 77/77 tests passing
Closes #252
2026-03-02 13:22:38 -05:00
a55cd9c97f
Merge pull request 'feat(bringup): floor surface type classifier on D435i RGB (Issue #249 )' ( #256 ) from sl-perception/issue-249-floor-classifier into main
2026-03-02 13:22:17 -05:00