e247389b07
feat: Add Issue #203 - Node watchdog monitor (20Hz heartbeat detection)
...
Implements node watchdog ROS2 node that monitors heartbeats from critical
systems and triggers safety fallback when motor driver is lost >2s.
Features:
- Monitor heartbeats from: balance, motor_driver, emergency, docking
- Alert on /saltybot/node_watchdog (JSON) if heartbeat lost >1s
- Safety fallback: zero cmd_vel if motor driver lost >2s
- Republish cmd_vel on /saltybot/cmd_vel_safe with safety checks
- 20Hz monitoring and publishing frequency
- Configurable heartbeat timeout thresholds
Heartbeat Topics:
- /saltybot/balance_heartbeat (std_msgs/UInt32)
- /saltybot/motor_driver_heartbeat (std_msgs/UInt32)
- /saltybot/emergency_heartbeat (std_msgs/UInt32)
- /saltybot/docking_heartbeat (std_msgs/UInt32)
- /cmd_vel (geometry_msgs/Twist)
Published Topics:
- /saltybot/node_watchdog (std_msgs/String) - JSON status
- /saltybot/cmd_vel_safe (geometry_msgs/Twist) - Safety-checked velocity
Package: saltybot_node_watchdog
Entry point: node_watchdog_node
Launch file: node_watchdog.launch.py
Tests: 20+ unit tests covering:
- Heartbeat reception and timeout detection
- Motor driver critical timeout (>2s)
- Safety fallback logic
- cmd_vel zeroing on motor driver loss
- Health status JSON serialization
- Multi-node failure scenarios
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-02 11:17:19 -05:00
bc45208768
feat: Add Issue #194 - Speed limiter node (50Hz safety constraints)
...
Implements speed limiter ROS2 node that enforces safety constraints from:
- Terrain type (via /saltybot/terrain_speed_scale)
- Battery level (via /saltybot/speed_limit)
- Emergency system state (via /saltybot/emergency)
Node computes minimum scale factor and applies to /cmd_vel, forwarding
limited commands to /saltybot/cmd_vel_limited. Publishes JSON telemetry
on /saltybot/speed_limit_info at 50Hz (20ms cycle).
Features:
- Subscription to terrain, battery, and emergency constraints
- Twist velocity scaling with min-of-three constraint logic
- JSON telemetry with timestamp and all scale factors
- 50Hz timer-based publishing for real-time control
- Emergency state mapping (NOMINAL/RECOVERING/STOPPING/ESCALATED)
- Comprehensive unit tests (18 test cases)
Package: saltybot_speed_limiter
Entry point: speed_limiter_node
Launch file: speed_limiter.launch.py
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-02 11:14:10 -05:00
e6065e1531
feat(jetson): camera health watchdog node (issue #198 )
...
Adds camera_health_node.py + _camera_state.py to saltybot_bringup:
• _camera_state.py — pure-Python CameraState dataclass (no ROS2):
on_frame(), age_s, fps(window_s), status(),
should_reset() + mark_reset() with 30s cooldown
• camera_health_node.py — subscribes 6 image topics (D435i color/depth
+ 4× IMX219 CSI front/right/rear/left);
1 Hz tick: WARNING at >2s silence, ERROR at
>10s + v4l2 stream-off/on reset for CSI cams;
publishes /saltybot/camera_health JSON with
per-camera status, age_s, fps, total_frames
• test/test_camera_health.py — 15 unit tests (15/15 pass, no ROS2 needed)
• setup.py — adds camera_health_monitor console_scripts entry point
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 11:11:48 -05:00
d6d7e7b75a
feat(jetson): AprilTag landmark detector — DICT_APRILTAG_36h11 (issue #191 )
...
saltybot_apriltag_msgs (ament_cmake):
• DetectedTag.msg — tag_id, family, decision_margin, corners[8],
center, 6-DOF pose + pose_valid flag
• DetectedTagArray.msg — DetectedTag[], count
saltybot_apriltag (ament_python, single node):
• _aruco_utils.py — ROS2-free: ArucoBackend (cv2 4.6/4.7+ API shim),
rvec_tvec_to_pose (Rodrigues → Shepperd quaternion)
• apriltag_node.py — 10 Hz timer; subscribes image + latched camera_info;
cv2.solvePnP IPPE_SQUARE for 6-DOF pose when K
available; publishes /saltybot/apriltags
• test/test_apriltag.py — 11 unit tests (11/11 pass, no ROS2 needed):
pose math + rendered tag detection + multi-tag
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 11:07:35 -05:00
c26293d000
feat(jetson): depth confidence filter node (issue #190 )
...
Adds depth_confidence_filter_node.py to saltybot_bringup:
- Synchronises /camera/depth/image_rect_raw + /camera/depth/confidence
via ApproximateTimeSynchronizer (10ms slop)
- Zeros pixels where confidence uint8 < threshold * 255 (default 0.5)
- Republishes filtered float32 depth on /camera/depth/filtered
- Registered as depth_confidence_filter console_scripts entry point
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 11:02:15 -05:00
90c8b427fc
feat(social): multi-language support — Whisper LID + per-lang Piper TTS (Issue #167 )
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 10s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
- Add SpeechTranscript.language (BCP-47), ConversationResponse.language fields
- speech_pipeline_node: whisper_language param (""=auto-detect via Whisper LID);
detected language published in every transcript
- conversation_node: track per-speaker language; inject "[Please respond in X.]"
hint for non-English speakers; propagate language to ConversationResponse.
_LANG_NAMES: 24 BCP-47 codes -> English names. Also adds Issue #161 emotion
context plumbing (co-located in same branch for clean merge)
- tts_node: voice_map_json param (JSON BCP-47->ONNX path); lazy voice loading
per language; playback queue now carries (text, lang) tuples for voice routing
- speech_params.yaml, tts_params.yaml: new language params with docs
- 47/47 tests pass (test_multilang.py)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:57:34 -05:00
728d1b0c0e
Merge pull request 'feat(webui): live camera viewer — multi-stream + detection overlays (Issue #177 )' ( #182 ) from sl-webui/issue-177-camera-viewer into main
2026-03-02 10:48:50 -05:00
57420807ca
feat(webui): live camera viewer — multi-stream + detection overlays (Issue #177 )
...
UI (src/hooks/useCamera.js, src/components/CameraViewer.jsx):
- 7 camera sources: front/left/rear/right CSI, D435i RGB/depth, panoramic
- Compressed image subscription via rosbridge (sensor_msgs/CompressedImage)
- Client-side 15fps gate (drops excess frames, reduces JS pressure)
- Per-camera FPS indicator with quality badge (FULL/GOOD/LOW/NO SIGNAL)
- Detection overlays: face boxes + names (/social/faces/detections),
gesture icons (/social/gestures), scene object labels + hazard colours
(/social/scene/objects); overlay mode selector (off/faces/gestures/objects/all)
- 360° panoramic equirect viewer with mouse/touch drag azimuth pan
- Picture-in-picture: up to 3 pinned cameras via ⊕ button
- One-click recording (MediaRecorder → MP4/WebM download)
- Snapshot to PNG with detection overlay composite + timestamp watermark
- Cameras tab added to TELEMETRY group in App.jsx
Jetson (rosbridge bringup):
- rosbridge_params.yaml: whitelist + /camera/depth/image_rect_raw/compressed,
/camera/panoramic/compressed, /social/faces/detections,
/social/gestures, /social/scene/objects
- rosbridge.launch.py: D435i colour republisher (JPEG 75%) +
depth republisher (compressedDepth/PNG16 preserving uint16 values)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:47:01 -05:00
9ca0e0844c
Merge pull request 'feat(social): facial expression recognition — TRT FP16 emotion CNN (Issue #161 )' ( #180 ) from sl-jetson/issue-161-emotion into main
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 10s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
2026-03-02 10:46:21 -05:00
54668536c1
Merge pull request 'feat(jetson): dynamic obstacle tracking — LIDAR motion detection, Kalman tracking, trajectory prediction, Nav2 costmap ( #176 )' ( #181 ) from sl-perception/issue-176-dynamic-obstacles into main
2026-03-02 10:45:22 -05:00
2f4540f1d3
feat(jetson): add dynamic obstacle tracking package (issue #176 )
...
Implements real-time moving obstacle detection, Kalman tracking, trajectory
prediction, and Nav2 costmap integration at 10 Hz / <50ms latency:
saltybot_dynamic_obs_msgs (ament_cmake):
• TrackedObject.msg — id, PoseWithCovariance, velocity, predicted_path,
predicted_times, speed, confidence, age, hits
• MovingObjectArray.msg — TrackedObject[], active_count, tentative_count,
detector_latency_ms
saltybot_dynamic_obstacles (ament_python):
• object_detector.py — LIDAR background subtraction (EMA occupancy grid),
foreground dilation + scipy connected-component
clustering → Detection list
• kalman_tracker.py — CV Kalman filter, state [px,py,vx,vy], Joseph-form
covariance update, predict_horizon() (non-mutating)
• tracker_manager.py — up to 20 tracks, Hungarian assignment
(scipy.optimize.linear_sum_assignment), TENTATIVE→
CONFIRMED lifecycle, miss-prune
• dynamic_obs_node.py — 10 Hz timer: detect→track→publish
/saltybot/moving_objects + MarkerArray viz
• costmap_layer_node.py — predicted paths → PointCloud2 inflation smear
→ /saltybot/dynamic_obs_cloud for Nav2 ObstacleLayer
• launch/dynamic_obstacles.launch.py + config/dynamic_obstacles_params.yaml
• test/test_dynamic_obstacles.py — 27 unit tests (27/27 pass, no ROS2 needed)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:44:32 -05:00
50971c0946
feat(social): facial expression recognition — TRT FP16 emotion CNN (Issue #161 )
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
- Add Expression.msg / ExpressionArray.msg ROS2 message definitions
- Add emotion_classifier.py: 7-class CNN (happy/sad/angry/surprised/fearful/disgusted/neutral)
via TensorRT FP16 with landmark-geometry fallback; EMA per-person smoothing; opt-out registry
- Add emotion_node.py: subscribes /social/faces/detections, runs TRT crop inference (<5ms),
publishes /social/faces/expressions and /social/emotion/context JSON for LLM
- Wire emotion context into conversation_node.py: emotion hint injected into LLM prompt
when speaker shows non-neutral affect; subscribes /social/emotion/context
- Add emotion_params.yaml config and emotion.launch.py launch file
- Add 67-test suite (test_emotion_classifier.py): classifier, tracker, opt-out, heuristic
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:40:54 -05:00
3b2f219d66
feat( #169 ): emergency behavior system — obstacle stop, fall prevention, stuck detection, recovery FSM
...
Two new packages:
- saltybot_emergency_msgs: EmergencyEvent.msg, RecoveryAction.msg
- saltybot_emergency: threat_detector, alert_manager, recovery_sequencer, emergency_fsm, emergency_node
Implements:
- ObstacleDetector: MAJOR <30cm, CRITICAL <10cm; suppressed when not moving
- FallDetector: MINOR/MAJOR/CRITICAL tilt thresholds; floor-drop edge detection
- StuckDetector: MAJOR after 3s wheel stall (cmd>threshold, actual~0)
- BumpDetector: jerk = |Δ|a||/dt with gravity removal; MAJOR/CRITICAL thresholds
- AlertManager: per-(type,level) suppression; MAJOR×N within window → CRITICAL escalation
- RecoverySequencer: REVERSING→TURNING→RETRYING FSM; max_retries before GAVE_UP
- EmergencyFSM: NOMINAL→STOPPING→RECOVERING→ESCALATED; acknowledge to clear
- EmergencyNode: 20Hz ROS2 node; /saltybot/emergency, /saltybot/e_stop, cmd_vel mux
59/59 tests passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:39:37 -05:00
7dcdd2088a
feat(jetson): add saltybot_night_vision package (issue #168 )
...
Implements ambient-light-aware night vision mode for the D435i + IMX219
stack on the Jetson Orin Nano Super:
• light_analyser.py — histogram-based intensity FSM with hysteresis:
day → twilight → night → ir_only
• camera_controller.py — D435i IR emitter via realsense2_camera param
service + IMX219 gain/exposure via v4l2-ctl
• gpio_headlight.py — physical pin 33 headlight; Jetson.GPIO PWM
primary, sysfs on/off fallback, sim mode
• light_monitor_node.py — subscribes IMX219/IR1, publishes
/saltybot/ambient_light + /saltybot/vision_mode
• night_vision_controller_node.py — reacts to mode changes; drives
D435i emitter, IMX219 gain, headlight
• ir_slam_bridge_node.py — mono8 IR1 → rgb8 republish so RTAB-Map
keeps loop-closing in darkness
• launch/night_vision.launch.py + config/night_vision_params.yaml
• test/test_night_vision.py — 18 unit tests (18/18 pass, no ROS2 needed)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:36:21 -05:00
77b3d614dc
Merge pull request 'feat( #158 ): docking station auto-return — ArUco/IR detection, visual servo, charge monitoring' ( #165 ) from sl-controls/issue-158-docking into main
2026-03-02 10:26:17 -05:00
06f72521c9
Merge pull request 'feat(vo): visual odometry fallback — CUDA optical flow + EKF fusion + slip failover (Issue #157 )' ( #164 ) from sl-perception/issue-157-visual-odom into main
2026-03-02 10:26:09 -05:00
783dedf4d4
feat( #158 ): docking station auto-return with ArUco/IR detection and visual servo
...
Two new ROS2 packages implementing Issue #158 :
saltybot_docking_msgs (ament_cmake)
- DockingStatus.msg: stamp, state, dock_detected, distance_m, lateral_error_m,
battery_pct, charging, aligned
- Dock.srv / Undock.srv: force + resume_mission flags
saltybot_docking (ament_python, 20 Hz)
- dock_detector.py: ArucoDetector (cv2.aruco PnP → DockPose) + IRBeaconDetector
(EMA envelope with amplitude threshold); both gracefully degrade if unavailable
- visual_servo.py: IBVS proportional controller — v = k_lin×(d−target),
ω = −k_ang×yaw; aligned when |lateral| < 5mm AND d < contact_distance
- charge_monitor.py: edge-triggered events (CHARGING_STARTED/STOPPED,
THRESHOLD_LOW at 15%, THRESHOLD_HIGH at 80%)
- docking_state_machine.py: 7-state FSM (IDLE→DETECTING→NAV2_APPROACH→
VISUAL_SERVO→CONTACT→CHARGING→UNDOCKING); mission_resume signal on
auto-dock cycle; contact retry on timeout; lost-detection timeout
- docking_node.py: 20Hz ROS2 node; Nav2 NavigateToPose action client (optional);
/saltybot/dock + /saltybot/undock services; /saltybot/docking_cmd_vel;
/saltybot/resume_mission; /saltybot/docking_status
- config/docking_params.yaml, launch/docking.launch.py
Tests: 64/64 passing (IRBeaconDetector, VisualServo, ChargeMonitor,
DockingStateMachine — all state transitions and guard conditions covered)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:19:22 -05:00
572e578069
feat(vo): visual odometry fallback — CUDA optical flow + EKF fusion + slip failover (Issue #157 )
...
New package: saltybot_visual_odom (13 files, ~900 lines)
Nodes:
visual_odom_node — D435i IR1 stereo VO at 30 Hz
CUDA: SparsePyrLKOpticalFlow + FastFeatureDetector (GPU)
CPU fallback: calcOpticalFlowPyrLK + goodFeaturesToTrack
Essential matrix (5-pt RANSAC) + depth-aided metric scale
forward-backward consistency check on tracked points
Publishes /saltybot/visual_odom (Odometry)
odom_fusion_node — 5-state EKF [px, py, θ, v, ω] (unicycle model)
Fuses: wheel odom (/saltybot/rover_odom or tank_odom)
+ visual odom (/saltybot/visual_odom)
Slip failover: /saltybot/terrain JSON → 10× wheel noise on slip
Loop closure: /rtabmap/odom jump > 0.3m → EKF soft-correct
TF: publishes odom → base_link at 30 Hz
Publishes /saltybot/odom_fused + /saltybot/visual_odom_status
Modules:
optical_flow_tracker.py — CUDA/CPU sparse LK tracker with re-detection,
forward-backward consistency, ROI masking
stereo_vo.py — Essential matrix decomposition, camera→base_link
frame rotation, depth median scale recovery,
loop closure soft-correct, accumulated SE(3) pose
kalman_odom_filter.py — 5-state EKF: predict (unicycle), update_wheel,
update_visual, update_rtabmap (absolute pose);
Joseph-form covariance for numerical stability
Tests:
test/test_kalman_odom.py — 8 unit tests for EKF + StereoVO (no ROS deps)
Topic/TF map:
/camera/infra1/image_rect_raw → visual_odom_node
/camera/depth/image_rect_raw → visual_odom_node
/saltybot/visual_odom ← visual_odom_node (30 Hz)
/saltybot/rover_odom → odom_fusion_node
/saltybot/terrain → odom_fusion_node (slip signal)
/rtabmap/odom → odom_fusion_node (loop closure)
/saltybot/odom_fused ← odom_fusion_node (30 Hz)
odom → base_link TF ← odom_fusion_node (30 Hz)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:15:32 -05:00
ce6d5ee249
feat(social): multi-camera gesture recognition — MediaPipe Hands + Pose (Issue #140 )
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 9s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Delivers Issue #140 (P2): real-time gesture detection from 4 CSI cameras via
MediaPipe Hands and Pose, publishing classified gestures on /social/gestures.
New files:
- saltybot_social_msgs/msg/Gesture.msg + GestureArray.msg — ROS2 message types
- saltybot_social/gesture_classifier.py — pure-Python geometric classifier
(stop_palm, thumbs_up/down, point, come_here, follow, wave, arms_up,
arms_spread, crouch); WaveDetector temporal sliding-window oscillation tracker
- saltybot_social/gesture_node.py — ROS2 node; round-robin multi-camera
_FrameBuffer, lazy MediaPipe init, person-ID correlation via PersonState
- saltybot_social/test/test_gesture_classifier.py — 70 unit tests, all passing
- saltybot_social/config/gesture_params.yaml — tuned defaults for Orin Nano
- saltybot_social/launch/gesture.launch.py — all params overridable at launch
Modified:
- saltybot_social_msgs/CMakeLists.txt — register Gesture + GestureArray msgs
- saltybot_social/setup.py — add gesture_node console_scripts entry point
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:10:54 -05:00
33007fb5ed
Merge pull request 'feat( #142 ): terrain adaptation — surface detection + dynamic speed/PID/bias' ( #154 ) from sl-controls/issue-142-terrain into main
2026-03-02 10:07:38 -05:00
fb3e9f1bf1
feat( #142 ): terrain adaptation — surface detection, speed/PID/bias adaptation
...
Two new ROS2 packages implementing Issue #142 :
saltybot_terrain_msgs (ament_cmake)
- Terrain.msg: stamp, surface_type, roughness, incline_rad, grip, slip_ratio, speed_scale
saltybot_terrain_adaptation (ament_python, 50 Hz)
- terrain_analyzer.py: IMU vibration RMS → roughness [0,1] + surface classification
(smooth/tile/carpet/grass/gravel) with hysteresis-protected switching
- incline_detector.py: quaternion pitch → first-order LPF → incline_rad; motor_bias()
- slip_detector.py: cmd_vel vs odom → slip_ratio with EMA smoothing + speed gate
- terrain_adapter.py: TerrainState, TerrainAdapter (speed_scale, pid_scales, motor_bias),
TerrainHistory (rolling deque, JSON type-change log)
- terrain_adaptation_node.py: 50 Hz node; IMU watchdog; publishes terrain JSON + typed msg,
speed_scale, pid_scales (JSON), motor_bias, terrain_history (on type change)
- config/terrain_params.yaml, launch/terrain_adaptation.launch.py
Tests: 60/60 passing (TerrainAnalyzer, SlipDetector, InclineDetector, TerrainAdapter,
TerrainHistory all covered)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:04:36 -05:00
a9717cd602
feat(scene): semantic scene understanding — YOLOv8n TRT + room classification + hazards (Issue #141 )
...
New packages:
saltybot_scene_msgs — 4 msgs (SceneObject, SceneObjectArray, RoomClassification, BehaviorHint)
saltybot_scene — 3 nodes + launch + config + TRT build script
Nodes:
scene_detector_node — YOLOv8-nano TRT FP16 (target ≥15 FPS @ 640×640);
synchronized RGB+depth input; filters scene classes
(chairs, tables, doors, stairs, pets, appliances);
3D back-projection via aligned depth; depth-based hazard
scan (HazardClassifier); room classification at 2Hz;
publishes /social/scene/objects + /social/scene/hazards
+ /social/scene/room_type
behavior_adapter_node — adapts speed_limit_mps + personality_mode from room
type and hazard severity; publishes BehaviorHint on
/social/scene/behavior_hint (on-change + 1Hz heartbeat)
costmap_publisher_node — converts SceneObjectArray → PointCloud2 disc rings
for Nav2 obstacle_layer + MarkerArray for RViz;
publishes /social/scene/obstacle_cloud
Modules:
yolo_utils.py — YOLOv8 preprocess/postprocess (letterbox, cx/cy/w/h decode,
NMS), COCO+custom class table (door=80, stairs=81, wet=82),
hazard-by-class mapping
room_classifier.py — rule-based (object co-occurrence weights + softmax) with
optional MobileNetV2 TRT/ONNX backend (Places365-style 8-class)
hazard_classifier.py — depth-only hazard patterns: drop (row-mean cliff), stairs
(alternating depth bands), wet floor (depth std-dev), glass
(zero depth + strong Sobel edges in RGB)
scripts/build_scene_trt.py — export YOLOv8n → ONNX → TRT FP16; optionally build
MobileNetV2 room classifier engine; includes benchmark
Topic map:
/social/scene/objects SceneObjectArray ~15+ FPS
/social/scene/room_type RoomClassification ~2 Hz
/social/scene/hazards SceneObjectArray on hazard
/social/scene/behavior_hint BehaviorHint on-change + 1 Hz
/social/scene/obstacle_cloud PointCloud2 Nav2 obstacle_layer
/social/scene/object_markers MarkerArray RViz debug
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:59:53 -05:00
732045a086
Merge pull request 'feat(controls): adaptive PID balance controller with gain scheduling (Issue #136 )' ( #149 ) from sl-controls/issue-136-adaptive-pid into main
2026-03-02 09:51:19 -05:00
4cda5d0f19
Merge pull request 'feat(social): voice command NLU — 30+ intents with confirmation flow (Issue #137 )' ( #148 ) from sl-jetson/issue-137-voice-commands into main
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 47s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
2026-03-02 09:51:12 -05:00
4d2b008c48
feat(controls): adaptive PID balance controller with gain scheduling (Issue #136 )
...
Pure modules (no ROS2 dep, fully unit-tested):
- pid_controller.py:
GainSet — (kp,ki,kd) with safety clamp helper
PIDController — anti-windup integral, D-on-error, output clamping
GainScheduler — 3-class weight table (empty/light/heavy), exponential
gain blending (alpha per tick), safety bounds clamping, manual
override, immediate revert-to-defaults on instability
InstabilityDetector — dual criteria: tilt threshold (>50% of window)
+ sign-reversal count (oscillation)
- weight_estimator.py:
WeightEstimator — rolling-window current→weight, steady-state gating
(|tilt|≤threshold), change detection (threshold_kg)
CalibrationRoutine — IDLE→ROCKING→SETTLING→DONE/FAILED state machine;
sinusoidal rocking output, settling current sampling, weight estimate
from avg current; abort() / restart supported
- adaptive_pid_node.py: 100 Hz ROS2 node
Sub: /saltybot/imu (Imu, pitch from quaternion), /saltybot/motor_current
Pub: /saltybot/balance_effort (Float32), /saltybot/weight_estimate,
/saltybot/pid_state (JSON: gains, class, weight_kg, flags)
Srv: /saltybot/calibrate_balance (std_srvs/Trigger)
IMU watchdog (0.5 s), dynamic reconfigure via override_enabled param,
instability → revert + PID reset, structured INFO/WARN logging
- config/adaptive_pid_params.yaml, launch/adaptive_pid.launch.py,
package.xml, setup.py, setup.cfg
- test/test_adaptive_pid.py: 68/68 unit tests passing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:38:46 -05:00
fabb988e72
feat(social): voice command NLU — 30+ intents, confirmation flow (Issue #137 )
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 4s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 3s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
## New files
- saltybot_social_msgs/msg/VoiceCommand.msg
intent + entities[] + confidence + confirmation_token + requires_confirmation
- saltybot_social/voice_command_parser.py
Pure-Python regex NLU, zero ROS2/ML deps, < 1 ms/call
30+ named intents across nav.*, social.*, system.*, config.*, confirm.*
Entity extraction: location, name, mode, level, route name
Dangerous-command flag: system.shutdown, system.restart, social.forget_me
- saltybot_social/voice_command_node.py
Subscribes /social/speech/transcript, publishes /social/voice_command
Confirmation flow with UUID token + 10 s timeout
Below-threshold → intent=fallback → LLM conversation engine
- saltybot_social/test/test_voice_command_parser.py
191 unit tests (all pass), no ROS2 runtime required
- saltybot_social/config/voice_command_params.yaml
- saltybot_social/launch/voice_command.launch.py
## Intent taxonomy
nav: go_to, go_home, follow_me, stop, wait, come_here, patrol,
set_mode (shadow/lead/side/orbit/loose/tight),
teach_route, stop_teaching, replay_route
social: remember_me, forget_me [CONFIRM], whats_my_name, tell_joke
system: battery_status, map_status, shutdown [CONFIRM], restart [CONFIRM],
volume_up, volume_down, volume_set
config: personality, sass_level, follow_mode
## Updated
- saltybot_social_msgs/CMakeLists.txt: register VoiceCommand.msg
- saltybot_social/setup.py: add voice_command_node entry point
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:38:17 -05:00
b9b866854f
feat(fleet): multi-robot SLAM — map sharing + cooperative exploration (Issue #134 )
...
New packages:
saltybot_fleet_msgs — 4 msgs (RobotInfo, MapChunk, Frontier, FrontierArray)
+ 2 srvs (RegisterRobot, RequestFrontier)
saltybot_fleet — 4 nodes + 2 launch files + config + CLI tool
Nodes:
map_broadcaster_node — zlib-compress local OccupancyGrid → /fleet/maps @ 1Hz
+ /fleet/robots heartbeat with battery/status
fleet_manager_node — robot registry, MapMerger (multi-grid SE2-aligned merge),
frontier aggregation, /fleet/request_frontier service,
heartbeat timeout + stale frontier re-assignment
frontier_detector_node — scipy label-based frontier detection on merged map
→ /fleet/exploration_frontiers_raw
fleet_explorer_node — Nav2 NavigateToPose cooperative exploration state machine:
IDLE→request→NAVIGATING→ARRIVED→IDLE + STALLED backoff
Supporting modules:
map_compressor.py — binary serialise + zlib OccupancyGrid encode/decode
map_merger.py — SE(2)-transform-aware multi-grid merge with conservative
obstacle inflation (occupied beats free on conflict)
frontier_detector.py — numpy frontier mask + scipy connected-components + scoring
landmark_aligner.py — ArUco-landmark SE(2) estimation (Horn 2D least-squares)
to align robot map frames into common fleet_map frame
Topic layout:
/fleet/maps MapChunk per-robot compressed grids
/fleet/robots RobotInfo heartbeats + status
/fleet/merged_map OccupancyGrid coordinator merged output
/fleet/exploration_frontiers FrontierArray consolidated frontiers
/fleet/status String (JSON) coordinator health
/<robot_ns>/rtabmap/map input per robot
/<robot_ns>/rtabmap/odom input per robot
/<robot_ns>/navigate_to_pose Nav2 action per robot
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:36:17 -05:00
ea26cda76a
feat(bridge): battery management node (Issue #125 )
...
Add battery_node.py:
- Subscribes /saltybot/telemetry/battery (JSON from stm32_cmd_node)
- Publishes sensor_msgs/BatteryState on /saltybot/battery at 1 Hz
- SoC source priority: STM32 fuel gauge soc_pct field → fallback to
3S LiPo voltage curve (12-point lookup with linear interpolation)
- Charging detection: current_ma < -100 mA threshold
- Alert levels: WARNING (20%)→speed 60%, CRITICAL (10%)→speed 30%,
EMERGENCY (5%)→zero /cmd_vel + /saltybot/arm(False) disarm
- /saltybot/battery/alert JSON topic on threshold crossings
- /saltybot/speed_limit Float32 (0.0-1.0) for nav speed capping
- SQLite history logging: /var/log/saltybot/battery.db, 7-day retention
- Hourly prune timer to keep DB bounded
Add test_battery.py (70+ tests, no ROS2 runtime):
- SoC lookup: all curve points, interpolation, clamping, 3S/4S packs
- Alert level thresholds and transitions for all levels
- Speed factor assignments per alert level
- Charging detection logic
- sensor_msgs/BatteryState field population
- SQLite insert/retrieve/prune (in-memory and on-disk)
- JSON telemetry parsing: normal, charging, soc_pct=0 fallback
Add battery_params.yaml, battery.launch.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:27:06 -05:00
2e41f05c69
Merge pull request 'feat(mapping): RTAB-Map persistence + multi-session mapping (Issue #123 )' ( #129 ) from sl-perception/issue-123-map-persistence into main
2026-03-02 09:26:29 -05:00
5cfacdda3f
Merge pull request 'feat(bridge): binary STM32 command protocol (Issue #119 )' ( #128 ) from sl-jetson/issue-119-cmd-protocol into main
2026-03-02 09:26:20 -05:00
9341e9d986
feat(mapping): RTAB-Map persistence + multi-session + map management (Issue #123 )
...
- Add saltybot_mapping package: MapDatabase, MapExporter, MapManagerNode
- 6 ROS2 services: list/save_as/load/delete maps + export occupancy/pointcloud
- Auto-save current.db every 5 min; keep last 5 autosaves; warn at 2 GB
- Update rtabmap_params.yaml: database_path, Mem/InitWMWithAllNodes=true,
Rtabmap/StartNewMapOnLoopClosure=false (multi-session persistence by default)
- Update slam_rtabmap.launch.py: remove --delete_db_on_start, add fresh_start
arg (deletes DB before launch) and database_path arg (load named map)
- CLI tools: backup_map.py, export_map.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:17:54 -05:00
972bc8cb27
chore(bridge): update setup.py for stm32_cmd_node and battery_node entries
...
Add stm32_cmd_node and battery_node console_scripts, new launch/config files
to data_files list.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:17:13 -05:00
30be0d8cd3
feat(bridge): binary STM32 command protocol (Issue #119 )
...
Add stm32_protocol.py — pure-Python binary frame codec:
- Frame: STX(0x02)+TYPE+LEN+PAYLOAD+CRC16-CCITT(BE)+ETX(0x03)
- CRC covers TYPE+LEN+PAYLOAD; polynomial 0x1021, init 0xFFFF
- Encoders: HEARTBEAT, SPEED_STEER(-1000..+1000 int16), ARM, SET_MODE, PID_UPDATE
- Telemetry decoders: ImuFrame, BatteryFrame, MotorRpmFrame, ArmStateFrame, ErrorFrame
- FrameParser: streaming byte-by-byte state machine, resync on corrupt data
Add stm32_cmd_node.py — ROS2 bidirectional bridge node:
- /cmd_vel → SPEED_STEER at up to 50 Hz
- HEARTBEAT timer (default 200ms); STM32 watchdog fires at 500ms
- Jetson-side watchdog: no /cmd_vel for 500ms → send SPEED_STEER(0,0)
- /saltybot/arm service (SetBool) → ARM frame
- /saltybot/pid_update topic → PID_UPDATE frame
- Telemetry RX: IMU→/saltybot/imu, BATTERY→/saltybot/telemetry/battery,
MOTOR_RPM→/saltybot/telemetry/motor_rpm, ARM_STATE→/saltybot/arm_state,
ERROR→/saltybot/error
- Auto-reconnect on USB disconnect (serial.SerialException caught)
- Zero-speed + disarm sent on node shutdown
- /diagnostics with serial health, frame counts, last cmd age
Add test_stm32_protocol.py (60+ tests):
- CRC16-CCITT correctness, known test vectors
- All encoder output structures and payload values
- FrameParser: all telemetry types, bad CRC, bad ETX, resync, streaming,
oversized payload, frame counters, reset
Add test_stm32_cmd_node.py (30+ tests):
- MockSerial: TX/RX byte-level testing without hardware
- Speed/steer clamping, scaling, frame structure
- Watchdog fires/doesn't fire relative to timeout
- CRC error counted, resync after garbage
Add stm32_cmd_params.yaml, stm32_cmd.launch.py.
Update package.xml (add std_srvs, geometry_msgs deps).
Update setup.py (add stm32_cmd_node entry point + new config/launch).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:17:04 -05:00
413810e6ba
feat(tank): SaltyTank tracked-vehicle ESC driver (Issue #122 )
...
- kinematics.py: tank_speeds with slip compensation (angular_cmd scaled
by 1/(1-slip_factor)), skid_steer_speeds (no slip), speed_to_pwm,
compute_track_speeds (2wd|4wd|tracked modes, ±max clip), expand_to_4ch,
odometry_from_track_speeds (angular scaled by (1-slip_factor) — inverse
of command path, consistent dead-reckoning across all slip values)
- tank_driver_node.py: 50 Hz ROS2 node; serial P<ch1>,<ch2>,<ch3>,<ch4>\n;
H\n heartbeat; dual stop paths (watchdog 0.3s + /saltybot/e_stop topic,
latching); runtime drive_mode + slip_factor param switch; dead-reckoning
odometry with slip compensation; publishes /saltybot/tank_pwm (JSON) +
/saltybot/tank_odom
- config/tank_params.yaml, launch/tank_driver.launch.py, package.xml,
setup.py, setup.cfg
- test/test_tank_kinematics.py: 71 unit tests, all passing (includes
parametric round-trip over slip ∈ {0.0, 0.1, 0.3, 0.5})
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:14:37 -05:00
3c438595e8
feat(rover): SaltyRover 4-wheel ESC motor driver (Issue #110 )
...
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
- kinematics.py: pure unicycle→differential/skid-steer kinematics,
speed_to_pwm (1000–2000µs), compute_wheel_speeds with ±max clip,
odometry_from_wheel_speeds inverse helper
- rover_driver_node.py: 50 Hz ROS2 node; serial P<ch1>,<ch2>,<ch3>,<ch4>\n
protocol; heartbeat H\n; deadman on /cmd_vel silence; runtime 2WD/4WD
variant switch via four_wheel param; dead-reckoning odometry;
publishes /saltybot/rover_pwm (JSON) + /saltybot/rover_odom
- config/rover_params.yaml, launch/rover_driver.launch.py, package.xml,
setup.py, setup.cfg
- test/test_rover_kinematics.py: 51 unit tests, all passing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:03:28 -05:00
ee8438fd04
feat(tests): social-bot integration test suite (Issue #108 )
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 3m58s
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 3m3s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Add saltybot_social_tests package with full pytest + launch_testing harness:
- test_launch.py: start social_test.launch.py, verify all nodes up within 30s
- test_topic_rates.py: measure topic Hz over 3s window vs. minimum SLAs
- test_services.py: /social/enroll, /social/nav/set_mode, person CRUD, mood query
- test_gpu_memory.py: total allocation < 6 GB, no leak over 30s
- test_latency.py: inject→transcript→LLM→TTS state-machine SLA profiling
- test_shutdown.py: no zombies, GPU memory released, audio device freed
- test_helpers.py: TopicRateChecker, NodeChecker, ServiceChecker, GpuMemoryChecker
- mock_sensors.py: camera/faces/fused/persons/uwb publishers at correct rates
- social_test.launch.py: CI-mode launch (no mic/speaker, mock sensors auto-started)
- conftest.py + pytest.ini: gpu_required / full_stack / stack_running markers
- docker/Dockerfile.ci + docker-compose.ci.yml: CPU-only CI container
- docker/entrypoint-ci.sh: launch stack + wait 10s + run pytest
- bags/record_social_test.sh + bags/README.md: rosbag recording for replay
- .gitea/workflows/social-tests-ci.yml: lint + core-tests + latency-gpu jobs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 08:50:22 -05:00
6a96c73b2d
feat(panoramic): 360° equirectangular stitching + RTSP stream (Issue #105 )
...
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 08:41:40 -05:00
b1abdccf03
Merge pull request 'feat(controls): Autonomous/RC mode switch with 500ms blend ramp (Issue #104 )' ( #114 ) from sl-controls/issue-104-mode-switch into main
2026-03-02 08:41:24 -05:00
9733f5f097
feat(controls): Autonomous/RC mode switch with 500ms blend ramp (Issue #104 )
...
New package: saltybot_mode_switch
mode_logic.py (pure, no ROS2 dep — 72/72 tests pass):
State machine: RC → RAMP_TO_AUTO → AUTO → RAMP_TO_RC → RC
• CH6 (axes[5] > 0.5) requests AUTO; CH6 low → RAMP_TO_RC
• Stick >10% in AUTO/RAMP_TO_AUTO/RAMP_TO_RC → instant RC (no ramp)
• Sticks neutral ≥ 2 s after override → override cleared → RAMP_TO_AUTO
• RC link lost (Joy silent > 0.5 s) → instant RC from any state
• SLAM fix lost → RAMP_TO_RC (graceful exit from AUTO)
• No AUTO entry without slam_ok AND rc_link_ok
blend_alpha: 0.0 (RC) → linear ramp over 500 ms → 1.0 (AUTO)
led_pattern: solid_yellow(RC) | blink_green_slow(ramp) |
solid_green(AUTO) | blink_orange_fast(slam lost) |
blink_red_fast(RC link lost)
mode_switch_node.py (ROS2, 20 Hz):
Sub: /rc/joy (Joy), /saltybot/balance_state (String),
/slam_toolbox/pose_with_covariance_stamped (PoseWithCovarianceStamped)
Pub: /saltybot/control_mode (String JSON: mode+blend_alpha+slam_ok+rc_link_ok+override_active)
/saltybot/led_pattern (String)
cmd_vel_mux_node.py (ROS2, 20 Hz):
Sub: /cmd_vel_auto (Twist from Nav2/follower), /saltybot/control_mode
Pub: /cmd_vel (Twist to bridge, scaled by blend_alpha)
Note: remap Nav2/follower output to /cmd_vel_auto in launch.
Tests: 72/72 pass (no ROS2 runtime required).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 08:38:49 -05:00
1f594538fd
feat(calibration): IMX219 intrinsic + extrinsic calibration workflow (Issue #106 )
2026-03-02 08:38:24 -05:00
5043578934
feat(social): speech pipeline + LLM conversation + TTS + orchestrator ( #81 #83 #85 #89 )
...
Issue #81 — Speech pipeline:
- speech_pipeline_node.py: OpenWakeWord "hey_salty" → Silero VAD → faster-whisper
STT (Orin GPU, <500ms wake-to-transcript) → ECAPA-TDNN speaker diarization
- speech_utils.py: pcm16↔float32, EnergyVad, UtteranceSegmenter (pre-roll, max-
duration), cosine speaker identification — all pure Python, no ROS2/GPU needed
- Publishes /social/speech/transcript (SpeechTranscript) + /social/speech/vad_state
Issue #83 — Conversation engine:
- conversation_node.py: llama-cpp-python GGUF (Phi-3-mini Q4_K_M, 20 GPU layers),
streaming token output, per-person sliding-window context (4K tokens), summary
compression, SOUL.md system prompt, group mode
- llm_context.py: PersonContext, ContextStore (JSON persistence), build_llama_prompt
(ChatML format), context compression via LLM summarization
- Publishes /social/conversation/response (ConversationResponse, partial + final)
Issue #85 — Streaming TTS:
- tts_node.py: Piper ONNX streaming synthesis, sentence-by-sentence first-chunk
streaming (<200ms to first audio), sounddevice USB speaker playback, volume control
- tts_utils.py: split_sentences, pcm16_to_wav_bytes, chunk_pcm, apply_volume, strip_ssml
Issue #89 — Pipeline orchestrator:
- orchestrator_node.py: IDLE→LISTENING→THINKING→SPEAKING state machine, GPU memory
watchdog (throttle at <2GB free), rolling latency stats (p50/p95 per stage),
VAD watchdog (alert if speech pipeline hangs), /social/orchestrator/state JSON pub
- social_bot.launch.py: brings up all 4 nodes with TimerAction delays
New messages: SpeechTranscript.msg, VadState.msg, ConversationResponse.msg
Config YAMLs: speech_params, conversation_params, tts_params, orchestrator_params
Tests: 58 tests (28 speech_utils + 30 llm_context/tts_utils), all passing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 08:23:19 -05:00
a9b2242a2c
feat(social): Orin dev environment — JetPack 6 + TRT conversion + systemd ( #88 )
...
- Dockerfile.social: social-bot container with faster-whisper, llama-cpp-python
(CUDA), piper-tts, insightface, pyannote.audio, OpenWakeWord, pyaudio
- scripts/convert_models.sh: TRT FP16 conversion for SCRFD-10GF, ArcFace-R100,
ECAPA-TDNN; CTranslate2 setup for Whisper; Piper voice download; benchmark suite
- config/asound.conf: ALSA USB mic (card1) + USB speaker (card2) config
- models/README.md: version-pinned model table, /models/ layout, perf targets
- systemd/: saltybot-social.service + saltybot.target + install_systemd.sh
- docker-compose.yml: saltybot-social service with GPU, audio device passthrough,
NVMe volume mounts for /models and /social_db
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 08:08:57 -05:00
fa0162fadc
feat(social): multi-modal tracking fusion — UWB+camera Kalman filter (Issue #92 )
...
New packages:
saltybot_social_msgs — FusedTarget.msg custom message
saltybot_social_tracking — 4-state Kalman fusion node
saltybot_social_tracking/tracking_fusion_node.py
Subscribes to /uwb/target (PoseStamped, ~10 Hz) and /person/target
(PoseStamped, ~30 Hz) and publishes /social/tracking/fused_target
(FusedTarget) at 20 Hz.
Source arbitration:
• "fused" — both UWB and camera are fresh; confidence-weighted blend
• "uwb" — UWB fresh, camera stale
• "camera" — camera fresh, UWB stale
• "predicted" — all sources stale; KF coasts for up to predict_timeout (3 s)
Kalman filter (kalman_tracker.py):
State [x, y, vx, vy] with discrete Wiener acceleration noise model
(process_noise=3.0 m/s²) sized for EUC speeds (20-30 km/h, ≈5.5-8.3 m/s).
Separate UWB (0.20 m) and camera (0.12 m) measurement noise.
Velocity estimate converges after ~3 s of 10 Hz UWB measurements.
Confidence model (source_arbiter.py):
Per-source confidence = quality × max(0, 1 - age/timeout).
Composite confidence accounts for KF positional uncertainty and
is capped at 0.4 during dead-reckoning ("predicted") mode.
Tests: 58/58 pass (no ROS2 runtime required).
Note: saltybot_social_msgs here adds FusedTarget.msg; PR #98
(Issue #84 ) adds PersonalityState.msg + QueryMood.srv to the same
package. The maintainer should squash-merge #98 first and rebase
this branch on top of it before merging to avoid the package.xml
conflict.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 23:59:10 -05:00
44771751e2
feat(social): personality system — SOUL.md persona, mood engine, relationship DB (Issue #84 )
...
New packages:
- saltybot_social_msgs: PersonalityState.msg + QueryMood.srv custom interfaces
- saltybot_social_personality: full personality node
Features:
- SOUL.md YAML/Markdown persona file: name, humor_level (0-10), sass_level (0-10),
base_mood, per-tier greeting templates, mood prefix strings
- Hot-reload: SoulWatcher polls SOUL.md every reload_interval seconds, applies
changes live without restarting the node
- Per-person relationship memory in SQLite: score, interaction_count,
first/last_seen, learned preferences (JSON), full interaction log
- Mood engine (pure functions): happy | curious | annoyed | playful
driven by relationship score, interaction count, recent event window (120s)
- Greeting personalisation: stranger | regular | favorite tiers
keyed on interaction count thresholds from SOUL.md
- Publishes /social/personality/state (PersonalityState) at publish_rate Hz
- /social/personality/query_mood (QueryMood) service for on-demand mood query
- Full ROS2 dynamic reconfigure: soul_file, db_path, reload_interval, publish_rate
- 52 unit tests, no ROS2 runtime required
ROS2 interfaces:
Sub: /social/person_detected (std_msgs/String JSON)
Pub: /social/personality/state (saltybot_social_msgs/PersonalityState)
Srv: /social/personality/query_mood (saltybot_social_msgs/QueryMood)
2026-03-01 23:56:05 -05:00
dc746ccedc
Merge pull request 'feat(social): face detection + recognition #80 ' ( #96 ) from sl-perception/social-face-detection into main
2026-03-01 23:55:18 -05:00
d6a6965af6
Merge pull request 'feat(social): person enrollment system #87 ' ( #95 ) from sl-perception/social-enrollment into main
2026-03-01 23:55:16 -05:00
5143e5bfac
feat(social): Issue #86 — physical expression + motor attention
...
ESP32-C3 NeoPixel sketch (esp32/social_expression/social_expression.ino):
- Adafruit NeoPixel + ArduinoJson, serial JSON protocol 115200 8N1
- Mood→colour: happy=green, curious=blue, annoyed=red, playful=rainbow
- Idle breathing animation (sine-modulated warm white)
- Auto-falls to idle after IDLE_TIMEOUT_MS (3 s) with no command
ROS2 saltybot_social_msgs (new package):
- Mood.msg — {mood, intensity}
- Person.msg — {track_id, bearing_rad, distance_m, confidence, is_speaking, source}
- PersonArray.msg — {persons[], active_id}
ROS2 saltybot_social (new package):
- expression_node: subscribes /social/mood → JSON serial to ESP32-C3
reconnects on port error; sends idle frame after idle_timeout_s
- attention_node: subscribes /social/persons → /cmd_vel rotation-only
proportional control with dead zone; prefers active speaker, falls
back to highest-confidence person; lost-target idle after 2 s
- launch/social.launch.py — combined launch
- config YAML for both nodes with documented parameters
- test/test_attention.py — 15 pytest-only unit tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 23:35:59 -05:00
5c4f18e46c
feat(social): person enrollment system — SQLite gallery + voice trigger (Issue #87 )
...
- saltybot_social_msgs: 6 msg + 5 srv definitions for social interaction
- saltybot_social_enrollment: enrollment_node + enrollment_cli
- PersonDB: thread-safe SQLite-backed gallery (embeddings, voice samples)
- Voice-triggered enrollment via "remember me my name is X" phrase
- CLI: enroll/list/delete/rename via ros2 run
- Services: /social/enroll, /social/persons/list|delete|update
- Gallery sync from /social/faces/embeddings topic
2026-03-01 23:32:26 -05:00
f61a03b3c5
feat(social): face detection + recognition (SCRFD + ArcFace TRT FP16, Issue #80 )
...
Add two new ROS2 packages for the social sprint:
saltybot_social_msgs (ament_cmake):
- FaceDetection, FaceDetectionArray, FaceEmbedding, FaceEmbeddingArray
- PersonState, PersonStateArray
- EnrollPerson, ListPersons, DeletePerson, UpdatePerson services
saltybot_social_face (ament_python):
- SCRFDDetector: SCRFD face detection with TRT FP16 + ONNX fallback
- 640x640 input, 3-stride anchor decoding, NMS
- ArcFaceRecognizer: 512-dim embedding extraction with gallery matching
- 5-point landmark alignment to 112x112, cosine similarity
- FaceGallery: thread-safe persistent gallery (npz + JSON sidecar)
- FaceRecognitionNode: ROS2 node subscribing /camera/color/image_raw,
publishing /social/faces/detections, /social/faces/embeddings
- Enrollment via /social/enroll service (N-sample face averaging)
- Launch file, config YAML, TRT engine builder script
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 23:31:48 -05:00
d9c983f666
Merge pull request 'feat(social): navigation & path planning #91 ' ( #97 ) from sl-perception/social-nav into main
2026-03-01 23:30:40 -05:00
