6dbbbb9adc
Merge pull request 'feat(jetson): depth confidence filter node (issue #190 )' ( #196 ) from sl-perception/issue-190-depth-filter into main
2026-03-02 11:02:53 -05:00
c26293d000
feat(jetson): depth confidence filter node (issue #190 )
...
Adds depth_confidence_filter_node.py to saltybot_bringup:
- Synchronises /camera/depth/image_rect_raw + /camera/depth/confidence
via ApproximateTimeSynchronizer (10ms slop)
- Zeros pixels where confidence uint8 < threshold * 255 (default 0.5)
- Republishes filtered float32 depth on /camera/depth/filtered
- Registered as depth_confidence_filter console_scripts entry point
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 11:02:15 -05:00
e3e4bd70a4
Merge pull request 'feat(social): multi-language support - Whisper LID + per-lang Piper TTS (Issue #167 )' ( #187 ) from sl-jetson/issue-167-multilang into main
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 8s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
2026-03-02 10:58:16 -05:00
90c8b427fc
feat(social): multi-language support — Whisper LID + per-lang Piper TTS (Issue #167 )
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 10s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
- Add SpeechTranscript.language (BCP-47), ConversationResponse.language fields
- speech_pipeline_node: whisper_language param (""=auto-detect via Whisper LID);
detected language published in every transcript
- conversation_node: track per-speaker language; inject "[Please respond in X.]"
hint for non-English speakers; propagate language to ConversationResponse.
_LANG_NAMES: 24 BCP-47 codes -> English names. Also adds Issue #161 emotion
context plumbing (co-located in same branch for clean merge)
- tts_node: voice_map_json param (JSON BCP-47->ONNX path); lazy voice loading
per language; playback queue now carries (text, lang) tuples for voice routing
- speech_params.yaml, tts_params.yaml: new language params with docs
- 47/47 tests pass (test_multilang.py)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:57:34 -05:00
077f26d9d6
Merge pull request 'feat(power): STOP-mode sleep/wake power manager — Issue #178 ' ( #186 ) from sl-firmware/issue-178-power-mgmt into main
2026-03-02 10:56:52 -05:00
f446e5766e
feat(power): STOP-mode sleep/wake power manager — Issue #178
...
Adds STM32F7 STOP-mode power management with <10ms wake latency:
- power_mgmt.c: state machine (ACTIVE→SLEEP_PENDING→SLEEPING→WAKING),
30s idle timeout (PM_IDLE_TIMEOUT_MS), 3s LED fade before STOP,
gate SPI3/I2S3+SPI2+USART6+UART5 on sleep (clock-only, state preserved),
EXTI1(PA1/CRSF)+EXTI7(PB7/JLink)+EXTI4(PC4/IMU) wake sources,
PLL restore after STOP (PLLM=8/N=216/P=2 → 216MHz), uwTick save/restore
- Peripheral gating: I2S3, SPI2(OSD), USART6, UART5 disabled during STOP;
SPI1(IMU), UART4(CRSF), USART1(JLink), I2C1 remain active as wake sources
- Sleep LED: triangle-wave pulse (2s period) on LED1 during SLEEP_PENDING,
software PWM in main loop (1-bit, pm_pwm_phase vs brightness)
- IWDG: fed just before WFI; <10ms wake << 50ms WATCHDOG_TIMEOUT_MS
- JLink: JLINK_CMD_SLEEP=0x09, JLINK_TLM_POWER=0x81 (11-byte power frame
at 1Hz: power_state, est_total_ma, est_audio_ma, est_osd_ma, idle_ms)
- main.c: power_mgmt_init(), activity() on CRSF/JLink/armed, tick() when
disarmed, sleep_req handler, LED PWM, JLINK_TLM_POWER telemetry
- config.h: PM_* constants, PM_CURRENT_*_MA estimates, PM_TLM_HZ
- test_power_mgmt.py: 72 tests passing (state machine, LED, gating,
current estimates, JLink protocol, wake latency, hardware constants)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:53:02 -05:00
728d1b0c0e
Merge pull request 'feat(webui): live camera viewer — multi-stream + detection overlays (Issue #177 )' ( #182 ) from sl-webui/issue-177-camera-viewer into main
2026-03-02 10:48:50 -05:00
57420807ca
feat(webui): live camera viewer — multi-stream + detection overlays (Issue #177 )
...
UI (src/hooks/useCamera.js, src/components/CameraViewer.jsx):
- 7 camera sources: front/left/rear/right CSI, D435i RGB/depth, panoramic
- Compressed image subscription via rosbridge (sensor_msgs/CompressedImage)
- Client-side 15fps gate (drops excess frames, reduces JS pressure)
- Per-camera FPS indicator with quality badge (FULL/GOOD/LOW/NO SIGNAL)
- Detection overlays: face boxes + names (/social/faces/detections),
gesture icons (/social/gestures), scene object labels + hazard colours
(/social/scene/objects); overlay mode selector (off/faces/gestures/objects/all)
- 360° panoramic equirect viewer with mouse/touch drag azimuth pan
- Picture-in-picture: up to 3 pinned cameras via ⊕ button
- One-click recording (MediaRecorder → MP4/WebM download)
- Snapshot to PNG with detection overlay composite + timestamp watermark
- Cameras tab added to TELEMETRY group in App.jsx
Jetson (rosbridge bringup):
- rosbridge_params.yaml: whitelist + /camera/depth/image_rect_raw/compressed,
/camera/panoramic/compressed, /social/faces/detections,
/social/gestures, /social/scene/objects
- rosbridge.launch.py: D435i colour republisher (JPEG 75%) +
depth republisher (compressedDepth/PNG16 preserving uint16 values)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:47:01 -05:00
9ca0e0844c
Merge pull request 'feat(social): facial expression recognition — TRT FP16 emotion CNN (Issue #161 )' ( #180 ) from sl-jetson/issue-161-emotion into main
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 10s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
2026-03-02 10:46:21 -05:00
54668536c1
Merge pull request 'feat(jetson): dynamic obstacle tracking — LIDAR motion detection, Kalman tracking, trajectory prediction, Nav2 costmap ( #176 )' ( #181 ) from sl-perception/issue-176-dynamic-obstacles into main
2026-03-02 10:45:22 -05:00
c4bf8c371f
Merge pull request 'feat( #169 ): emergency behavior system — obstacle stop, fall prevention, stuck detection, recovery FSM' ( #179 ) from sl-controls/issue-169-emergency into main
2026-03-02 10:44:49 -05:00
2f4540f1d3
feat(jetson): add dynamic obstacle tracking package (issue #176 )
...
Implements real-time moving obstacle detection, Kalman tracking, trajectory
prediction, and Nav2 costmap integration at 10 Hz / <50ms latency:
saltybot_dynamic_obs_msgs (ament_cmake):
• TrackedObject.msg — id, PoseWithCovariance, velocity, predicted_path,
predicted_times, speed, confidence, age, hits
• MovingObjectArray.msg — TrackedObject[], active_count, tentative_count,
detector_latency_ms
saltybot_dynamic_obstacles (ament_python):
• object_detector.py — LIDAR background subtraction (EMA occupancy grid),
foreground dilation + scipy connected-component
clustering → Detection list
• kalman_tracker.py — CV Kalman filter, state [px,py,vx,vy], Joseph-form
covariance update, predict_horizon() (non-mutating)
• tracker_manager.py — up to 20 tracks, Hungarian assignment
(scipy.optimize.linear_sum_assignment), TENTATIVE→
CONFIRMED lifecycle, miss-prune
• dynamic_obs_node.py — 10 Hz timer: detect→track→publish
/saltybot/moving_objects + MarkerArray viz
• costmap_layer_node.py — predicted paths → PointCloud2 inflation smear
→ /saltybot/dynamic_obs_cloud for Nav2 ObstacleLayer
• launch/dynamic_obstacles.launch.py + config/dynamic_obstacles_params.yaml
• test/test_dynamic_obstacles.py — 27 unit tests (27/27 pass, no ROS2 needed)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:44:32 -05:00
50971c0946
feat(social): facial expression recognition — TRT FP16 emotion CNN (Issue #161 )
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
- Add Expression.msg / ExpressionArray.msg ROS2 message definitions
- Add emotion_classifier.py: 7-class CNN (happy/sad/angry/surprised/fearful/disgusted/neutral)
via TensorRT FP16 with landmark-geometry fallback; EMA per-person smoothing; opt-out registry
- Add emotion_node.py: subscribes /social/faces/detections, runs TRT crop inference (<5ms),
publishes /social/faces/expressions and /social/emotion/context JSON for LLM
- Wire emotion context into conversation_node.py: emotion hint injected into LLM prompt
when speaker shows non-neutral affect; subscribes /social/emotion/context
- Add emotion_params.yaml config and emotion.launch.py launch file
- Add 67-test suite (test_emotion_classifier.py): classifier, tracker, opt-out, heuristic
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:40:54 -05:00
3b2f219d66
feat( #169 ): emergency behavior system — obstacle stop, fall prevention, stuck detection, recovery FSM
...
Two new packages:
- saltybot_emergency_msgs: EmergencyEvent.msg, RecoveryAction.msg
- saltybot_emergency: threat_detector, alert_manager, recovery_sequencer, emergency_fsm, emergency_node
Implements:
- ObstacleDetector: MAJOR <30cm, CRITICAL <10cm; suppressed when not moving
- FallDetector: MINOR/MAJOR/CRITICAL tilt thresholds; floor-drop edge detection
- StuckDetector: MAJOR after 3s wheel stall (cmd>threshold, actual~0)
- BumpDetector: jerk = |Δ|a||/dt with gravity removal; MAJOR/CRITICAL thresholds
- AlertManager: per-(type,level) suppression; MAJOR×N within window → CRITICAL escalation
- RecoverySequencer: REVERSING→TURNING→RETRYING FSM; max_retries before GAVE_UP
- EmergencyFSM: NOMINAL→STOPPING→RECOVERING→ESCALATED; acknowledge to clear
- EmergencyNode: 20Hz ROS2 node; /saltybot/emergency, /saltybot/e_stop, cmd_vel mux
59/59 tests passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:39:37 -05:00
7c62f9bf88
Merge pull request 'feat(mechanical): modular payload bay system (Issue #170 )' ( #174 ) from sl-mechanical/issue-170-payload-bay into main
2026-03-02 10:39:35 -05:00
796e343b78
feat(mechanical): modular payload bay system (Issue #170 )
...
Dovetail rail + tool-free swappable payload modules for all variants:
- payload_bay_rail.scad: 50×12 mm 60° dovetail rail (DXF for CNC Al bar),
spring ball detent (Ø6 mm, 50 mm pitch), continuous safety-lock groove
(M4 thumbscrew), 4-pin pogo connector housing (GND/5V/12V/UART),
lab/rover/tank deck adapter plates
- payload_bay_modules.scad: universal _module_base() (male tongue, detent
bore, 4× Ø4 mm target pads, lock bore) + 3 example modules: cargo tray
(200×100 mm, Velcro slots, bungee cord slots), camera boom (120 mm mast +
80 mm arm, 2020-rail-compatible head, 3-position tilt), cup holder
(Ø80 mm tapered, 8-slot flex grip). Includes copy-paste module template.
- payload_bay_BOM.md: hardware list, CNC spec (dovetail dimensions, surface
finish, connector pocket), load analysis (2 kg rated with Al rail + lock),
module developer guide with constraints table
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:38:07 -05:00
d2cada00a0
Merge pull request 'feat(jetson): night vision mode — IR emitter, headlight, ambient-light FSM, IR SLAM bridge ( #168 )' ( #175 ) from sl-perception/issue-168-night-vision into main
2026-03-02 10:36:55 -05:00
f2b24e29d8
Merge pull request 'feat(audio): I2S3 audio amplifier driver — Issue #143 ' ( #173 ) from sl-firmware/issue-143-audio-amp into main
2026-03-02 10:36:36 -05:00
7dcdd2088a
feat(jetson): add saltybot_night_vision package (issue #168 )
...
Implements ambient-light-aware night vision mode for the D435i + IMX219
stack on the Jetson Orin Nano Super:
• light_analyser.py — histogram-based intensity FSM with hysteresis:
day → twilight → night → ir_only
• camera_controller.py — D435i IR emitter via realsense2_camera param
service + IMX219 gain/exposure via v4l2-ctl
• gpio_headlight.py — physical pin 33 headlight; Jetson.GPIO PWM
primary, sysfs on/off fallback, sim mode
• light_monitor_node.py — subscribes IMX219/IR1, publishes
/saltybot/ambient_light + /saltybot/vision_mode
• night_vision_controller_node.py — reacts to mode changes; drives
D435i emitter, IMX219 gain, headlight
• ir_slam_bridge_node.py — mono8 IR1 → rgb8 republish so RTAB-Map
keeps loop-closing in darkness
• launch/night_vision.launch.py + config/night_vision_params.yaml
• test/test_night_vision.py — 18 unit tests (18/18 pass, no ROS2 needed)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:36:21 -05:00
c3ada4a156
feat(audio): I2S3 audio amplifier driver — Issue #143
...
Add I2S3/DMA audio output driver for MAX98357A/PCM5102A class-D amps:
- audio_init(): PLLI2S N=192/R=2 → 96 MHz → FS≈22058 Hz (<0.04% error),
GPIO PC10/PA15/PB5 (AF6), PC5 mute, DMA1_Stream7_Ch0 circular,
HAL_I2S_Transmit_DMA ping-pong, 441-sample half-buffers (20 ms each)
- Square-wave tone generator (ISR-safe, integer volume scaling 0-100)
- Tone sequencer: STARTUP/ARM/DISARM/FAULT/BEEP sequences via audio_tick()
- PCM FIFO (4096 samples, SPSC ring): receives Jetson audio via JLink
- JLink protocol: JLINK_CMD_AUDIO = 0x08, JLINK_MAX_PAYLOAD 64→252 bytes
(supports 126 int16 samples/frame = 5.7 ms @22050 Hz)
- main.c: audio_init(), STARTUP tone on boot, ARM/FAULT tones, audio_tick()
- config.h: AUDIO_BCLK/LRCK/DOUT/MUTE pin defines + PLLI2S constants
- test_audio.py: 45 tests, all passing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:34:35 -05:00
566cfc8811
Merge pull request 'fix: IWDG reset during gyro recal — refresh at i=0 not i=39 (P0 #42 )' ( #172 ) from sl-firmware/gyro-recal-button into main
2026-03-02 10:34:20 -05:00
e43c82d132
Merge pull request 'feat(webui): settings & configuration panel (Issue #160 )' ( #166 ) from sl-webui/issue-160-settings into main
2026-03-02 10:27:24 -05:00
da3ee19688
feat(webui): settings & configuration panel (Issue #160 )
...
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
- useSettings.js: PID parameter catalogue, step-response simulation,
ROS2 parameter apply via rcl_interfaces/srv/SetParameters, sensor
param management, firmware info extraction from /diagnostics,
diagnostics bundle export, JSON backup/restore, localStorage persist
- SettingsPanel.jsx: 6-view panel (PID, Sensors, Network, Firmware,
Diagnostics, Backup); StepResponseCanvas with stable/oscillating/
unstable colour-coding; GainSlider with range+number input; weight-
class tabs (empty/light/heavy); parameter validation badges
- App.jsx: CONFIG tab group (purple), settings tab render, FLEET_TABS
set to gate ConnectionBar and footer for fleet/missions/settings
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:26:42 -05:00
77b3d614dc
Merge pull request 'feat( #158 ): docking station auto-return — ArUco/IR detection, visual servo, charge monitoring' ( #165 ) from sl-controls/issue-158-docking into main
2026-03-02 10:26:17 -05:00
06f72521c9
Merge pull request 'feat(vo): visual odometry fallback — CUDA optical flow + EKF fusion + slip failover (Issue #157 )' ( #164 ) from sl-perception/issue-157-visual-odom into main
2026-03-02 10:26:09 -05:00
cc67e33003
Merge pull request 'feat(mechanical): universal charging dock station (Issue #159 )' ( #163 ) from sl-mechanical/issue-159-charging-dock into main
2026-03-02 10:26:05 -05:00
ecb95c738b
Merge pull request 'feat(social): multi-camera gesture recognition — MediaPipe Hands + Pose (Issue #140 )' ( #162 ) from sl-jetson/issue-140-gestures into main
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 13s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
2026-03-02 10:26:03 -05:00
783dedf4d4
feat( #158 ): docking station auto-return with ArUco/IR detection and visual servo
...
Two new ROS2 packages implementing Issue #158 :
saltybot_docking_msgs (ament_cmake)
- DockingStatus.msg: stamp, state, dock_detected, distance_m, lateral_error_m,
battery_pct, charging, aligned
- Dock.srv / Undock.srv: force + resume_mission flags
saltybot_docking (ament_python, 20 Hz)
- dock_detector.py: ArucoDetector (cv2.aruco PnP → DockPose) + IRBeaconDetector
(EMA envelope with amplitude threshold); both gracefully degrade if unavailable
- visual_servo.py: IBVS proportional controller — v = k_lin×(d−target),
ω = −k_ang×yaw; aligned when |lateral| < 5mm AND d < contact_distance
- charge_monitor.py: edge-triggered events (CHARGING_STARTED/STOPPED,
THRESHOLD_LOW at 15%, THRESHOLD_HIGH at 80%)
- docking_state_machine.py: 7-state FSM (IDLE→DETECTING→NAV2_APPROACH→
VISUAL_SERVO→CONTACT→CHARGING→UNDOCKING); mission_resume signal on
auto-dock cycle; contact retry on timeout; lost-detection timeout
- docking_node.py: 20Hz ROS2 node; Nav2 NavigateToPose action client (optional);
/saltybot/dock + /saltybot/undock services; /saltybot/docking_cmd_vel;
/saltybot/resume_mission; /saltybot/docking_status
- config/docking_params.yaml, launch/docking.launch.py
Tests: 64/64 passing (IRBeaconDetector, VisualServo, ChargeMonitor,
DockingStateMachine — all state transitions and guard conditions covered)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:19:22 -05:00
572e578069
feat(vo): visual odometry fallback — CUDA optical flow + EKF fusion + slip failover (Issue #157 )
...
New package: saltybot_visual_odom (13 files, ~900 lines)
Nodes:
visual_odom_node — D435i IR1 stereo VO at 30 Hz
CUDA: SparsePyrLKOpticalFlow + FastFeatureDetector (GPU)
CPU fallback: calcOpticalFlowPyrLK + goodFeaturesToTrack
Essential matrix (5-pt RANSAC) + depth-aided metric scale
forward-backward consistency check on tracked points
Publishes /saltybot/visual_odom (Odometry)
odom_fusion_node — 5-state EKF [px, py, θ, v, ω] (unicycle model)
Fuses: wheel odom (/saltybot/rover_odom or tank_odom)
+ visual odom (/saltybot/visual_odom)
Slip failover: /saltybot/terrain JSON → 10× wheel noise on slip
Loop closure: /rtabmap/odom jump > 0.3m → EKF soft-correct
TF: publishes odom → base_link at 30 Hz
Publishes /saltybot/odom_fused + /saltybot/visual_odom_status
Modules:
optical_flow_tracker.py — CUDA/CPU sparse LK tracker with re-detection,
forward-backward consistency, ROI masking
stereo_vo.py — Essential matrix decomposition, camera→base_link
frame rotation, depth median scale recovery,
loop closure soft-correct, accumulated SE(3) pose
kalman_odom_filter.py — 5-state EKF: predict (unicycle), update_wheel,
update_visual, update_rtabmap (absolute pose);
Joseph-form covariance for numerical stability
Tests:
test/test_kalman_odom.py — 8 unit tests for EKF + StereoVO (no ROS deps)
Topic/TF map:
/camera/infra1/image_rect_raw → visual_odom_node
/camera/depth/image_rect_raw → visual_odom_node
/saltybot/visual_odom ← visual_odom_node (30 Hz)
/saltybot/rover_odom → odom_fusion_node
/saltybot/terrain → odom_fusion_node (slip signal)
/rtabmap/odom → odom_fusion_node (loop closure)
/saltybot/odom_fused ← odom_fusion_node (30 Hz)
odom → base_link TF ← odom_fusion_node (30 Hz)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:15:32 -05:00
8222a0c42e
feat(mechanical): charging dock station (Issue #159 )
...
Universal 5 V/5 A charging dock for SaltyLab/Rover/Tank robots:
- charging_dock.scad: weighted base (ballast pockets, floor anchors),
back wall with 2× 5 A pogo pin housing + wiring channel, V-guide
funnel rails (±20 mm alignment tolerance), ArUco marker mast (100×100 mm,
15° tilt), PSU bracket (IRM-30-5), 4-LED status bezel
(Searching/Aligned/Charging/Full)
- charging_dock_receiver.scad: 3-variant robot-side contact plate with
Ø12 mm brass pad press-fit, V-nose self-alignment; SaltyLab stem
collar, SaltyRover deck flange, SaltyTank skid-plate mount
- charging_dock_BOM.md: hardware list, ASCII wiring diagram, INA219
current-sense LED state logic, pogo height cross-variant shim table,
assembly sequence, export commands
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:15:14 -05:00
ce6d5ee249
feat(social): multi-camera gesture recognition — MediaPipe Hands + Pose (Issue #140 )
...
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 9s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 2s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Delivers Issue #140 (P2): real-time gesture detection from 4 CSI cameras via
MediaPipe Hands and Pose, publishing classified gestures on /social/gestures.
New files:
- saltybot_social_msgs/msg/Gesture.msg + GestureArray.msg — ROS2 message types
- saltybot_social/gesture_classifier.py — pure-Python geometric classifier
(stop_palm, thumbs_up/down, point, come_here, follow, wave, arms_up,
arms_spread, crouch); WaveDetector temporal sliding-window oscillation tracker
- saltybot_social/gesture_node.py — ROS2 node; round-robin multi-camera
_FrameBuffer, lazy MediaPipe init, person-ID correlation via PersonState
- saltybot_social/test/test_gesture_classifier.py — 70 unit tests, all passing
- saltybot_social/config/gesture_params.yaml — tuned defaults for Orin Nano
- saltybot_social/launch/gesture.launch.py — all params overridable at launch
Modified:
- saltybot_social_msgs/CMakeLists.txt — register Gesture + GestureArray msgs
- saltybot_social/setup.py — add gesture_node console_scripts entry point
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:10:54 -05:00
0f42e701e9
Merge pull request 'feat(firmware): OTA firmware update — USB DFU + dual-bank + CRC32 (Issue #124 )' ( #156 ) from sl-firmware/issue-124-ota into main
2026-03-02 10:08:40 -05:00
8facb80eab
Merge pull request 'feat(webui): mission planner — waypoint editor, routes, geofences, schedule (Issue #145 )' ( #155 ) from sl-webui/issue-145-mission-planner into main
2026-03-02 10:07:43 -05:00
33007fb5ed
Merge pull request 'feat( #142 ): terrain adaptation — surface detection + dynamic speed/PID/bias' ( #154 ) from sl-controls/issue-142-terrain into main
2026-03-02 10:07:38 -05:00
cb961edb9f
Merge pull request 'feat(scene): semantic scene understanding — YOLOv8n TRT + room classification + hazards (Issue #141 )' ( #153 ) from sl-perception/issue-141-scene-understanding into main
2026-03-02 10:07:29 -05:00
52250e28d6
Merge pull request 'feat(mechanical): IP54 weatherproofing kit (Issue #144 )' ( #152 ) from sl-mechanical/issue-144-weatherproofing into main
2026-03-02 10:07:27 -05:00
51dcc01bfa
feat(webui): mission planner — waypoint editor, routes, geofences, schedule (Issue #145 )
...
useMissions.js:
- Waypoints: click-to-place, drag, dwell time, localStorage persistence
- Routes: loop/oneshot/pingpong, per-robot assignment, waypoint sequence
- Geofences: polygon draw (no-go / allowed zones)
- Templates: save/load profiles + JSON export/import
- Schedules: time+day triggers with client-side runner
- Executions: running/paused/aborted state per route
MissionPlanner.jsx:
- Canvas: waypoints, route lines + direction arrows, geofence fills,
robot position overlays, scale bar, zoom/pan, execution progress
- 7 sub-views: Map | Waypoints | Routes | Geofences | Templates | Schedule | Execute
- Execute: start/pause/resume/abort, waypoint-by-waypoint advance,
sends /goal_pose (single) and /outdoor/waypoints (patrol PoseArray)
- Integrates useFleet for robot selection and /goal_pose publishing
App.jsx: adds Missions tab to FLEET group
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:04:38 -05:00
fb3e9f1bf1
feat( #142 ): terrain adaptation — surface detection, speed/PID/bias adaptation
...
Two new ROS2 packages implementing Issue #142 :
saltybot_terrain_msgs (ament_cmake)
- Terrain.msg: stamp, surface_type, roughness, incline_rad, grip, slip_ratio, speed_scale
saltybot_terrain_adaptation (ament_python, 50 Hz)
- terrain_analyzer.py: IMU vibration RMS → roughness [0,1] + surface classification
(smooth/tile/carpet/grass/gravel) with hysteresis-protected switching
- incline_detector.py: quaternion pitch → first-order LPF → incline_rad; motor_bias()
- slip_detector.py: cmd_vel vs odom → slip_ratio with EMA smoothing + speed gate
- terrain_adapter.py: TerrainState, TerrainAdapter (speed_scale, pid_scales, motor_bias),
TerrainHistory (rolling deque, JSON type-change log)
- terrain_adaptation_node.py: 50 Hz node; IMU watchdog; publishes terrain JSON + typed msg,
speed_scale, pid_scales (JSON), motor_bias, terrain_history (on type change)
- config/terrain_params.yaml, launch/terrain_adaptation.launch.py
Tests: 60/60 passing (TerrainAnalyzer, SlipDetector, InclineDetector, TerrainAdapter,
TerrainHistory all covered)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 10:04:36 -05:00
a9717cd602
feat(scene): semantic scene understanding — YOLOv8n TRT + room classification + hazards (Issue #141 )
...
New packages:
saltybot_scene_msgs — 4 msgs (SceneObject, SceneObjectArray, RoomClassification, BehaviorHint)
saltybot_scene — 3 nodes + launch + config + TRT build script
Nodes:
scene_detector_node — YOLOv8-nano TRT FP16 (target ≥15 FPS @ 640×640);
synchronized RGB+depth input; filters scene classes
(chairs, tables, doors, stairs, pets, appliances);
3D back-projection via aligned depth; depth-based hazard
scan (HazardClassifier); room classification at 2Hz;
publishes /social/scene/objects + /social/scene/hazards
+ /social/scene/room_type
behavior_adapter_node — adapts speed_limit_mps + personality_mode from room
type and hazard severity; publishes BehaviorHint on
/social/scene/behavior_hint (on-change + 1Hz heartbeat)
costmap_publisher_node — converts SceneObjectArray → PointCloud2 disc rings
for Nav2 obstacle_layer + MarkerArray for RViz;
publishes /social/scene/obstacle_cloud
Modules:
yolo_utils.py — YOLOv8 preprocess/postprocess (letterbox, cx/cy/w/h decode,
NMS), COCO+custom class table (door=80, stairs=81, wet=82),
hazard-by-class mapping
room_classifier.py — rule-based (object co-occurrence weights + softmax) with
optional MobileNetV2 TRT/ONNX backend (Places365-style 8-class)
hazard_classifier.py — depth-only hazard patterns: drop (row-mean cliff), stairs
(alternating depth bands), wet floor (depth std-dev), glass
(zero depth + strong Sobel edges in RGB)
scripts/build_scene_trt.py — export YOLOv8n → ONNX → TRT FP16; optionally build
MobileNetV2 room classifier engine; includes benchmark
Topic map:
/social/scene/objects SceneObjectArray ~15+ FPS
/social/scene/room_type RoomClassification ~2 Hz
/social/scene/hazards SceneObjectArray on hazard
/social/scene/behavior_hint BehaviorHint on-change + 1 Hz
/social/scene/obstacle_cloud PointCloud2 Nav2 obstacle_layer
/social/scene/object_markers MarkerArray RViz debug
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:59:53 -05:00
a70d9a2a71
feat(mechanical): IP54 weatherproofing kit (Issue #144 )
...
Add sealed enclosures and sensor housings for outdoor IP54 protection:
- ip54_enclosure.scad: main electronics box (Jetson/FC/ESC), O-ring lid,
fan+filter duct, PG7/PG9 cable glands, quarter-turn latches, heat sink
recesses; gasket DXF export
- ip54_sensor_housings.scad: IMX219 clear PC dome (O-ring + anti-fog
pocket), D435i IR-transparent window housing (PG7 rear cap), RPLIDAR
static clear PC dome base ring (120 mm OD, O-ring, quarter-turn clips)
- ip54_BOM.md: hardware list, thermal analysis (≤52°C at 40°C ambient),
IP54 compliance checklist, mass ~930g total kit
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:58:22 -05:00
4beef8da03
feat(firmware): OTA DFU entry via JLink command and Python flash script (Issue #124 )
...
- Add ota.h / ota.c: ota_enter_dfu() (armed guard, writes BKP15R, resets),
ota_fw_crc32() using STM32F7 hardware CRC peripheral (CRC-32/MPEG-2, 512 KB)
- Add JLINK_CMD_DFU_ENTER (0x06) and dfu_req flag to jlink.h / jlink.c
- Handle dfu_req in main loop: calls ota_enter_dfu(is_armed) — no-op if armed
- Update usbd_cdc_if.c: move DFU magic from BKP0R to BKP15R (OTA_DFU_BKP_IDX)
resolving BKP register conflict with BNO055 calibration (BKP0R–6R, PR #150 )
- Add scripts/flash_firmware.py: CRC-32/MPEG-2 + ISO-HDLC verification,
dfu-util flash, host-side backup/rollback, --trigger-dfu JLink serial path
- Add test/test_ota.py: 42 tests passing (CRC-32/MPEG-2, CRC-16/XMODEM,
DFU_ENTER frame structure, BKP register safety, flash constants)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:56:18 -05:00
07b4edaa2d
Merge pull request 'feat(mechanical): universal sensor mount rail system (Issue #138 )' ( #151 ) from sl-mechanical/issue-138-sensor-rail into main
2026-03-02 09:51:30 -05:00
f5d8180776
Merge pull request 'feat(firmware): BNO055 NDOF IMU driver on I2C1 (Issue #135 )' ( #150 ) from sl-firmware/issue-135-bno055 into main
2026-03-02 09:51:25 -05:00
732045a086
Merge pull request 'feat(controls): adaptive PID balance controller with gain scheduling (Issue #136 )' ( #149 ) from sl-controls/issue-136-adaptive-pid into main
2026-03-02 09:51:19 -05:00
4cda5d0f19
Merge pull request 'feat(social): voice command NLU — 30+ intents with confirmation flow (Issue #137 )' ( #148 ) from sl-jetson/issue-137-voice-commands into main
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 47s
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
2026-03-02 09:51:12 -05:00
e1acadc2aa
Merge pull request 'feat(webui): fleet management dashboard (Issue #139 )' ( #147 ) from sl-webui/issue-139-fleet-dashboard into main
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been cancelled
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
social-bot integration tests / Lint (flake8 + pep257) (push) Has been cancelled
2026-03-02 09:51:11 -05:00
3d9a47336a
Merge pull request 'feat(fleet): multi-robot SLAM — map sharing + cooperative exploration (Issue #134 )' ( #146 ) from sl-perception/issue-134-multi-robot-slam into main
2026-03-02 09:51:10 -05:00
3992b80bcb
feat(mechanical): universal sensor mount rail system (Issue #138 )
...
Add 2020 T-slot quick-swap sensor rail for SaltyLab/Rover/Tank variants:
- sensor_rail.scad: 2020 T-slot profile, T-nut, thumbscrew, end cap, index
pins, stem/post/tank clamp adapters
- sensor_rail_brackets.scad: universal T-nut base + RPLIDAR A1M8, D435i,
IMX219, UWB anchor, cable clip brackets (tool-free M3 thumbscrew retention)
- sensor_rail_BOM.md: purchased hardware, print settings, export commands
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:46:27 -05:00
ad2b599420
feat(firmware): BNO055 NDOF IMU driver on I2C1 (Issue #135 )
...
Auto-detected alongside MPU6000. Acts as balance primary when MPU6000
fails, or provides NDOF-fused yaw/pitch when both sensors are present.
- include/bno055.h: full API — bno055_init/read/is_ready/calib_status/
temperature/save_offsets/restore_offsets
- src/bno055.c: I2C1 driver; probes 0x28/0x29, resets via SYS_TRIGGER,
enters NDOF mode; 2-burst 12-byte reads (gyro+euler, LIA+gravity);
Euler/gyro/accel scaling (÷16, ÷16, ÷100); auto-saves offsets to
RTC backup regs BKP0R–BKP6R on first full cal; restores on boot
(bno055_is_ready() returns true immediately); temperature updated 1Hz
- include/config.h: BNO055_BKP_MAGIC = 0xB055CA10
- src/main.c: bno055_init() in I2C probe block (before IWDG); imu_calibrated()
macro dispatches mpu6000_is_calibrated() vs bno055_is_ready();
BNO055 read deferred inside balance gate to avoid stalling main loop;
USB JSON reports bno_cs (calib status) and bno_t (temperature)
- test/test_bno055_data.py: 43 pytest tests (43/43 pass) — calib status
bit extraction, Euler/gyro/accel scaling, burst parsing, offset
round-trip packing, temperature signed-byte encoding
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:40:44 -05:00
85e5777994
feat(webui): add FLEET tab group to App.jsx — Issue #139
...
Adds FleetPanel import and FLEET tab group (green) to the main
dashboard navigation. Fleet tab is self-contained via useFleet.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 09:40:27 -05:00
