seb/saltylab-firmware
1
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity
140 Commits 388 Branches 0 Tags
Commit Graph

140 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
sl-mechanical
d93919e26f feat: SaltyTank tracked chassis — drive sprockets, tensioners, skid plate (#121) 
Three new chassis design files for the SaltyTank continuous-track variant:

• saltytank_chassis.scad — Deck plate (500×360×8mm Al, DXF export), 2×
  side track frames (6mm Al, CNC/laser), idler tensioner sliding block,
  4× CSI corner camera mounts (45°/20°), D435i front bracket (8° tilt),
  stem collar (Ø25mm shared).  Drive sprocket mounts accept hoverboard hub
  motors with caliper-verified D-cut bore (16.11mm/13mm flat) + 52mm BC
  hub flange bolt pattern.  M6 tensioner bolt adjusts idler ±15mm for
  track tension. Shared FC 30.5×30.5mm + Jetson 58×49mm M3 patterns.
  Electronics bay footprint matches rover_electronics_bay.scad exactly.

• saltytank_skid_plate.scad — Sacrificial underside skid panel (360×500mm).
  4mm HDPE (DXF) or PETG print; countersunk M4 FHCS bolt-on.  4× drain/
  inspection slots; optional printed ribs (RIB_PRINT=true).  Ground
  clearance of hull between tracks: 90mm (exceeds 50mm requirement).

• saltytank_BOM.md — Full BOM: deck plate, side frames, drive sprockets,
  idler wheels + tensioners, road wheels (2/side), track belts (1109mm
  circumference calc), skid plate, sensor brackets, electronics bay
  (rover_electronics_bay.scad reused unchanged). Frame mass ≈ 2.98 kg
  (just under 3 kg target). Assembly sequence and track tensioning
  procedure included.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 09:20:41 -05:00
sl-jetson
7de55accc3 Merge pull request 'feat(rover): SaltyRover 4-wheel ESC motor driver (Issue #110)' (#117) from sl-controls/issue-110-rover-driver into main 2026-03-02 09:04:01 -05:00
sl-controls
3c438595e8 feat(rover): SaltyRover 4-wheel ESC motor driver (Issue #110)
Some checks failed
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 2s
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
Details
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Details 
- kinematics.py: pure unicycle→differential/skid-steer kinematics,
  speed_to_pwm (1000–2000µs), compute_wheel_speeds with ±max clip,
  odometry_from_wheel_speeds inverse helper
- rover_driver_node.py: 50 Hz ROS2 node; serial P<ch1>,<ch2>,<ch3>,<ch4>\n
  protocol; heartbeat H\n; deadman on /cmd_vel silence; runtime 2WD/4WD
  variant switch via four_wheel param; dead-reckoning odometry;
  publishes /saltybot/rover_pwm (JSON) + /saltybot/rover_odom
- config/rover_params.yaml, launch/rover_driver.launch.py, package.xml,
  setup.py, setup.cfg
- test/test_rover_kinematics.py: 51 unit tests, all passing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 09:03:28 -05:00
sl-jetson
f278f0fd06 Merge pull request 'feat(tests): social-bot integration test suite (Issue #108)' (#118) from sl-jetson/issue-108-integration-tests into main
Some checks failed
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 1m8s
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
Details
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
Details
2026-03-02 09:03:18 -05:00
sl-jetson
5bb1ec6d3e Merge pull request 'feat: SaltyRover chassis Rev 2 — 4-wheel rover with spring suspension (#109)' (#116) from sl-mechanical/issue-109-rover-chassis into main 2026-03-02 09:03:12 -05:00
sl-jetson
ee8438fd04 feat(tests): social-bot integration test suite (Issue #108)
Some checks failed
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 3m58s
Details
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 3m3s
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
Details
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
Details
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Details 
Add saltybot_social_tests package with full pytest + launch_testing harness:

- test_launch.py: start social_test.launch.py, verify all nodes up within 30s
- test_topic_rates.py: measure topic Hz over 3s window vs. minimum SLAs
- test_services.py: /social/enroll, /social/nav/set_mode, person CRUD, mood query
- test_gpu_memory.py: total allocation < 6 GB, no leak over 30s
- test_latency.py: inject→transcript→LLM→TTS state-machine SLA profiling
- test_shutdown.py: no zombies, GPU memory released, audio device freed
- test_helpers.py: TopicRateChecker, NodeChecker, ServiceChecker, GpuMemoryChecker
- mock_sensors.py: camera/faces/fused/persons/uwb publishers at correct rates
- social_test.launch.py: CI-mode launch (no mic/speaker, mock sensors auto-started)
- conftest.py + pytest.ini: gpu_required / full_stack / stack_running markers
- docker/Dockerfile.ci + docker-compose.ci.yml: CPU-only CI container
- docker/entrypoint-ci.sh: launch stack + wait 10s + run pytest
- bags/record_social_test.sh + bags/README.md: rosbag recording for replay
- .gitea/workflows/social-tests-ci.yml: lint + core-tests + latency-gpu jobs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:50:22 -05:00
sl-mechanical
2fa72e169e feat: SaltyRover chassis Rev 2 — 4-wheel rover with spring suspension (#109) 
New chassis design files for the SaltyRover rough-terrain variant:

• saltyrover_chassis_r2.scad — Deck plate (500×480×6mm Al, laser-cut DXF),
  4× M3-adjustable pivot brackets, 4× CSI corner camera mounts (45° outward,
  20° down), D435i front bracket (8° tilt), stem collar. All RENDER modes
  for STL and DXF export included.

• rover_spring_arm.scad — Trailing-arm spring suspension (×4). Pivot on M8
  bolt; captured 14mm OD compression spring (50mm free, ~5 N/mm); open-end
  axle dropout slot with retainer cap. Provides 25mm bump + 15mm droop travel.
  Bearing-seat recess for caliper-verified 37.8mm collar OD.

• rover_electronics_bay.scad — PETG electronics bay (240×200×80mm internal).
  FC standoffs 30.5×30.5mm M3 and Jetson Orin 58×49mm M3 — shared SaltyLab
  swappable pattern. Ventilation slots all 4 walls + lid. Lid integrates
  100mm RPLIDAR A1M8 tower (58mm BC, matched to rplidar_mount.scad).
  Split-print halves for 220mm beds included.

• rover_chassis_r2_BOM.md — Full BOM, mass estimate (frame ~2.15kg; reduce
  to <2kg by setting DECK_T=5), assembly sequence, critical dimensions.

Sensor positions: RPLIDAR top-centre on bay lid, D435i front, 4× IMX219 at
deck corners. Shares 30.5mm FC + 58mm Jetson + Ø25mm stem patterns with
SaltyLab for swappable electronics.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:42:44 -05:00
sl-jetson
6a30b20aaa Merge pull request 'feat(panoramic): 360° equirectangular stitching + RTSP #105' (#115) from sl-perception/issue-105-equirect into main 2026-03-02 08:42:14 -05:00
sl-perception
6a96c73b2d feat(panoramic): 360° equirectangular stitching + RTSP stream (Issue #105) 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:41:40 -05:00
sl-jetson
b1abdccf03 Merge pull request 'feat(controls): Autonomous/RC mode switch with 500ms blend ramp (Issue #104)' (#114) from sl-controls/issue-104-mode-switch into main 2026-03-02 08:41:24 -05:00
sl-jetson
25f2bab24a Merge pull request 'feat(calibration): IMX219 intrinsic + extrinsic calibration workflow #106' (#113) from sl-perception/issue-106-calibration into main 2026-03-02 08:41:20 -05:00
sl-jetson
b23e8432a2 Merge pull request 'feat(ui): social-bot web dashboard (issue #107)' (#112) from sl-webui/issue-107-dashboard into main 2026-03-02 08:41:18 -05:00
sl-jetson
23668d1d98 Merge pull request 'feat(rc): CRSF/ELRS RC integration — telemetry uplink + channel fix (Issue #103)' (#111) from sl-firmware/issue-103-crsf-rc into main 2026-03-02 08:41:16 -05:00
sl-controls
9733f5f097 feat(controls): Autonomous/RC mode switch with 500ms blend ramp (Issue #104) 
New package: saltybot_mode_switch

mode_logic.py (pure, no ROS2 dep — 72/72 tests pass):
  State machine: RC → RAMP_TO_AUTO → AUTO → RAMP_TO_RC → RC
  • CH6 (axes[5] > 0.5) requests AUTO; CH6 low → RAMP_TO_RC
  • Stick >10% in AUTO/RAMP_TO_AUTO/RAMP_TO_RC → instant RC (no ramp)
  • Sticks neutral ≥ 2 s after override → override cleared → RAMP_TO_AUTO
  • RC link lost (Joy silent > 0.5 s) → instant RC from any state
  • SLAM fix lost → RAMP_TO_RC (graceful exit from AUTO)
  • No AUTO entry without slam_ok AND rc_link_ok
  blend_alpha: 0.0 (RC) → linear ramp over 500 ms → 1.0 (AUTO)
  led_pattern: solid_yellow(RC) | blink_green_slow(ramp) |
               solid_green(AUTO) | blink_orange_fast(slam lost) |
               blink_red_fast(RC link lost)

mode_switch_node.py (ROS2, 20 Hz):
  Sub: /rc/joy (Joy), /saltybot/balance_state (String),
       /slam_toolbox/pose_with_covariance_stamped (PoseWithCovarianceStamped)
  Pub: /saltybot/control_mode (String JSON: mode+blend_alpha+slam_ok+rc_link_ok+override_active)
       /saltybot/led_pattern (String)

cmd_vel_mux_node.py (ROS2, 20 Hz):
  Sub: /cmd_vel_auto (Twist from Nav2/follower), /saltybot/control_mode
  Pub: /cmd_vel (Twist to bridge, scaled by blend_alpha)
  Note: remap Nav2/follower output to /cmd_vel_auto in launch.

Tests: 72/72 pass (no ROS2 runtime required).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:38:49 -05:00
sl-perception
1f594538fd feat(calibration): IMX219 intrinsic + extrinsic calibration workflow (Issue #106) 2026-03-02 08:38:24 -05:00
sl-webui
1cd8ebeb32 feat(ui): add social-bot dashboard (issue #107) 
React + Vite + TailwindCSS dashboard served on port 8080.
Connects to ROS2 via rosbridge_server WebSocket (default ws://localhost:9090).

Panels:
- StatusPanel: pipeline state (idle/listening/thinking/speaking/throttled)
  with animated pulse indicator, GPU memory bar, per-stage latency stats
- FaceGallery: enrolled persons grid with enroll/delete via
  /social/enrollment/* services; live detection indicator
- ConversationLog: real-time transcript with human/bot bubbles,
  streaming partial support, auto-scroll
- PersonalityTuner: sass/humor/verbosity sliders (0–10) writing to
  personality_node via rcl_interfaces/srv/SetParameters; live
  PersonalityState display
- NavModeSelector: shadow/lead/side/orbit/loose/tight mode buttons
  publishing to /social/nav/mode; voice command reference table

Usage:
  cd ui/social-bot && npm install && npm run dev   # dev server port 8080
  npm run build && npm run preview                  # production preview

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:36:51 -05:00
sl-firmware
4a46fad002 feat(rc): CRSF/ELRS RC integration — telemetry uplink + channel fix (Issue #103) 
## Summary
- config.h: CH1[0]=steer, CH2[1]=throttle (was CH4/CH3); CRSF_FAILSAFE_MS→500ms
- include/battery.h + src/battery.c: ADC3 Vbat reading on PC1 (11:1 divider)
  battery_read_mv(), battery_estimate_pct() for 3S/4S auto-detection
- include/crsf.h + src/crsf.c: CRSF telemetry TX uplink
  crsf_send_battery() — type 0x08, voltage/current/SoC to ELRS TX module
  crsf_send_flight_mode() — type 0x21, "ARMED\0"/"DISARM\0" for handset OSD
- src/main.c: battery_init() after crsf_init(); 1Hz telemetry tick calls
  crsf_send_battery(vbat_mv, 0, soc_pct) + crsf_send_flight_mode(armed)
- test/test_crsf_frames.py: 28 pytest tests — CRC8-DVB-S2, battery frame
  layout/encoding, flight-mode frame, battery_estimate_pct SoC math

Existing (already complete from crsf-elrs branch):
  CRSF frame decoder UART4 420000 baud DMA circular + IDLE interrupt
  Mode manager: RC↔autonomous blend, CH6 3-pos switch, 500ms smooth transition
  Failsafe in main.c: disarm if crsf_state.last_rx_ms stale > CRSF_FAILSAFE_MS
  CH5 arm switch with ARMING_HOLD_MS interlock + edge detection
  RC override: mode_manager blends steer/speed per mode (CH6)

Closes #103

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:35:48 -05:00
sl-firmware
00c97bd902 feat(rc): CRSF/ELRS RC integration — telemetry uplink + channel fix (Issue #103) 
## Summary
- config.h: CH1[0]=steer, CH2[1]=throttle (was CH4/CH3); CRSF_FAILSAFE_MS→500ms
- include/battery.h + src/battery.c: ADC3 Vbat reading on PC1 (11:1 divider)
  battery_read_mv(), battery_estimate_pct() for 3S/4S auto-detection
- include/crsf.h + src/crsf.c: CRSF telemetry TX uplink
  crsf_send_battery() — type 0x08, voltage/current/SoC to ELRS TX module
  crsf_send_flight_mode() — type 0x21, "ARMED\0"/"DISARM\0" for handset OSD
- src/main.c: battery_init() after crsf_init(); 1Hz telemetry tick calls
  crsf_send_battery(vbat_mv, 0, soc_pct) + crsf_send_flight_mode(armed)
- test/test_crsf_frames.py: 28 pytest tests — CRC8-DVB-S2, battery frame
  layout/encoding, flight-mode frame, battery_estimate_pct SoC math

Existing (already complete from crsf-elrs branch):
  CRSF frame decoder UART4 420000 baud DMA circular + IDLE interrupt
  Mode manager: RC↔autonomous blend, CH6 3-pos switch, 500ms smooth transition
  Failsafe in main.c: disarm if crsf_state.last_rx_ms stale > CRSF_FAILSAFE_MS
  CH5 arm switch with ARMING_HOLD_MS interlock + edge detection
  RC override: mode_manager blends steer/speed per mode (CH6)

Closes #103

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:35:39 -05:00
sl-jetson
0f2ea7931b Merge pull request 'feat(social): speech + LLM + TTS + orchestrator (#81 #83 #85 #89)' (#102) from sl-jetson/social-speech-llm-tts into main 2026-03-02 08:24:23 -05:00
sl-jetson
5043578934 feat(social): speech pipeline + LLM conversation + TTS + orchestrator (#81 #83 #85 #89) 
Issue #81 — Speech pipeline:
- speech_pipeline_node.py: OpenWakeWord "hey_salty" → Silero VAD → faster-whisper
  STT (Orin GPU, <500ms wake-to-transcript) → ECAPA-TDNN speaker diarization
- speech_utils.py: pcm16↔float32, EnergyVad, UtteranceSegmenter (pre-roll, max-
  duration), cosine speaker identification — all pure Python, no ROS2/GPU needed
- Publishes /social/speech/transcript (SpeechTranscript) + /social/speech/vad_state

Issue #83 — Conversation engine:
- conversation_node.py: llama-cpp-python GGUF (Phi-3-mini Q4_K_M, 20 GPU layers),
  streaming token output, per-person sliding-window context (4K tokens), summary
  compression, SOUL.md system prompt, group mode
- llm_context.py: PersonContext, ContextStore (JSON persistence), build_llama_prompt
  (ChatML format), context compression via LLM summarization
- Publishes /social/conversation/response (ConversationResponse, partial + final)

Issue #85 — Streaming TTS:
- tts_node.py: Piper ONNX streaming synthesis, sentence-by-sentence first-chunk
  streaming (<200ms to first audio), sounddevice USB speaker playback, volume control
- tts_utils.py: split_sentences, pcm16_to_wav_bytes, chunk_pcm, apply_volume, strip_ssml

Issue #89 — Pipeline orchestrator:
- orchestrator_node.py: IDLE→LISTENING→THINKING→SPEAKING state machine, GPU memory
  watchdog (throttle at <2GB free), rolling latency stats (p50/p95 per stage),
  VAD watchdog (alert if speech pipeline hangs), /social/orchestrator/state JSON pub
- social_bot.launch.py: brings up all 4 nodes with TimerAction delays

New messages: SpeechTranscript.msg, VadState.msg, ConversationResponse.msg
Config YAMLs: speech_params, conversation_params, tts_params, orchestrator_params
Tests: 58 tests (28 speech_utils + 30 llm_context/tts_utils), all passing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:23:19 -05:00
sl-jetson
efc7f0f815 Merge pull request 'feat(social): Orin dev environment — JetPack 6 + TRT conversion + systemd (#88)' (#101) from sl-jetson/social-orin-dev into main 2026-03-02 08:23:06 -05:00
sl-jetson
a9b2242a2c feat(social): Orin dev environment — JetPack 6 + TRT conversion + systemd (#88) 
- Dockerfile.social: social-bot container with faster-whisper, llama-cpp-python
  (CUDA), piper-tts, insightface, pyannote.audio, OpenWakeWord, pyaudio
- scripts/convert_models.sh: TRT FP16 conversion for SCRFD-10GF, ArcFace-R100,
  ECAPA-TDNN; CTranslate2 setup for Whisper; Piper voice download; benchmark suite
- config/asound.conf: ALSA USB mic (card1) + USB speaker (card2) config
- models/README.md: version-pinned model table, /models/ layout, perf targets
- systemd/: saltybot-social.service + saltybot.target + install_systemd.sh
- docker-compose.yml: saltybot-social service with GPU, audio device passthrough,
  NVMe volume mounts for /models and /social_db

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:08:57 -05:00
sl-jetson
79d9a1daa1 Merge pull request 'feat(social): multi-modal tracking fusion — UWB+camera Kalman filter (Issue #92)' (#100) from sl-controls/tracking-fusion into main 2026-03-02 07:00:54 -05:00
sl-controls
fa0162fadc feat(social): multi-modal tracking fusion — UWB+camera Kalman filter (Issue #92) 
New packages:
  saltybot_social_msgs   — FusedTarget.msg custom message
  saltybot_social_tracking — 4-state Kalman fusion node

saltybot_social_tracking/tracking_fusion_node.py
  Subscribes to /uwb/target (PoseStamped, ~10 Hz) and /person/target
  (PoseStamped, ~30 Hz) and publishes /social/tracking/fused_target
  (FusedTarget) at 20 Hz.

  Source arbitration:
    • "fused"     — both UWB and camera are fresh; confidence-weighted blend
    • "uwb"       — UWB fresh, camera stale
    • "camera"    — camera fresh, UWB stale
    • "predicted" — all sources stale; KF coasts for up to predict_timeout (3 s)

  Kalman filter (kalman_tracker.py):
    State [x, y, vx, vy] with discrete Wiener acceleration noise model
    (process_noise=3.0 m/s²) sized for EUC speeds (20-30 km/h, ≈5.5-8.3 m/s).
    Separate UWB (0.20 m) and camera (0.12 m) measurement noise.
    Velocity estimate converges after ~3 s of 10 Hz UWB measurements.

  Confidence model (source_arbiter.py):
    Per-source confidence = quality × max(0, 1 - age/timeout).
    Composite confidence accounts for KF positional uncertainty and
    is capped at 0.4 during dead-reckoning ("predicted") mode.

Tests: 58/58 pass (no ROS2 runtime required).

Note: saltybot_social_msgs here adds FusedTarget.msg; PR #98
(Issue #84) adds PersonalityState.msg + QueryMood.srv to the same
package. The maintainer should squash-merge #98 first and rebase
this branch on top of it before merging to avoid the package.xml
conflict.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-01 23:59:10 -05:00
sl-jetson
d48edf4092 Merge pull request 'feat(social): personality system — SOUL.md persona, mood engine, relationship DB (Issue #84)' (#98) from sl-controls/social-personality into main 2026-03-01 23:58:43 -05:00
sl-controls
44771751e2 feat(social): personality system — SOUL.md persona, mood engine, relationship DB (Issue #84) 
New packages:
- saltybot_social_msgs: PersonalityState.msg + QueryMood.srv custom interfaces
- saltybot_social_personality: full personality node

Features:
- SOUL.md YAML/Markdown persona file: name, humor_level (0-10), sass_level (0-10),
  base_mood, per-tier greeting templates, mood prefix strings
- Hot-reload: SoulWatcher polls SOUL.md every reload_interval seconds, applies
  changes live without restarting the node
- Per-person relationship memory in SQLite: score, interaction_count,
  first/last_seen, learned preferences (JSON), full interaction log
- Mood engine (pure functions): happy | curious | annoyed | playful
  driven by relationship score, interaction count, recent event window (120s)
- Greeting personalisation: stranger | regular | favorite tiers
  keyed on interaction count thresholds from SOUL.md
- Publishes /social/personality/state (PersonalityState) at publish_rate Hz
- /social/personality/query_mood (QueryMood) service for on-demand mood query
- Full ROS2 dynamic reconfigure: soul_file, db_path, reload_interval, publish_rate
- 52 unit tests, no ROS2 runtime required

ROS2 interfaces:
  Sub: /social/person_detected  (std_msgs/String JSON)
  Pub: /social/personality/state (saltybot_social_msgs/PersonalityState)
  Srv: /social/personality/query_mood (saltybot_social_msgs/QueryMood)
 2026-03-01 23:56:05 -05:00
sl-jetson
dc746ccedc Merge pull request 'feat(social): face detection + recognition #80' (#96) from sl-perception/social-face-detection into main 2026-03-01 23:55:18 -05:00
sl-jetson
d6a6965af6 Merge pull request 'feat(social): person enrollment system #87' (#95) from sl-perception/social-enrollment into main 2026-03-01 23:55:16 -05:00
sl-jetson
35b940e1f5 Merge pull request 'feat(social): Issue #86 — physical expression + motor attention' (#94) from sl-firmware/social-expression into main 2026-03-01 23:55:14 -05:00
sl-firmware
5143e5bfac feat(social): Issue #86 — physical expression + motor attention 
ESP32-C3 NeoPixel sketch (esp32/social_expression/social_expression.ino):
  - Adafruit NeoPixel + ArduinoJson, serial JSON protocol 115200 8N1
  - Mood→colour: happy=green, curious=blue, annoyed=red, playful=rainbow
  - Idle breathing animation (sine-modulated warm white)
  - Auto-falls to idle after IDLE_TIMEOUT_MS (3 s) with no command

ROS2 saltybot_social_msgs (new package):
  - Mood.msg — {mood, intensity}
  - Person.msg — {track_id, bearing_rad, distance_m, confidence, is_speaking, source}
  - PersonArray.msg — {persons[], active_id}

ROS2 saltybot_social (new package):
  - expression_node: subscribes /social/mood → JSON serial to ESP32-C3
    reconnects on port error; sends idle frame after idle_timeout_s
  - attention_node: subscribes /social/persons → /cmd_vel rotation-only
    proportional control with dead zone; prefers active speaker, falls
    back to highest-confidence person; lost-target idle after 2 s
  - launch/social.launch.py — combined launch
  - config YAML for both nodes with documented parameters
  - test/test_attention.py — 15 pytest-only unit tests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-01 23:35:59 -05:00
sl-perception
5c4f18e46c feat(social): person enrollment system — SQLite gallery + voice trigger (Issue #87) 
- saltybot_social_msgs: 6 msg + 5 srv definitions for social interaction
- saltybot_social_enrollment: enrollment_node + enrollment_cli
- PersonDB: thread-safe SQLite-backed gallery (embeddings, voice samples)
- Voice-triggered enrollment via "remember me my name is X" phrase
- CLI: enroll/list/delete/rename via ros2 run
- Services: /social/enroll, /social/persons/list|delete|update
- Gallery sync from /social/faces/embeddings topic
 2026-03-01 23:32:26 -05:00
sl-perception
f61a03b3c5 feat(social): face detection + recognition (SCRFD + ArcFace TRT FP16, Issue #80) 
Add two new ROS2 packages for the social sprint:

saltybot_social_msgs (ament_cmake):
- FaceDetection, FaceDetectionArray, FaceEmbedding, FaceEmbeddingArray
- PersonState, PersonStateArray
- EnrollPerson, ListPersons, DeletePerson, UpdatePerson services

saltybot_social_face (ament_python):
- SCRFDDetector: SCRFD face detection with TRT FP16 + ONNX fallback
  - 640x640 input, 3-stride anchor decoding, NMS
- ArcFaceRecognizer: 512-dim embedding extraction with gallery matching
  - 5-point landmark alignment to 112x112, cosine similarity
- FaceGallery: thread-safe persistent gallery (npz + JSON sidecar)
- FaceRecognitionNode: ROS2 node subscribing /camera/color/image_raw,
  publishing /social/faces/detections, /social/faces/embeddings
- Enrollment via /social/enroll service (N-sample face averaging)
- Launch file, config YAML, TRT engine builder script

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-01 23:31:48 -05:00
sl-jetson
d9c983f666 Merge pull request 'feat(social): navigation & path planning #91' (#97) from sl-perception/social-nav into main 2026-03-01 23:30:40 -05:00
sl-jetson
54e9274405 Merge pull request 'feat(uwb): MaUWB ESP32-S3 DW3000 dual-anchor bearing driver (Issue #90)' (#99) from sl-firmware/uwb-integration into main 2026-03-01 23:30:12 -05:00
sl-jetson
b432492785 Merge pull request 'feat(social): multi-modal person state tracker #82' (#93) from sl-perception/social-person-state into main 2026-03-01 23:30:04 -05:00
sl-firmware
9a68dfdb2e feat(uwb): MaUWB ESP32-S3 DW3000 dual-anchor bearing driver (Issue #90) 
## Summary
- saltybot_uwb_msgs: add UwbBearing.msg, add tag_id to UwbRange.msg,
  register UwbBearing in CMakeLists.txt
- ranging_math.py: add bearing_from_pos(x, y) helper (atan2-based)
- uwb_driver_node.py: dual-rate architecture
    • 100 Hz /uwb/ranges  — raw TWR ranges with tag_id attribution
    • 10 Hz  /uwb/bearing — Kalman-fused bearing + range estimate
    • enrolled_tag_ids parameter for tag pairing filter
    • AT+RANGE_ADDR=<tag> pairing command on connect
- uwb_config.yaml: range_rate / bearing_rate / enrolled_tag_ids params
- uwb.launch.py: expose new params as launch arguments
- test_ranging_math.py: 7 new bearing_from_pos unit tests

Closes #90

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-01 23:25:08 -05:00
sl-perception
d872ea5e34 feat(social): navigation + follow modes + MiDaS depth + waypoints (Issue #91) 
- saltybot_social_msgs: full message/service definitions (standalone compilation)
- saltybot_social_nav: social navigation orchestrator
  - Follow modes: shadow/lead/side/orbit/loose/tight
  - Voice steering: mode switching + route commands via /social/speech/*
  - A* obstacle avoidance on Nav2/SLAM occupancy grid (8-directional, inflation)
  - MiDaS monocular depth for CSI cameras (TRT FP16 + ONNX fallback)
  - Waypoint teaching + replay with WaypointRoute persistence
  - High-speed EUC tracking (5.5 m/s = ~20 km/h)
  - Predictive position extrapolation (0.3s ahead at high speed)
- Launch: social_nav.launch.py (social_nav + midas_depth + waypoint_teacher)
- Config: social_nav_params.yaml
- Script: build_midas_trt_engine.py (ONNX -> TRT FP16)
 2026-03-01 23:15:00 -05:00
sl-perception
84790412d6 feat(social): multi-modal person state tracker (Issue #82) 2026-03-01 23:08:22 -05:00
Sebastien Vayrette
ac6fcb9a42 docs: add Leap Motion, fix ESC step inset 12mm 2026-03-01 17:49:17 -05:00
Sebastien Vayrette
ea18b9ad72 docs: add ReSpeaker 2-Mic + SIM7600A 4G to wiring diagram 2026-03-01 16:32:34 -05:00
Sebastien Vayrette
0c40a1c4f4 docs: add full wiring diagram (Orin ↔ FC ↔ ESC) 2026-03-01 14:51:56 -05:00
sl-jetson
fc8faa0dab Merge pull request 'feat(safety): remote e-stop over 4G MQTT (Issue #63)' (#69) from sl-firmware/remote-estop into main 2026-03-01 04:58:58 -05:00
sl-firmware
d41a9dfe10 feat(safety): remote e-stop over 4G MQTT (Issue #63) 
STM32 firmware:
- safety.h/c: EstopSource enum, safety_remote_estop/clear/get/active()
  CDC 'E'=ESTOP_REMOTE, 'F'=ESTOP_CELLULAR_TIMEOUT, 'Z'=clear latch
- usbd_cdc_if: cdc_estop_request/cdc_estop_clear_request volatile flags
- status: status_update() +remote_estop param; both LEDs fast-blink 200ms
- main.c: immediate motor cutoff highest-priority; arming gated by
  !safety_remote_estop_active(); motor estop auto-clear gated; telemetry
  'es' field 0-4; status_update() updated to 5 args

Safety: IMMEDIATE motor cutoff, latched until explicit Z + DISARMED,
cannot re-arm via MQTT alone (requires RC arm hold). IWDG-safe.

Jetson bridge:
- remote_estop_node.py: paho-mqtt + pyserial, cellular watchdog 5s
- estop_params.yaml, remote_estop.launch.py
- setup.py / package.xml: register node + paho-mqtt dep
- docker-compose.yml: remote-estop service
- test_remote_estop.py: kill/clear/watchdog/latency unit tests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-01 04:55:54 -05:00
sl-jetson
f7acf1554c Merge pull request 'feat: semantic sidewalk segmentation — TensorRT FP16 (#72)' (#78) from sl-jetson/sidewalk-segmentation into main 2026-03-01 01:20:50 -05:00
sl-jetson
e964d632bf feat: semantic sidewalk segmentation — TensorRT FP16 (#72) 
New packages
────────────
saltybot_segmentation (ament_python)
  • seg_utils.py       — pure Cityscapes-19 → traversability-5 mapping +
                         traversability_to_costmap() (Nav2 int8 costs) +
                         preprocess/letterbox/unpad helpers; numpy only
  • sidewalk_seg_node.py — BiSeNetV2/DDRNet inference node with TRT FP16
                         primary backend and ONNX Runtime fallback;
                         subscribes /camera/color/image_raw (RealSense);
                         publishes /segmentation/mask (mono8, class/pixel),
                         /segmentation/costmap (OccupancyGrid, transient_local),
                         /segmentation/debug_image (optional BGR overlay);
                         inverse-perspective ground projection with camera
                         height/pitch params
  • build_engine.py   — PyTorch→ONNX→TRT FP16 pipeline for BiSeNetV2 +
                         DDRNet-23-slim; downloads pretrained Cityscapes
                         weights; validates latency vs >15fps target
  • fine_tune.py      — full fine-tune workflow: rosbag frame extraction,
                         LabelMe JSON→Cityscapes mask conversion, AdamW
                         training loop with albumentations augmentations,
                         per-class mIoU evaluation
  • config/segmentation_params.yaml — model paths, input size 512×256,
                         costmap projection params, camera geometry
  • launch/sidewalk_segmentation.launch.py
  • docs/training_guide.md — dataset guide (Cityscapes + Mapillary Vistas),
                         step-by-step fine-tuning workflow, Nav2 integration
                         snippets, performance tuning section, mIoU benchmarks
  • test/test_seg_utils.py — 24 unit tests (class mapping + cost generation)

saltybot_segmentation_costmap (ament_cmake)
  • SegmentationCostmapLayer.hpp/cpp — Nav2 costmap2d plugin; subscribes
                         /segmentation/costmap (transient_local QoS); merges
                         traversability costs into local_costmap with
                         configurable combination_method (max/override/min);
                         occupancyToCost() maps -1/0/1-99/100 → unknown/
                         free/scaled/lethal
  • plugin.xml, CMakeLists.txt, package.xml

Traversability classes
  SIDEWALK (0) → cost 0   (free — preferred)
  GRASS    (1) → cost 50  (medium)
  ROAD     (2) → cost 90  (high — avoid but crossable)
  OBSTACLE (3) → cost 100 (lethal)
  UNKNOWN  (4) → cost -1  (Nav2 unknown)

Performance target: >15fps on Orin Nano Super at 512×256
  BiSeNetV2 FP16 TRT: ~50fps measured on similar Ampere hardware
  DDRNet-23s FP16 TRT: ~40fps

Tests: 24/24 pass (seg_utils — no GPU/ROS2 required)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-01 01:15:13 -05:00
seb
4be93669a1 Merge pull request 'feat: outdoor adaptive speed controller — walk/jog/ride profiles up to 8 m/s' (#76) from sl-controls/outdoor-speed into main 2026-03-01 01:11:10 -05:00
seb
d3168b9c07 Merge pull request 'feat: route recording + autonomous replay (#71)' (#75) from sl-perception/route-record-replay into main 2026-03-01 01:10:02 -05:00
sl-perception
5dcaa7bd49 feat: route recording + autonomous replay (#71) 
Implements Phase 3 ride-once-replay-forever route system.

saltybot_routes package:
- route_recorder_node: samples GPS+odom+heading at 1Hz during follow-me
  rides; 2m waypoint spacing; JSON-Lines .jsonl on NVMe /data/routes/;
  services start_recording/stop_recording/save/discard
- route_replayer_node: loads .jsonl, GPS->ENU flat-earth conversion,
  heading->quaternion, 3m subsampling for Nav2 navigate_through_poses;
  2m GPS tolerance (SIM7600X +-2.5m); pause/resume/stop services
- route_manager_node: list/info/delete services for saved routes
- route_system.launch.py: all three nodes with shared params
- route_params.yaml: waypoint_spacing_m=2.0, replay_spacing_m=3.0

GPS: /gps/fix from SIM7600X (PR #65)
UWB: /uwb/target from follow-me (PR #66)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-01 01:07:06 -05:00
sl-controls
118d2b3add feat: outdoor adaptive speed controller (saltybot_speed_controller) 
Adds saltybot_speed_controller ROS2 package — sits between person_follower
(/cmd_vel_raw) and cmd_vel_bridge (/cmd_vel), providing adaptive speed profiles
tuned for balance stability during outdoor follow-me up to 8 m/s (EUC ride mode).

Key features:
- walk/jog/ride profiles (1.5/3.0/8.0 m/s) selected via UWB target velocity
- Hysteresis-based switching (5 ticks up, 15 ticks down) prevents oscillation
- Trapezoidal accel/decel ramps per profile; ride accel 0.3 m/s² (balance-safe)
- Emergency decel (2.0 m/s²) triggered by sudden target stop or hard decel
- GPS runaway protection: if GPS > commanded×1.5 AND > 50% profile_max → brake
- 52/52 unit tests (no ROS2 runtime required)

Topics: /cmd_vel_raw → [speed_controller] → /cmd_vel, /speed_controller/profile
Launch: ros2 launch saltybot_speed_controller outdoor_speed.launch.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-01 01:06:50 -05:00
seb
dcc26e6937 Merge pull request 'feat: SIM7600X mount + LTE/GNSS antenna brackets' (#70) from sl-mechanical/cellular-mount into main 2026-03-01 01:00:50 -05:00