seb/saltylab-firmware
1
0
Fork 0
Code Issues Pull Requests 5 Actions Packages Projects Releases Wiki Activity
350 Commits 390 Branches 0 Tags
Commit Graph

350 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
sl-jetson
2e41f05c69 Merge pull request 'feat(mapping): RTAB-Map persistence + multi-session mapping (Issue #123)' (#129) from sl-perception/issue-123-map-persistence into main 2026-03-02 09:26:29 -05:00
sl-jetson
5cfacdda3f Merge pull request 'feat(bridge): binary STM32 command protocol (Issue #119)' (#128) from sl-jetson/issue-119-cmd-protocol into main 2026-03-02 09:26:20 -05:00
sl-jetson
89c7c5fa3e Merge pull request 'feat(tank): SaltyTank tracked-vehicle ESC driver (Issue #122)' (#127) from sl-controls/issue-122-tank-driver into main 2026-03-02 09:26:19 -05:00
sl-firmware
6f0ad8e92e feat(firmware): Jetson binary serial protocol on USART1 (Issue #120) 
New jlink module replaces ASCII-over-USB-CDC jetson_cmd with a dedicated
hardware UART binary protocol at 921600 baud for reliable Jetson comms.

- include/jlink.h: JLinkState struct, jlink_tlm_status_t (20-byte packed),
  command/telemetry IDs (0x01-0x07 cmd, 0x80 status), API declarations
- src/jlink.c: USART1 DMA2_Stream2_Channel4 circular RX (128 bytes),
  IDLE interrupt, CRC16-XModem (poly 0x1021) frame parser state machine,
  command dispatch (HEARTBEAT/DRIVE/ARM/DISARM/PID_SET/ESTOP),
  jlink_send_telemetry() blocking TX (≈0.28 ms per frame)
- include/config.h: JLINK_BAUD=921600, JLINK_HB_TIMEOUT_MS=1000,
  JLINK_TLM_HZ=50, FW_MAJOR/MINOR/PATCH version constants
- src/main.c: jlink_init(), jlink_process() in main loop, arm/disarm/
  estop/PID flag handling, 50 Hz STATUS telemetry TX, jlink takes
  priority over legacy jetson_cmd for speed/steer injection
- test/test_jlink_frames.py: 39 pytest tests (39/39 pass) — CRC16,
  frame building, parser state machine, drive/PID/status encoding

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 09:22:34 -05:00
sl-mechanical
d93919e26f feat: SaltyTank tracked chassis — drive sprockets, tensioners, skid plate (#121) 
Three new chassis design files for the SaltyTank continuous-track variant:

• saltytank_chassis.scad — Deck plate (500×360×8mm Al, DXF export), 2×
  side track frames (6mm Al, CNC/laser), idler tensioner sliding block,
  4× CSI corner camera mounts (45°/20°), D435i front bracket (8° tilt),
  stem collar (Ø25mm shared).  Drive sprocket mounts accept hoverboard hub
  motors with caliper-verified D-cut bore (16.11mm/13mm flat) + 52mm BC
  hub flange bolt pattern.  M6 tensioner bolt adjusts idler ±15mm for
  track tension. Shared FC 30.5×30.5mm + Jetson 58×49mm M3 patterns.
  Electronics bay footprint matches rover_electronics_bay.scad exactly.

• saltytank_skid_plate.scad — Sacrificial underside skid panel (360×500mm).
  4mm HDPE (DXF) or PETG print; countersunk M4 FHCS bolt-on.  4× drain/
  inspection slots; optional printed ribs (RIB_PRINT=true).  Ground
  clearance of hull between tracks: 90mm (exceeds 50mm requirement).

• saltytank_BOM.md — Full BOM: deck plate, side frames, drive sprockets,
  idler wheels + tensioners, road wheels (2/side), track belts (1109mm
  circumference calc), skid plate, sensor brackets, electronics bay
  (rover_electronics_bay.scad reused unchanged). Frame mass ≈ 2.98 kg
  (just under 3 kg target). Assembly sequence and track tensioning
  procedure included.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 09:20:41 -05:00
sl-webui
78374668bf feat(ui): telemetry dashboard panels (issue #126) 
Adds 6 telemetry tabs to the social-bot dashboard extending PR #112.

IMU Panel (/saltybot/imu, /saltybot/balance_state):
  - Canvas artificial horizon + compass tape
  - Three.js 3D robot orientation cube with quaternion SLERP
  - Angular velocity readouts, balance state detail

Battery Panel (/diagnostics):
  - Voltage, SoC, estimated current, runtime estimate
  - 120-point voltage history sparkline (2 min)
  - Reads battery_voltage_v, battery_soc_pct KeyValues from DiagnosticArray

Motor Panel (/saltybot/balance_state, /saltybot/rover_pwm):
  - Auto-detects balance vs rover mode
  - Bidirectional duty bars for left/right motors
  - Motor temp from /diagnostics, PID detail for balance bot

Map Viewer (/map, /odom, /outdoor/route):
  - OccupancyGrid canvas renderer (unknown/free/occupied colour-coded)
  - Robot position + heading arrow, Nav2/OSM path overlay (dashed)
  - Pan (mouse/touch) + zoom, 2 m scale bar

Control Mode (/saltybot/control_mode):
  - RC / RAMP_TO_AUTO / AUTO / RAMP_TO_RC state badge
  - Blend alpha progress bar
  - Safety flags: SLAM ok, RC link ok, stick override active
  - State machine diagram

System Health (/diagnostics):
  - CPU/GPU temperature gauges with colour-coded bars
  - RAM, GPU memory, disk resource bars
  - ROS2 node status list sorted by severity (ERROR → WARN → OK)

Also:
  - Three.js vendor chunk split (471 kB separate lazy chunk)
  - Updated App.jsx with grouped SOCIAL + TELEMETRY tab nav

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 09:18:39 -05:00
sl-perception
9341e9d986 feat(mapping): RTAB-Map persistence + multi-session + map management (Issue #123) 
- Add saltybot_mapping package: MapDatabase, MapExporter, MapManagerNode
- 6 ROS2 services: list/save_as/load/delete maps + export occupancy/pointcloud
- Auto-save current.db every 5 min; keep last 5 autosaves; warn at 2 GB
- Update rtabmap_params.yaml: database_path, Mem/InitWMWithAllNodes=true,
  Rtabmap/StartNewMapOnLoopClosure=false (multi-session persistence by default)
- Update slam_rtabmap.launch.py: remove --delete_db_on_start, add fresh_start
  arg (deletes DB before launch) and database_path arg (load named map)
- CLI tools: backup_map.py, export_map.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 09:17:54 -05:00
sl-jetson
972bc8cb27 chore(bridge): update setup.py for stm32_cmd_node and battery_node entries 
Add stm32_cmd_node and battery_node console_scripts, new launch/config files
to data_files list.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 09:17:13 -05:00
sl-jetson
30be0d8cd3 feat(bridge): binary STM32 command protocol (Issue #119) 
Add stm32_protocol.py — pure-Python binary frame codec:
- Frame: STX(0x02)+TYPE+LEN+PAYLOAD+CRC16-CCITT(BE)+ETX(0x03)
- CRC covers TYPE+LEN+PAYLOAD; polynomial 0x1021, init 0xFFFF
- Encoders: HEARTBEAT, SPEED_STEER(-1000..+1000 int16), ARM, SET_MODE, PID_UPDATE
- Telemetry decoders: ImuFrame, BatteryFrame, MotorRpmFrame, ArmStateFrame, ErrorFrame
- FrameParser: streaming byte-by-byte state machine, resync on corrupt data

Add stm32_cmd_node.py — ROS2 bidirectional bridge node:
- /cmd_vel → SPEED_STEER at up to 50 Hz
- HEARTBEAT timer (default 200ms); STM32 watchdog fires at 500ms
- Jetson-side watchdog: no /cmd_vel for 500ms → send SPEED_STEER(0,0)
- /saltybot/arm service (SetBool) → ARM frame
- /saltybot/pid_update topic → PID_UPDATE frame
- Telemetry RX: IMU→/saltybot/imu, BATTERY→/saltybot/telemetry/battery,
  MOTOR_RPM→/saltybot/telemetry/motor_rpm, ARM_STATE→/saltybot/arm_state,
  ERROR→/saltybot/error
- Auto-reconnect on USB disconnect (serial.SerialException caught)
- Zero-speed + disarm sent on node shutdown
- /diagnostics with serial health, frame counts, last cmd age

Add test_stm32_protocol.py (60+ tests):
- CRC16-CCITT correctness, known test vectors
- All encoder output structures and payload values
- FrameParser: all telemetry types, bad CRC, bad ETX, resync, streaming,
  oversized payload, frame counters, reset

Add test_stm32_cmd_node.py (30+ tests):
- MockSerial: TX/RX byte-level testing without hardware
- Speed/steer clamping, scaling, frame structure
- Watchdog fires/doesn't fire relative to timeout
- CRC error counted, resync after garbage

Add stm32_cmd_params.yaml, stm32_cmd.launch.py.
Update package.xml (add std_srvs, geometry_msgs deps).
Update setup.py (add stm32_cmd_node entry point + new config/launch).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 09:17:04 -05:00
sl-controls
413810e6ba feat(tank): SaltyTank tracked-vehicle ESC driver (Issue #122) 
- kinematics.py: tank_speeds with slip compensation (angular_cmd scaled
  by 1/(1-slip_factor)), skid_steer_speeds (no slip), speed_to_pwm,
  compute_track_speeds (2wd|4wd|tracked modes, ±max clip), expand_to_4ch,
  odometry_from_track_speeds (angular scaled by (1-slip_factor) — inverse
  of command path, consistent dead-reckoning across all slip values)
- tank_driver_node.py: 50 Hz ROS2 node; serial P<ch1>,<ch2>,<ch3>,<ch4>\n;
  H\n heartbeat; dual stop paths (watchdog 0.3s + /saltybot/e_stop topic,
  latching); runtime drive_mode + slip_factor param switch; dead-reckoning
  odometry with slip compensation; publishes /saltybot/tank_pwm (JSON) +
  /saltybot/tank_odom
- config/tank_params.yaml, launch/tank_driver.launch.py, package.xml,
  setup.py, setup.cfg
- test/test_tank_kinematics.py: 71 unit tests, all passing (includes
  parametric round-trip over slip ∈ {0.0, 0.1, 0.3, 0.5})

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 09:14:37 -05:00
sl-jetson
7de55accc3 Merge pull request 'feat(rover): SaltyRover 4-wheel ESC motor driver (Issue #110)' (#117) from sl-controls/issue-110-rover-driver into main 2026-03-02 09:04:01 -05:00
sl-controls
3c438595e8 feat(rover): SaltyRover 4-wheel ESC motor driver (Issue #110)
Some checks failed
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 2s
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
Details
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Details 
- kinematics.py: pure unicycle→differential/skid-steer kinematics,
  speed_to_pwm (1000–2000µs), compute_wheel_speeds with ±max clip,
  odometry_from_wheel_speeds inverse helper
- rover_driver_node.py: 50 Hz ROS2 node; serial P<ch1>,<ch2>,<ch3>,<ch4>\n
  protocol; heartbeat H\n; deadman on /cmd_vel silence; runtime 2WD/4WD
  variant switch via four_wheel param; dead-reckoning odometry;
  publishes /saltybot/rover_pwm (JSON) + /saltybot/rover_odom
- config/rover_params.yaml, launch/rover_driver.launch.py, package.xml,
  setup.py, setup.cfg
- test/test_rover_kinematics.py: 51 unit tests, all passing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 09:03:28 -05:00
sl-jetson
f278f0fd06 Merge pull request 'feat(tests): social-bot integration test suite (Issue #108)' (#118) from sl-jetson/issue-108-integration-tests into main
Some checks failed
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 1m8s
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
Details
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
Details
2026-03-02 09:03:18 -05:00
sl-jetson
5bb1ec6d3e Merge pull request 'feat: SaltyRover chassis Rev 2 — 4-wheel rover with spring suspension (#109)' (#116) from sl-mechanical/issue-109-rover-chassis into main 2026-03-02 09:03:12 -05:00
sl-jetson
ee8438fd04 feat(tests): social-bot integration test suite (Issue #108)
Some checks failed
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 3m58s
Details
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 3m3s
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
Details
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
Details
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Details 
Add saltybot_social_tests package with full pytest + launch_testing harness:

- test_launch.py: start social_test.launch.py, verify all nodes up within 30s
- test_topic_rates.py: measure topic Hz over 3s window vs. minimum SLAs
- test_services.py: /social/enroll, /social/nav/set_mode, person CRUD, mood query
- test_gpu_memory.py: total allocation < 6 GB, no leak over 30s
- test_latency.py: inject→transcript→LLM→TTS state-machine SLA profiling
- test_shutdown.py: no zombies, GPU memory released, audio device freed
- test_helpers.py: TopicRateChecker, NodeChecker, ServiceChecker, GpuMemoryChecker
- mock_sensors.py: camera/faces/fused/persons/uwb publishers at correct rates
- social_test.launch.py: CI-mode launch (no mic/speaker, mock sensors auto-started)
- conftest.py + pytest.ini: gpu_required / full_stack / stack_running markers
- docker/Dockerfile.ci + docker-compose.ci.yml: CPU-only CI container
- docker/entrypoint-ci.sh: launch stack + wait 10s + run pytest
- bags/record_social_test.sh + bags/README.md: rosbag recording for replay
- .gitea/workflows/social-tests-ci.yml: lint + core-tests + latency-gpu jobs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:50:22 -05:00
sl-mechanical
2fa72e169e feat: SaltyRover chassis Rev 2 — 4-wheel rover with spring suspension (#109) 
New chassis design files for the SaltyRover rough-terrain variant:

• saltyrover_chassis_r2.scad — Deck plate (500×480×6mm Al, laser-cut DXF),
  4× M3-adjustable pivot brackets, 4× CSI corner camera mounts (45° outward,
  20° down), D435i front bracket (8° tilt), stem collar. All RENDER modes
  for STL and DXF export included.

• rover_spring_arm.scad — Trailing-arm spring suspension (×4). Pivot on M8
  bolt; captured 14mm OD compression spring (50mm free, ~5 N/mm); open-end
  axle dropout slot with retainer cap. Provides 25mm bump + 15mm droop travel.
  Bearing-seat recess for caliper-verified 37.8mm collar OD.

• rover_electronics_bay.scad — PETG electronics bay (240×200×80mm internal).
  FC standoffs 30.5×30.5mm M3 and Jetson Orin 58×49mm M3 — shared SaltyLab
  swappable pattern. Ventilation slots all 4 walls + lid. Lid integrates
  100mm RPLIDAR A1M8 tower (58mm BC, matched to rplidar_mount.scad).
  Split-print halves for 220mm beds included.

• rover_chassis_r2_BOM.md — Full BOM, mass estimate (frame ~2.15kg; reduce
  to <2kg by setting DECK_T=5), assembly sequence, critical dimensions.

Sensor positions: RPLIDAR top-centre on bay lid, D435i front, 4× IMX219 at
deck corners. Shares 30.5mm FC + 58mm Jetson + Ø25mm stem patterns with
SaltyLab for swappable electronics.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:42:44 -05:00
sl-jetson
6a30b20aaa Merge pull request 'feat(panoramic): 360° equirectangular stitching + RTSP #105' (#115) from sl-perception/issue-105-equirect into main 2026-03-02 08:42:14 -05:00
sl-perception
6a96c73b2d feat(panoramic): 360° equirectangular stitching + RTSP stream (Issue #105) 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:41:40 -05:00
sl-jetson
b1abdccf03 Merge pull request 'feat(controls): Autonomous/RC mode switch with 500ms blend ramp (Issue #104)' (#114) from sl-controls/issue-104-mode-switch into main 2026-03-02 08:41:24 -05:00
sl-jetson
25f2bab24a Merge pull request 'feat(calibration): IMX219 intrinsic + extrinsic calibration workflow #106' (#113) from sl-perception/issue-106-calibration into main 2026-03-02 08:41:20 -05:00
sl-jetson
b23e8432a2 Merge pull request 'feat(ui): social-bot web dashboard (issue #107)' (#112) from sl-webui/issue-107-dashboard into main 2026-03-02 08:41:18 -05:00
sl-jetson
23668d1d98 Merge pull request 'feat(rc): CRSF/ELRS RC integration — telemetry uplink + channel fix (Issue #103)' (#111) from sl-firmware/issue-103-crsf-rc into main 2026-03-02 08:41:16 -05:00
sl-controls
9733f5f097 feat(controls): Autonomous/RC mode switch with 500ms blend ramp (Issue #104) 
New package: saltybot_mode_switch

mode_logic.py (pure, no ROS2 dep — 72/72 tests pass):
  State machine: RC → RAMP_TO_AUTO → AUTO → RAMP_TO_RC → RC
  • CH6 (axes[5] > 0.5) requests AUTO; CH6 low → RAMP_TO_RC
  • Stick >10% in AUTO/RAMP_TO_AUTO/RAMP_TO_RC → instant RC (no ramp)
  • Sticks neutral ≥ 2 s after override → override cleared → RAMP_TO_AUTO
  • RC link lost (Joy silent > 0.5 s) → instant RC from any state
  • SLAM fix lost → RAMP_TO_RC (graceful exit from AUTO)
  • No AUTO entry without slam_ok AND rc_link_ok
  blend_alpha: 0.0 (RC) → linear ramp over 500 ms → 1.0 (AUTO)
  led_pattern: solid_yellow(RC) | blink_green_slow(ramp) |
               solid_green(AUTO) | blink_orange_fast(slam lost) |
               blink_red_fast(RC link lost)

mode_switch_node.py (ROS2, 20 Hz):
  Sub: /rc/joy (Joy), /saltybot/balance_state (String),
       /slam_toolbox/pose_with_covariance_stamped (PoseWithCovarianceStamped)
  Pub: /saltybot/control_mode (String JSON: mode+blend_alpha+slam_ok+rc_link_ok+override_active)
       /saltybot/led_pattern (String)

cmd_vel_mux_node.py (ROS2, 20 Hz):
  Sub: /cmd_vel_auto (Twist from Nav2/follower), /saltybot/control_mode
  Pub: /cmd_vel (Twist to bridge, scaled by blend_alpha)
  Note: remap Nav2/follower output to /cmd_vel_auto in launch.

Tests: 72/72 pass (no ROS2 runtime required).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:38:49 -05:00
sl-perception
1f594538fd feat(calibration): IMX219 intrinsic + extrinsic calibration workflow (Issue #106) 2026-03-02 08:38:24 -05:00
sl-webui
1cd8ebeb32 feat(ui): add social-bot dashboard (issue #107) 
React + Vite + TailwindCSS dashboard served on port 8080.
Connects to ROS2 via rosbridge_server WebSocket (default ws://localhost:9090).

Panels:
- StatusPanel: pipeline state (idle/listening/thinking/speaking/throttled)
  with animated pulse indicator, GPU memory bar, per-stage latency stats
- FaceGallery: enrolled persons grid with enroll/delete via
  /social/enrollment/* services; live detection indicator
- ConversationLog: real-time transcript with human/bot bubbles,
  streaming partial support, auto-scroll
- PersonalityTuner: sass/humor/verbosity sliders (0–10) writing to
  personality_node via rcl_interfaces/srv/SetParameters; live
  PersonalityState display
- NavModeSelector: shadow/lead/side/orbit/loose/tight mode buttons
  publishing to /social/nav/mode; voice command reference table

Usage:
  cd ui/social-bot && npm install && npm run dev   # dev server port 8080
  npm run build && npm run preview                  # production preview

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:36:51 -05:00
sl-firmware
4a46fad002 feat(rc): CRSF/ELRS RC integration — telemetry uplink + channel fix (Issue #103) 
## Summary
- config.h: CH1[0]=steer, CH2[1]=throttle (was CH4/CH3); CRSF_FAILSAFE_MS→500ms
- include/battery.h + src/battery.c: ADC3 Vbat reading on PC1 (11:1 divider)
  battery_read_mv(), battery_estimate_pct() for 3S/4S auto-detection
- include/crsf.h + src/crsf.c: CRSF telemetry TX uplink
  crsf_send_battery() — type 0x08, voltage/current/SoC to ELRS TX module
  crsf_send_flight_mode() — type 0x21, "ARMED\0"/"DISARM\0" for handset OSD
- src/main.c: battery_init() after crsf_init(); 1Hz telemetry tick calls
  crsf_send_battery(vbat_mv, 0, soc_pct) + crsf_send_flight_mode(armed)
- test/test_crsf_frames.py: 28 pytest tests — CRC8-DVB-S2, battery frame
  layout/encoding, flight-mode frame, battery_estimate_pct SoC math

Existing (already complete from crsf-elrs branch):
  CRSF frame decoder UART4 420000 baud DMA circular + IDLE interrupt
  Mode manager: RC↔autonomous blend, CH6 3-pos switch, 500ms smooth transition
  Failsafe in main.c: disarm if crsf_state.last_rx_ms stale > CRSF_FAILSAFE_MS
  CH5 arm switch with ARMING_HOLD_MS interlock + edge detection
  RC override: mode_manager blends steer/speed per mode (CH6)

Closes #103

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:35:48 -05:00
sl-firmware
00c97bd902 feat(rc): CRSF/ELRS RC integration — telemetry uplink + channel fix (Issue #103) 
## Summary
- config.h: CH1[0]=steer, CH2[1]=throttle (was CH4/CH3); CRSF_FAILSAFE_MS→500ms
- include/battery.h + src/battery.c: ADC3 Vbat reading on PC1 (11:1 divider)
  battery_read_mv(), battery_estimate_pct() for 3S/4S auto-detection
- include/crsf.h + src/crsf.c: CRSF telemetry TX uplink
  crsf_send_battery() — type 0x08, voltage/current/SoC to ELRS TX module
  crsf_send_flight_mode() — type 0x21, "ARMED\0"/"DISARM\0" for handset OSD
- src/main.c: battery_init() after crsf_init(); 1Hz telemetry tick calls
  crsf_send_battery(vbat_mv, 0, soc_pct) + crsf_send_flight_mode(armed)
- test/test_crsf_frames.py: 28 pytest tests — CRC8-DVB-S2, battery frame
  layout/encoding, flight-mode frame, battery_estimate_pct SoC math

Existing (already complete from crsf-elrs branch):
  CRSF frame decoder UART4 420000 baud DMA circular + IDLE interrupt
  Mode manager: RC↔autonomous blend, CH6 3-pos switch, 500ms smooth transition
  Failsafe in main.c: disarm if crsf_state.last_rx_ms stale > CRSF_FAILSAFE_MS
  CH5 arm switch with ARMING_HOLD_MS interlock + edge detection
  RC override: mode_manager blends steer/speed per mode (CH6)

Closes #103

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:35:39 -05:00
sl-jetson
0f2ea7931b Merge pull request 'feat(social): speech + LLM + TTS + orchestrator (#81 #83 #85 #89)' (#102) from sl-jetson/social-speech-llm-tts into main 2026-03-02 08:24:23 -05:00
sl-jetson
5043578934 feat(social): speech pipeline + LLM conversation + TTS + orchestrator (#81 #83 #85 #89) 
Issue #81 — Speech pipeline:
- speech_pipeline_node.py: OpenWakeWord "hey_salty" → Silero VAD → faster-whisper
  STT (Orin GPU, <500ms wake-to-transcript) → ECAPA-TDNN speaker diarization
- speech_utils.py: pcm16↔float32, EnergyVad, UtteranceSegmenter (pre-roll, max-
  duration), cosine speaker identification — all pure Python, no ROS2/GPU needed
- Publishes /social/speech/transcript (SpeechTranscript) + /social/speech/vad_state

Issue #83 — Conversation engine:
- conversation_node.py: llama-cpp-python GGUF (Phi-3-mini Q4_K_M, 20 GPU layers),
  streaming token output, per-person sliding-window context (4K tokens), summary
  compression, SOUL.md system prompt, group mode
- llm_context.py: PersonContext, ContextStore (JSON persistence), build_llama_prompt
  (ChatML format), context compression via LLM summarization
- Publishes /social/conversation/response (ConversationResponse, partial + final)

Issue #85 — Streaming TTS:
- tts_node.py: Piper ONNX streaming synthesis, sentence-by-sentence first-chunk
  streaming (<200ms to first audio), sounddevice USB speaker playback, volume control
- tts_utils.py: split_sentences, pcm16_to_wav_bytes, chunk_pcm, apply_volume, strip_ssml

Issue #89 — Pipeline orchestrator:
- orchestrator_node.py: IDLE→LISTENING→THINKING→SPEAKING state machine, GPU memory
  watchdog (throttle at <2GB free), rolling latency stats (p50/p95 per stage),
  VAD watchdog (alert if speech pipeline hangs), /social/orchestrator/state JSON pub
- social_bot.launch.py: brings up all 4 nodes with TimerAction delays

New messages: SpeechTranscript.msg, VadState.msg, ConversationResponse.msg
Config YAMLs: speech_params, conversation_params, tts_params, orchestrator_params
Tests: 58 tests (28 speech_utils + 30 llm_context/tts_utils), all passing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:23:19 -05:00
sl-jetson
efc7f0f815 Merge pull request 'feat(social): Orin dev environment — JetPack 6 + TRT conversion + systemd (#88)' (#101) from sl-jetson/social-orin-dev into main 2026-03-02 08:23:06 -05:00
sl-jetson
a9b2242a2c feat(social): Orin dev environment — JetPack 6 + TRT conversion + systemd (#88) 
- Dockerfile.social: social-bot container with faster-whisper, llama-cpp-python
  (CUDA), piper-tts, insightface, pyannote.audio, OpenWakeWord, pyaudio
- scripts/convert_models.sh: TRT FP16 conversion for SCRFD-10GF, ArcFace-R100,
  ECAPA-TDNN; CTranslate2 setup for Whisper; Piper voice download; benchmark suite
- config/asound.conf: ALSA USB mic (card1) + USB speaker (card2) config
- models/README.md: version-pinned model table, /models/ layout, perf targets
- systemd/: saltybot-social.service + saltybot.target + install_systemd.sh
- docker-compose.yml: saltybot-social service with GPU, audio device passthrough,
  NVMe volume mounts for /models and /social_db

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-02 08:08:57 -05:00
sl-jetson
79d9a1daa1 Merge pull request 'feat(social): multi-modal tracking fusion — UWB+camera Kalman filter (Issue #92)' (#100) from sl-controls/tracking-fusion into main 2026-03-02 07:00:54 -05:00
sl-controls
fa0162fadc feat(social): multi-modal tracking fusion — UWB+camera Kalman filter (Issue #92) 
New packages:
  saltybot_social_msgs   — FusedTarget.msg custom message
  saltybot_social_tracking — 4-state Kalman fusion node

saltybot_social_tracking/tracking_fusion_node.py
  Subscribes to /uwb/target (PoseStamped, ~10 Hz) and /person/target
  (PoseStamped, ~30 Hz) and publishes /social/tracking/fused_target
  (FusedTarget) at 20 Hz.

  Source arbitration:
    • "fused"     — both UWB and camera are fresh; confidence-weighted blend
    • "uwb"       — UWB fresh, camera stale
    • "camera"    — camera fresh, UWB stale
    • "predicted" — all sources stale; KF coasts for up to predict_timeout (3 s)

  Kalman filter (kalman_tracker.py):
    State [x, y, vx, vy] with discrete Wiener acceleration noise model
    (process_noise=3.0 m/s²) sized for EUC speeds (20-30 km/h, ≈5.5-8.3 m/s).
    Separate UWB (0.20 m) and camera (0.12 m) measurement noise.
    Velocity estimate converges after ~3 s of 10 Hz UWB measurements.

  Confidence model (source_arbiter.py):
    Per-source confidence = quality × max(0, 1 - age/timeout).
    Composite confidence accounts for KF positional uncertainty and
    is capped at 0.4 during dead-reckoning ("predicted") mode.

Tests: 58/58 pass (no ROS2 runtime required).

Note: saltybot_social_msgs here adds FusedTarget.msg; PR #98
(Issue #84) adds PersonalityState.msg + QueryMood.srv to the same
package. The maintainer should squash-merge #98 first and rebase
this branch on top of it before merging to avoid the package.xml
conflict.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-01 23:59:10 -05:00
sl-jetson
d48edf4092 Merge pull request 'feat(social): personality system — SOUL.md persona, mood engine, relationship DB (Issue #84)' (#98) from sl-controls/social-personality into main 2026-03-01 23:58:43 -05:00
sl-controls
44771751e2 feat(social): personality system — SOUL.md persona, mood engine, relationship DB (Issue #84) 
New packages:
- saltybot_social_msgs: PersonalityState.msg + QueryMood.srv custom interfaces
- saltybot_social_personality: full personality node

Features:
- SOUL.md YAML/Markdown persona file: name, humor_level (0-10), sass_level (0-10),
  base_mood, per-tier greeting templates, mood prefix strings
- Hot-reload: SoulWatcher polls SOUL.md every reload_interval seconds, applies
  changes live without restarting the node
- Per-person relationship memory in SQLite: score, interaction_count,
  first/last_seen, learned preferences (JSON), full interaction log
- Mood engine (pure functions): happy | curious | annoyed | playful
  driven by relationship score, interaction count, recent event window (120s)
- Greeting personalisation: stranger | regular | favorite tiers
  keyed on interaction count thresholds from SOUL.md
- Publishes /social/personality/state (PersonalityState) at publish_rate Hz
- /social/personality/query_mood (QueryMood) service for on-demand mood query
- Full ROS2 dynamic reconfigure: soul_file, db_path, reload_interval, publish_rate
- 52 unit tests, no ROS2 runtime required

ROS2 interfaces:
  Sub: /social/person_detected  (std_msgs/String JSON)
  Pub: /social/personality/state (saltybot_social_msgs/PersonalityState)
  Srv: /social/personality/query_mood (saltybot_social_msgs/QueryMood)
 2026-03-01 23:56:05 -05:00
sl-jetson
dc746ccedc Merge pull request 'feat(social): face detection + recognition #80' (#96) from sl-perception/social-face-detection into main 2026-03-01 23:55:18 -05:00
sl-jetson
d6a6965af6 Merge pull request 'feat(social): person enrollment system #87' (#95) from sl-perception/social-enrollment into main 2026-03-01 23:55:16 -05:00
sl-jetson
35b940e1f5 Merge pull request 'feat(social): Issue #86 — physical expression + motor attention' (#94) from sl-firmware/social-expression into main 2026-03-01 23:55:14 -05:00
sl-firmware
5143e5bfac feat(social): Issue #86 — physical expression + motor attention 
ESP32-C3 NeoPixel sketch (esp32/social_expression/social_expression.ino):
  - Adafruit NeoPixel + ArduinoJson, serial JSON protocol 115200 8N1
  - Mood→colour: happy=green, curious=blue, annoyed=red, playful=rainbow
  - Idle breathing animation (sine-modulated warm white)
  - Auto-falls to idle after IDLE_TIMEOUT_MS (3 s) with no command

ROS2 saltybot_social_msgs (new package):
  - Mood.msg — {mood, intensity}
  - Person.msg — {track_id, bearing_rad, distance_m, confidence, is_speaking, source}
  - PersonArray.msg — {persons[], active_id}

ROS2 saltybot_social (new package):
  - expression_node: subscribes /social/mood → JSON serial to ESP32-C3
    reconnects on port error; sends idle frame after idle_timeout_s
  - attention_node: subscribes /social/persons → /cmd_vel rotation-only
    proportional control with dead zone; prefers active speaker, falls
    back to highest-confidence person; lost-target idle after 2 s
  - launch/social.launch.py — combined launch
  - config YAML for both nodes with documented parameters
  - test/test_attention.py — 15 pytest-only unit tests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-01 23:35:59 -05:00
sl-perception
5c4f18e46c feat(social): person enrollment system — SQLite gallery + voice trigger (Issue #87) 
- saltybot_social_msgs: 6 msg + 5 srv definitions for social interaction
- saltybot_social_enrollment: enrollment_node + enrollment_cli
- PersonDB: thread-safe SQLite-backed gallery (embeddings, voice samples)
- Voice-triggered enrollment via "remember me my name is X" phrase
- CLI: enroll/list/delete/rename via ros2 run
- Services: /social/enroll, /social/persons/list|delete|update
- Gallery sync from /social/faces/embeddings topic
 2026-03-01 23:32:26 -05:00
sl-perception
f61a03b3c5 feat(social): face detection + recognition (SCRFD + ArcFace TRT FP16, Issue #80) 
Add two new ROS2 packages for the social sprint:

saltybot_social_msgs (ament_cmake):
- FaceDetection, FaceDetectionArray, FaceEmbedding, FaceEmbeddingArray
- PersonState, PersonStateArray
- EnrollPerson, ListPersons, DeletePerson, UpdatePerson services

saltybot_social_face (ament_python):
- SCRFDDetector: SCRFD face detection with TRT FP16 + ONNX fallback
  - 640x640 input, 3-stride anchor decoding, NMS
- ArcFaceRecognizer: 512-dim embedding extraction with gallery matching
  - 5-point landmark alignment to 112x112, cosine similarity
- FaceGallery: thread-safe persistent gallery (npz + JSON sidecar)
- FaceRecognitionNode: ROS2 node subscribing /camera/color/image_raw,
  publishing /social/faces/detections, /social/faces/embeddings
- Enrollment via /social/enroll service (N-sample face averaging)
- Launch file, config YAML, TRT engine builder script

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-01 23:31:48 -05:00
sl-jetson
d9c983f666 Merge pull request 'feat(social): navigation & path planning #91' (#97) from sl-perception/social-nav into main 2026-03-01 23:30:40 -05:00
sl-jetson
54e9274405 Merge pull request 'feat(uwb): MaUWB ESP32-S3 DW3000 dual-anchor bearing driver (Issue #90)' (#99) from sl-firmware/uwb-integration into main 2026-03-01 23:30:12 -05:00
sl-jetson
b432492785 Merge pull request 'feat(social): multi-modal person state tracker #82' (#93) from sl-perception/social-person-state into main 2026-03-01 23:30:04 -05:00
sl-firmware
9a68dfdb2e feat(uwb): MaUWB ESP32-S3 DW3000 dual-anchor bearing driver (Issue #90) 
## Summary
- saltybot_uwb_msgs: add UwbBearing.msg, add tag_id to UwbRange.msg,
  register UwbBearing in CMakeLists.txt
- ranging_math.py: add bearing_from_pos(x, y) helper (atan2-based)
- uwb_driver_node.py: dual-rate architecture
    • 100 Hz /uwb/ranges  — raw TWR ranges with tag_id attribution
    • 10 Hz  /uwb/bearing — Kalman-fused bearing + range estimate
    • enrolled_tag_ids parameter for tag pairing filter
    • AT+RANGE_ADDR=<tag> pairing command on connect
- uwb_config.yaml: range_rate / bearing_rate / enrolled_tag_ids params
- uwb.launch.py: expose new params as launch arguments
- test_ranging_math.py: 7 new bearing_from_pos unit tests

Closes #90

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-01 23:25:08 -05:00
sl-perception
d872ea5e34 feat(social): navigation + follow modes + MiDaS depth + waypoints (Issue #91) 
- saltybot_social_msgs: full message/service definitions (standalone compilation)
- saltybot_social_nav: social navigation orchestrator
  - Follow modes: shadow/lead/side/orbit/loose/tight
  - Voice steering: mode switching + route commands via /social/speech/*
  - A* obstacle avoidance on Nav2/SLAM occupancy grid (8-directional, inflation)
  - MiDaS monocular depth for CSI cameras (TRT FP16 + ONNX fallback)
  - Waypoint teaching + replay with WaypointRoute persistence
  - High-speed EUC tracking (5.5 m/s = ~20 km/h)
  - Predictive position extrapolation (0.3s ahead at high speed)
- Launch: social_nav.launch.py (social_nav + midas_depth + waypoint_teacher)
- Config: social_nav_params.yaml
- Script: build_midas_trt_engine.py (ONNX -> TRT FP16)
 2026-03-01 23:15:00 -05:00
sl-perception
84790412d6 feat(social): multi-modal person state tracker (Issue #82) 2026-03-01 23:08:22 -05:00
Sebastien Vayrette
ac6fcb9a42 docs: add Leap Motion, fix ESC step inset 12mm 2026-03-01 17:49:17 -05:00
Sebastien Vayrette
ea18b9ad72 docs: add ReSpeaker 2-Mic + SIM7600A 4G to wiring diagram 2026-03-01 16:32:34 -05:00
Sebastien Vayrette
0c40a1c4f4 docs: add full wiring diagram (Orin ↔ FC ↔ ESC) 2026-03-01 14:51:56 -05:00