Extends the bd-66hx serial protocol with two new Orin→ESP32 commands:
CMD_OTA_CHECK (0x10): triggers gitea_ota_check_now(), responds with
TELEM_VERSION_INFO (0x84) for Balance and IO (current + available ver).
CMD_OTA_UPDATE (0x11): uint8 target (0=balance, 1=io, 2=both) — triggers
uart_ota_trigger() for IO or ota_self_trigger() for Balance.
NACK with ERR_OTA_BUSY or ERR_OTA_NO_UPDATE on failure.
New telemetry: TELEM_OTA_STATUS (0x83, target+state+progress+err),
TELEM_VERSION_INFO (0x84, target+current[16]+available[16]).
Wires OTA stack into app_main: ota_self_health_check on boot,
gitea_ota_init + ota_display_init after peripherals ready.
CMakeLists updated with all OTA component dependencies.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds ota_display_task (5 Hz) on GC9A01 240×240 round LCD:
- Idle: orange dot badge at top-right when update available, version text
- Progress: arc sweeping 0→360° around display perimeter with phase label
- States: Downloading/Verifying/Applying/Rebooting (Balance) and
Downloading/Sending/Done (IO via UART)
- Error: red arc + "FAILED RETRY?" prompt
Display primitives (fill_rect, draw_string, draw_arc) are stubs called
from the GC9A01 SPI driver layer (separate driver bead).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Balance side (uart_ota.c): downloads io-firmware.bin from Gitea to RAM,
computes SHA256, then streams to IO over UART1 (GPIO17/18, 460800 baud)
as OTA_BEGIN/OTA_DATA/OTA_END frames with CRC8 + per-chunk ACK/retry (×3).
IO side (uart_ota_recv.c): receives frames, writes to inactive OTA partition
via esp_ota_write, verifies SHA256 on OTA_END, sets boot partition, reboots.
IO board main.c + CMakeLists.txt scaffold included.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Downloads balance-firmware.bin from Gitea release URL to inactive OTA
partition, streams SHA256 verification via mbedTLS, sets boot partition
and reboots. Auto-rollback via CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE if
ota_self_health_check() not called within 30 s of boot. Progress 0-100%
in g_ota_self_progress for display task.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds gitea_ota_check_task on Balance board: fetches Gitea releases API
every 30 min and on boot, filters by esp32-balance/ and esp32-io/ tag
prefixes, compares semver against embedded FW version, stores update info
(version string, download URL, SHA256) in g_balance_update / g_io_update.
WiFi credentials read from NVS namespace "wifi"; falls back to compile-time
DEFAULT_WIFI_SSID/PASS if NVS is empty.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add dual OTA partitions (ota_0/ota_1 × 1.75 MB each) and otadata to
both esp32s3/balance/ and esp32s3/io/ on 4 MB flash layouts.
Enable CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE and OTA HTTP on Balance.
Create esp32s3/io/ project scaffold with config.h pin assignments.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements full boot-time auto-start for the SaltyBot ROS2 stack on
Jetson Orin. Everything comes up automatically after power-on with
correct dependency ordering and restart-on-failure for each service.
New systemd services:
saltybot-ros2.service full_stack.launch.py (perception + SLAM + Nav2)
saltybot-esp32-serial.service ESP32-S3 BALANCE UART bridge (bd-wim1, PR #727)
saltybot-here4.service Here4 DroneCAN GPS bridge (bd-p47c, PR #728)
saltybot-dashboard.service Web dashboard on port 8080
Updated:
saltybot.target now Wants all four new services with
boot-order comments
can-bringup.service bitrate 500 kbps → 1 Mbps (DroneCAN for Here4)
70-canable.rules remove bitrate from udev RUN+=; let service
own the bitrate, add TAG+=systemd for device unit
install_systemd.sh installs all services + udev rules, colcon
build, enables mosquitto, usermod dialout
full_stack.launch.py resolve 8 merge conflict markers (ESP32-S3
rename) and fix missing indent on
enable_mission_logging_arg — file was
un-launchable with SyntaxError
New:
scripts/ros2-launch.sh sources ROS2 Humble + workspace overlay,
then exec ros2 launch — used by all
ROS2 service units via ExecStart=
udev/80-esp32.rules /dev/esp32-balance (CH343) and
/dev/esp32-io (ESP32-S3 native USB CDC)
Resolves bd-1hyn
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds .gitea/workflows/ota-release.yml: triggered on esp32-balance/vX.Y.Z
or esp32-io/vX.Y.Z tags, builds the corresponding ESP32-S3 project with
espressif/idf:v5.2.2, and attaches <app>_<version>.bin + .sha256 to the
Gitea release for OTA download.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces Orin↔ESP32-S3 BALANCE CAN comms (0x300-0x303 / 0x400-0x401)
with binary serial framing over CH343 USB-CDC at 460800 baud.
Protocol matches bd-wim1 (sl-perception) exactly:
Frame: [0xAA][LEN][TYPE][PAYLOAD][CRC8-SMBUS]
CRC covers LEN+TYPE+PAYLOAD, big-endian multi-byte fields.
Commands (Orin→ESP32): HEARTBEAT/DRIVE/ESTOP/ARM/PID
Telemetry (ESP32→Orin): TELEM_STATUS, TELEM_VESC_LEFT (ID 56),
TELEM_VESC_RIGHT (ID 68), ACK/NACK
VESC CAN TWAI kept for motor control; drive commands from Orin
forwarded to VESCs via SET_RPM. Hardware note: SN65HVD230
rewired from IO43/44 to IO2/IO1 to free IO43/44 for CH343.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rclpy RcutilsLogger.info/warning/debug() do not accept printf-style
positional format args. Also fix p["use_phone_timestamp"] → p["use_phone_ts"]
key mismatch in __init__ log line.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add _TOPIC_IOS_GPS = 'saltybot/ios/gps' constant
- Subscribe to saltybot/ios/gps in _on_mqtt_connect
- Dispatch to _handle_ios_gps() in _dispatch()
- _handle_ios_gps(): same logic as _handle_gps(), frame_id='ios_gps',
publishes to /saltybot/ios/gps via self._ios_gps_pub
- Add rx/pub/err/last_rx_ts counters for the new topic
- Add /saltybot/ios/gps to rosbridge_params.yaml topics_glob
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add /saltybot/phone/gps, /saltybot/phone/imu, /saltybot/phone/battery,
/saltybot/phone/bridge/status, /gps/fix, /gps/vel to topics_glob so
the browser GPS dashboard can receive phone-bridged GPS data.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously showed only phone GPS. Now also subscribes via ROSLIB to
saltybot/gps/fix + saltybot/gps/vel on the same rosbridge URL for
robot (SAUL-TEE) position. Blue marker+trail for phone (raw WS
{type:gps}), orange marker+trail for robot (ROS topics). Sidebar shows
phone speed/alt/heading/accuracy + robot lat/lon/speed + distance
between the two. FIT ALL button auto-zooms to show both. Status bar
badges for phone staleness and robot fix/vel freshness.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Subscribes to saltybot/phone/gps (JSON: {ts, lat, lon, alt_m,
accuracy_m, speed_ms, bearing_deg, provider}) and renders a blue
Leaflet marker + blue breadcrumb trail alongside the robot's
orange/cyan marker. Status bar now shows PHONE badge with stale
detection. Sidebar adds phone lat/lon/speed/accuracy/provider section.
Clear button resets both trails.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
3D-printable PETG cradle for FSESC 6.7 Pro Mini Dual on 2020 T-slot rail.
4x M5 T-nut mounting, open-top heatsink exposure, XT60/XT30/CAN cutouts,
floor grille and side louvre ventilation, M3 heat-set insert posts for
board retention. No supports required.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add saltybot_can_e2e_test package with 64 tests covering the full
Orin↔Mamba↔VESC CAN pipeline: drive commands, heartbeat timeout,
e-stop escalation, mode switching, and FC_VESC status broadcasts.
Tests run with plain pytest — no ROS2 or real CAN hardware required.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- UwbTagBleActivity: BLE scan filtered to 'UWB_TAG_XXXX' device names
- Connects to GATT service 12345678-1234-5678-1234-56789abcdef0
- Read/write JSON config char: sleep_timeout_s, display_brightness,
tag_name, uwb_channel, ranging_interval_ms, battery_report
- Subscribes to status + battery notification characteristics
- Material Design UI with scan list, config form, and live status
- Runtime BLE permission handling for API 26+ / API 31+
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add hw_button driver (PC2 active-low, 20ms debounce) with gesture detection:
- Single short press + 500ms quiet -> BTN_EVENT_PARK
- SHORT+SHORT+LONG combo (within 3s) -> BTN_EVENT_REARM_COMBO
New BALANCE_PARKED state: PID frozen, motors off, quick re-arm via button
combo without the 3-second arm interlock required from DISARMED.
FC_BTN (0x404) CAN frame sent to Orin on each event:
event_id 1=PARKED, 2=UNPARKED, 3=UNPARK_FAILED (pitch > 20 deg)
Includes 11 unit tests (1016 assertions) exercising debounce, bounce
rejection, short/long classification, sequence detection, and timeout.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add vesc_mqtt_relay_node.py to saltybot_phone: subscribes to
/vesc/left/state, /vesc/right/state, /vesc/combined ROS2 topics and
publishes JSON telemetry to saltybot/phone/vesc_{left,right,combined}
MQTT topics at 5 Hz per motor. 32 unit tests, no ROS2/paho required.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- VESCCANOdometryNode subscriptions now use left_state_topic/right_state_topic
params (defaulting to /vesc/left/state and /vesc/right/state) instead of
building /vesc/can_<id>/state from CAN IDs — those topics never existed
- Update right_can_id default: 79 → 68 (Mamba F722S architecture update)
- Update vesc_odometry_params.yaml: CAN IDs 61/79 → 56/68; add explicit
left_state_topic and right_state_topic entries; remove stale can_N comments
- All IDs remain fully configurable via ROS2 params
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FSESC 6.7 Pro Mini Dual uses CAN IDs 56/68, not 61/79. Updates all
driver, telemetry, and odometry bridge files to use correct defaults.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Issue #680 — IMU mount angle calibration:
- imu_cal_flash.h/.c: store pitch/roll offsets in flash sector 7
(0x0807FF00, 64 bytes; preserves PID records across sector erase)
- mpu6000_set_mount_offset(): subtracts offsets from pitch/roll output
- mpu6000_has_mount_offset(): reports cal_status=2 to Orin
- 'O' CDC command: capture current pitch/roll → save to flash → ACK JSON
- Load offsets on boot; report in printf log
CAN telemetry correction (Tee: production has no USB to Orin):
- FC_IMU (0x402): pitch/roll/yaw/cal_status/balance_state at 50 Hz
- orin_can_broadcast_imu() rate-limited to ORIN_IMU_TLM_HZ (50 Hz)
- FC_BARO (0x403): pressure_pa/temp_x10/alt_cm at 1 Hz (Issue #672)
- orin_can_broadcast_baro() rate-limited to ORIN_BARO_TLM_HZ (1 Hz)
Issue #685 — LED CAN override:
- ORIN_CAN_ID_LED_CMD (0x304): pattern/brightness/duration_ms from Orin
- orin_can_led_override volatile state + orin_can_led_updated flag
- main.c: apply pattern to LED state machine on each LED_CMD received
Orin side:
- saltybot_can_node.py: production SocketCAN bridge — reads 0x400-0x403,
publishes /saltybot/imu, /saltybot/balance_state, /saltybot/barometer;
subscribes /cmd_vel → 0x301 DRIVE; /saltybot/leds → 0x304 LED_CMD;
sends 0x300 HEARTBEAT at 5 Hz; sends 0x303 ESTOP on shutdown
- setup.py: register saltybot_can_node entry point + uart_bridge launch
Fix: re-apply --defsym __stack_end=_estack-0x1000 linker fix to branch
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
UnboundLocalError when _ser is None — lines was only assigned inside
the else branch. Move initialisation to function scope so the for-loop
outside the lock always has a valid list.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- can_driver: add filter bank 15 (all ext IDs → FIFO1) and widen bank 14
to accept all standard IDs; add can_driver_send_ext/std and ext/std
frame callbacks (can_driver_set_ext_cb / can_driver_set_std_cb)
- vesc_can: VESC 29-bit extended CAN protocol driver — send RPM to IDs 56
and 68 (FSESC 6.7 Pro Mini Dual), parse STATUS/STATUS_4/STATUS_5
big-endian payloads, alive timeout, JLINK_TLM_VESC_STATE at 1 Hz
- orin_can: Orin↔FC standard CAN protocol — HEARTBEAT/DRIVE/MODE/ESTOP
commands in, FC_STATUS + FC_VESC broadcast at 10 Hz
- jlink: add JLINK_TLM_VESC_STATE (0x8E), jlink_tlm_vesc_state_t (22 bytes),
jlink_send_vesc_state_tlm()
- main: wire vesc_can_init/orin_can_init; replace can_driver_send_cmd with
vesc_can_send_rpm; inject Orin CAN speed/steer into balance PID; add
Orin CAN estop/clear handling; add orin_can_broadcast at 10 Hz
- test: 56-test host-side suite for vesc_can; test/stubs/stm32f7xx_hal.h
minimal HAL stub for all future host-side tests
Safety: balance PID runs independently on Mamba — if Orin CAN link drops
(orin_can_is_alive() == false) the robot continues balancing in-place.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds slcan setup script and saltybot_can_bridge ROS2 package implementing
full CAN bus integration between the Orin and the Mamba motor controller /
VESC motor controllers via a CANable 2.0 USB dongle (slcan interface).
- jetson/scripts/setup_can.sh: slcand-based bring-up/tear-down for slcan0
at 500 kbps with error handling (already up, device missing, retry)
- saltybot_can_bridge/mamba_protocol.py: CAN message ID constants and
encode/decode helpers for velocity, mode, e-stop, IMU, battery, VESC state
- saltybot_can_bridge/can_bridge_node.py: ROS2 node subscribing to /cmd_vel
and /estop, publishing /can/imu, /can/battery, /can/vesc/{left,right}/state
and /can/connection_status; background reader thread, watchdog zero-vel,
auto-reconnect every 5 s on CAN error
- config/can_bridge_params.yaml: default params (slcan0, VESC IDs 56/68,
Mamba ID 1, 0.5 s command timeout)
- test/test_can_bridge.py: 30 unit tests covering encode/decode round-trips
and edge cases — all pass without ROS2 or CAN hardware
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
STM32Cube ld script provides _estack but not __stack_end. Define
__stack_end = _estack - 0x1000 (_Min_Stack_Size) via --defsym so
fault_mpu_guard_init() and fault_mem_c() can locate the stack bottom.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Mamba is mounted at ~12° on the frame, causing all three arm-interlock
checks to block arming. Raise fabsf(bal.pitch_deg) < 10.0f to 20.0f
at lines 375, 512, 532 (JLink arm, RC arm rising-edge, CDC arm).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Mamba F722S MK2 does not expose PB12/PB13 externally. Waveshare CAN
module is wired to the SCL (PB8) and SDA (PB9) header pads.
Changes in can_driver_init():
- Drop __HAL_RCC_CAN2_CLK_ENABLE() — CAN1 needs no slave clock
- GPIO: GPIO_PIN_12/13 → GPIO_PIN_8/9, GPIO_AF9_CAN2 → GPIO_AF9_CAN1
- Instance: CAN2 → CAN1
- Filter bank: 14 → 0 (CAN1 master banks start at 0; bank 14 is the
CAN2 slave-start boundary, unused here)
I2C1 is free: BME280 has been moved to I2C2 (PB10/PB11), so PB8/PB9
are available for CAN1 without any peripheral conflict.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Standalone panel ui/vesc_panel.{html,js,css} with live CAN telemetry
via rosbridge. Subscribes to /vesc/left/state, /vesc/right/state
(std_msgs/String JSON) and /vesc/combined for battery voltage.
Features:
- Canvas arc gauge per motor showing RPM + direction (FWD/REV/STOP)
- Current draw bar (motor + input), duty cycle bar, temperature bars
- FET and motor temperature boxes with warn/crit colour coding
- Sparkline charts for RPM and current (last 60 s, 120 samples)
- Battery card: voltage, total draw, both RPMs, SOC progress bar
- Colour-coded health: green/amber/red at configurable thresholds
- E-stop button: publishes zero /cmd_vel + /saltybot/emergency event
- Stale detection (2 s timeout → OFFLINE state)
- Hz counter + last-stamp display in header
- Mobile-responsive layout (single-column below 640 px)
- WS URL persisted in localStorage
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add vesc_mqtt_relay_node.py to saltybot_phone: subscribes to
/vesc/left/state, /vesc/right/state, /vesc/combined ROS2 topics and
publishes JSON telemetry to saltybot/phone/vesc_{left,right,combined}
MQTT topics at 5 Hz per motor. 32 unit tests, no ROS2/paho required.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- udev: 70-canable.rules — gs_usb VID/PID 1d50:606f, names iface can0 and brings it up at 500 kbps on plug-in
- systemd: can-bringup.service — oneshot service bound to sys-subsystem-net-devices-can0.device
- scripts: can_setup.sh — manual up/down/verify helper; candump verify for VESC IDs 61 (0x3D) and 79 (0x4F)
- install_systemd.sh updated to install can-bringup.service and all udev rules
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces ui/index.html (old USB-serial HUD) with a full rosbridge
dashboard. Adds ui/dashboard.{css,js}.
Top bar:
- Robot name + ⚡ SALTYBOT logo
- Live battery % + voltage with fill bar (4S LiPo: 12.0V–16.8V)
- Safety state from /saltybot/safety_zone/status (GREEN/AMBER/RED)
- E-stop state display
- Drive mode display
- ROS uptime counter
- rosbridge WS input + CONNECT button
Panel grid (auto-fill responsive):
- MAP VIEW (#587) — /saltybot/pose/fused liveness dot
- GAMEPAD TELEOP (#598) — /cmd_vel activity indicator
- DIAGNOSTICS (#562) — /diagnostics liveness dot
- EVENT LOG (#576) — /rosout liveness dot
- SETTINGS (#614) — param service (config state, no topic)
- GIMBAL (#551) — /gimbal/state liveness dot
Each card shows: icon, title, issue #, description, topic chips,
and a LIVE/IDLE/OFFLINE status badge updated every second. Cards
open the linked standalone panel in the same tab.
Auto-detect rosbridge:
- Probes: page hostname:9090, localhost:9090, saltybot.local:9090
- Progress dots per candidate (trying/ok/fail)
- Falls back to manual URL entry
- Saves last successful URL to localStorage
Bottom bar:
- ⛔ E-STOP button (latches, publishes zero Twist to /cmd_vel)
Space bar shortcut from dashboard
- RESUME button
- Drive mode switcher: MANUAL / AUTO / FOLLOW / DOCK
(publishes to /saltybot/drive_mode std_msgs/String)
- Session timer (HH:MM:SS since page load)
Info strip: rosbridge URL · msg rate · latency (5s ping via
/rosapi/get_time) · robot IP
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
phone/voice_cmd.py — listens via termux-speech-to-text, parses commands
(go forward/back, turn left/right, stop, e-stop, go to waypoint, speed
up/down, status) and publishes structured JSON to saltybot/phone/voice_cmd.
TTS confirmation via termux-tts-speak. Manual text fallback via --text flag.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Closed-loop yaw-rate controller that converts Jetson Twist.angular.z
to a differential wheel speed offset using IMU gyro Z as feedback.
- include/steering_pid.h + src/steering_pid.c: PID with anti-windup
(integral clamped to ±200 counts) and rate limiter (10 counts/ms
max output change) to protect balance PID from sudden steering steps.
JLINK_TLM_STEERING (0x8A) telemetry at 10 Hz.
- include/mpu6000.h + src/mpu6000.c: expose yaw_rate (board_gz) in
IMUData so callers have direct bias-corrected gyro Z feedback.
- include/jlink.h + src/jlink.c: add JLINK_TLM_STEERING (0x8A),
jlink_tlm_steering_t (8 bytes), jlink_send_steering_tlm().
- test/test_steering_pid.c: 78 unit tests (host build with gcc),
all passing.
Usage (main loop):
steering_pid_set_target(&s, jlink_state.steer * STEER_OMEGA_SCALE);
int16_t steer_out = steering_pid_update(&s, imu.yaw_rate, dt);
motor_driver_update(&motor, balance_cmd, steer_out, now_ms);
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a slow-adapting terrain slope estimator (IIR tau=5s) that decouples
the robot's balance offset from genuine ground incline. The balance
controller subtracts the slope estimate from measured pitch so the PID
balances around the slope surface rather than absolute vertical.
- include/slope_estimator.h + src/slope_estimator.c: first-order IIR
filter clamped to ±15°; JLINK_TLM_SLOPE (0x88) telemetry at 1 Hz
- include/jlink.h + src/jlink.c: add JLINK_TLM_SLOPE (0x88),
jlink_tlm_slope_t (4 bytes), jlink_send_slope_tlm()
- include/balance.h + src/balance.c: integrate slope_estimator into
balance_t; update, reset on tilt-fault and disarm
- test/test_slope_estimator.c: 35 unit tests, all passing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a slow-adapting terrain slope estimator (IIR tau=5s) that decouples
the robot's balance offset from genuine ground incline. The balance
controller subtracts the slope estimate from measured pitch so the PID
balances around the slope surface rather than absolute vertical.
- include/slope_estimator.h + src/slope_estimator.c: first-order IIR
filter clamped to ±15°; JLINK_TLM_SLOPE (0x88) telemetry at 1 Hz
- include/jlink.h + src/jlink.c: add JLINK_TLM_SLOPE (0x88),
jlink_tlm_slope_t (4 bytes), jlink_send_slope_tlm()
- include/balance.h + src/balance.c: integrate slope_estimator into
balance_t; update, reset on tilt-fault and disarm
- test/test_slope_estimator.c: 35 unit tests, all passing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds saltybot_bringup.launch.py with ordered startup groups (drivers→
perception→navigation→UI), timer-based health gates, configurable
profiles (minimal/full/debug), and estop on Ctrl-C shutdown.
Also adds launch_profiles.py dataclass module and 53-test coverage for
profile hierarchy, timing gates, safety bounds, and to_dict serialization.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add saltybot_safety_zone — ROS2 Python node that processes the RPLIDAR
A1M8 /scan into three concentric 360° safety zones, latches an e-stop
when DANGER is detected in the forward arc, and overrides /cmd_vel to
zero while the latch is active.
Zone thresholds (default):
DANGER < 0.30 m — latching e-stop in forward arc
WARN < 1.00 m — advisory (published in sector data)
CLEAR otherwise
Sector grid:
36 sectors of 10° each (sector 0 = robot forward, CCW positive).
Per-sector: angle_deg, zone, min_range_m, in_forward_arc flag.
E-stop behaviour:
- Latches after estop_debounce_frames (2) consecutive DANGER scans
in the forward arc (configurable ±30°, or all-arcs mode).
- While latched: zero Twist published to /cmd_vel every scan + every
incoming /cmd_vel_input message is blocked.
- Clear only via service (obstacle must be gone):
/saltybot/safety_zone/clear_estop (std_srvs/Trigger)
Published topics:
/saltybot/safety_zone String/JSON every scan
— per-sector {sector, angle_deg, zone, min_range_m, forward}
— estop_active, estop_reason, danger_sectors[], warn_sectors[]
/saltybot/safety_zone/status String/JSON 10 Hz
— forward_zone, closest_obstacle_m, danger/warn counts
/cmd_vel Twist zero when e-stopped
Subscribed topics:
/scan LaserScan — RPLIDAR A1M8
/cmd_vel_input Twist — upstream velocity (pass-through / block)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Anchor firmware for Makerfabs ESP32 UWB Pro (DW3000 chip). Two anchors
mount on SaltyBot (port/starboard), USB-connected to Jetson Orin.
- DS-TWR responder: Poll→Resp→Final with ±10cm accuracy
- Streams +RANGE:<id>,<mm>,<rssi_dbm> on Serial 115200
- AT command interface: AT+RANGE?, AT+RANGE_ADDR=, AT+ID?
- ANCHOR_ID 0/1 set at build time (env:anchor0 / env:anchor1)
- PlatformIO config for Makerfabs MaUWB_DW3000 library
- udev rules for /dev/uwb-anchor0 /dev/uwb-anchor1 USB symlinks
- Pin map: SCK=18 MISO=19 MOSI=23 CS=21 RST=27 IRQ=34
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add saltybot_uwb_position — ROS2 Python package that reads JSON range
measurements from an ESP32 DW3000 UWB tag over USB serial, trilaterates
the robot's absolute position from 3+ fixed infrastructure anchors, and
publishes position + TF2 to the rest of the stack.
Serial protocol (one JSON line per frame):
Full frame: {"ts":…, "ranges": [{"id":0,"d_mm":1500,"rssi":-65}, …]}
Per-anchor: {"id":0, "d_mm":1500, "rssi":-65.0}
Accepts both "d_mm" and "range_mm" field names.
Trilateration (trilateration.py, numpy, no ROS deps):
Linear least-squares: linearise sphere equations around anchor 0,
solve (N-1)x2 (2D) or (N-1)x3 (3D) system via np.linalg.lstsq.
2D mode (default): robot_z fixed, needs >=3 anchors.
3D mode (solve_z=true): full 3D, needs >=4 anchors.
Outlier rejection:
After initial solve, compute per-anchor residual |r_meas - r_pred|.
Reject anchors with residual > outlier_threshold_m (0.4 m default).
Re-solve with inliers if >= min_anchors remain.
Track consecutive outlier strikes; flag in /status after N strikes.
Kalman filter (KalmanFilter3D, constant-velocity, 6-state, numpy):
Predict-only coasting when anchors drop below minimum.
Q=0.05, R=0.10 (tunable).
Topics:
/saltybot/uwb/pose PoseStamped 10 Hz Kalman-filtered position
/saltybot/uwb/range/<id> UwbRange on arrival, raw per-anchor ranges
/saltybot/uwb/status String/JSON 10 Hz state+residuals+flags
TF2: uwb_link -> map (identity rotation)
Anchor config: flat float arrays in YAML.
Default layout: 4-anchor 5x5m room at 2m height.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a full gimbal control panel with live camera preview:
Standalone page (ui/gimbal_panel.html + .js + .css):
- Self-contained HTML page, no build step, served directly
- roslib.js via CDN, connects to rosbridge WebSocket
- 2-D canvas pan/tilt pad: click-drag + touch pointer capture
- Live camera stream (front/rear/left/right selector, base64 CompressedImage)
- FPS badge + angle overlay on video feed
- Preset positions: CENTER / LEFT / RIGHT / UP / DOWN
- Home button (0° / 0°)
- Person-tracking toggle → /gimbal/tracking_enabled
- Current angle display from /gimbal/state feedback
- WS URL persisted in localStorage
React component (GimbalPanel.jsx) + App.jsx integration:
- Same features in dashboard — TELEOP group → Gimbal tab
- Shares rosbridge connection from parent
- Mobile-responsive: stacks vertically on mobile, side-by-side on lg+
ROS topics:
PUB /gimbal/cmd geometry_msgs/Vector3
SUB /gimbal/state geometry_msgs/Vector3
PUB /gimbal/tracking_enabled std_msgs/Bool
SUB /camera/*/image_raw/compressed sensor_msgs/CompressedImage
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add saltybot_head_tracking — ROS2 Python node for automatic person-
following using dual-axis PID control targeting the pan/tilt camera head.
Pipeline:
1. Subscribe to /saltybot/objects (DetectedObjectArray from YOLOv8n)
2. Filter for class_id==0 (person); select best target by score:
score = 0.6 * 1/(1+dist_m) + 0.4 * confidence
(falls back to confidence-only when distance_m==0 / unknown)
3. Compute pixel error of bbox centre from image centre
4. Apply dead-zone (10 px default) to suppress micro-jitter
5. Convert pixel error to angle error via camera FOV
6. Independent PID controllers for pan and tilt axes
7. Accumulate PID output into absolute angle setpoint
8. Publish geometry_msgs/Point to /saltybot/gimbal/cmd:
x = pan_angle_deg, y = tilt_angle_deg, z = confidence
State machine:
IDLE -> waiting for first detection
TRACKING -> active PID
LOST -> hold last angle for hold_duration_s (3 s)
CENTERING -> return to (0, 0) at 20 deg/s -> IDLE
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add phone/voice_commander.py — Termux-based voice command listener for SaltyBot:
- Continuous wake word detection ('Hey Salty') via Whisper STT on short audio clips
- Command recording after wake word, transcribed with local Whisper (tiny/base/small)
- Parses go forward/back/left/right, stop, follow me, go home, look at me
- Publishes JSON to /saltybot/voice/cmd via ROS2 (rclpy) or rosbridge WebSocket
- TTS confirmation via termux-tts-speak; 'Yes?' prompt on wake word
- Fuzzy token-overlap fallback for wake word matching
- Flags: --host, --port, --model, --threshold, --record-sec, --no-tts, --debug
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Keep both Issue #531 (PID_RESULT telemetry) and Issue #533 (BATTERY
telemetry) additions in include/jlink.h and src/jlink.c.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add VoskSTT class to audio_utils.py: offline Vosk STT backend as
low-latency CPU alternative to Whisper for Jetson deployments
- Update audio_pipeline_node.py: stt_backend param ("whisper"/"vosk"),
Vosk loading with Whisper fallback, CPU auto-detection for Whisper,
dual-backend _process_utterance dispatch, STT/<backend> log prefix
- Update audio_pipeline_params.yaml: add stt_backend and vosk_model_path
- Add test/test_audio_pipeline.py: 40 unit tests covering EnergyVAD,
PCM conversion, AudioBuffer, UtteranceSegmenter, VoskSTT, JabraAudioDevice,
AudioMetrics, AudioState
- Integrate into full_stack.launch.py: audio_pipeline at t=5s with
enable_audio_pipeline and audio_stt_backend args
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implement Ziegler-Nichols relay feedback auto-tuning with flash persistence:
Firmware (STM32F722):
- pid_flash.c/h: erase+write Kp/Ki/Kd to flash sector 7 (0x0807FFC0),
magic-validated; load on boot to restore saved tune
- jlink.h: add JLINK_CMD_PID_SAVE (0x0A) and JLINK_TLM_PID_RESULT (0x83)
with jlink_tlm_pid_result_t struct and pid_save_req flag in JLinkState
- jlink.c: dispatch JLINK_CMD_PID_SAVE -> pid_save_req; add
jlink_send_pid_result() to confirm flash write outcome over USART1
- main.c: load saved PID from flash after balance_init(); handle
pid_save_req in main loop (disarmed-only, erase stalls CPU ~1s)
Jetson ROS2 (saltybot_pid_autotune):
- pid_autotune_node.py: add Ki to Ziegler-Nichols formula (ZN PID:
Kp=0.6Ku, Ki=1.2Ku/Tu, Kd=0.075KuTu); add JLink serial client that
sends JLINK_CMD_PID_SET + JLINK_CMD_PID_SAVE after tuning completes
- autotune_config.yaml: add jlink_serial_port and jlink_baud_rate params
Trigger: ros2 service call /saltybot/autotune_pid std_srvs/srv/Trigger
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add saltybot_depth_costmap — a Nav2 costmap2d plugin that converts
D435i depth images directly into obstacle markings on both local and
global costmaps.
Pipeline:
1. Subscribe to /camera/depth/image_rect_raw (16UC1 mm) + camera_info
2. Back-project depth pixels to 3D using pinhole camera intrinsics
3. Transform points to costmap global_frame via TF2
4. Apply configurable height filter (min_height..max_height above ground)
5. Mark obstacle cells as LETHAL_OBSTACLE
6. Inflate neighbours within inflation_radius as INSCRIBED_INFLATED_OBSTACLE
Parameters:
min_height: 0.05 m — floor clearance (ignores ground returns)
max_height: 0.80 m — ceiling cutoff (ignores lights/ceiling)
obstacle_range: 3.5 m — max marking distance from camera
clearing_range: 4.0 m — max distance processed at all
inflation_radius: 0.10 m — in-layer inflation (works before inflation_layer)
downsample_factor: 4 — process 1 of N rows+cols (~19k pts @ 640×480)
Integration (#478):
- Added depth_costmap_layer to local_costmap plugins list
- Added depth_costmap_layer to global_costmap plugins list
- Plugin registered via pluginlib (plugin.xml)
Files:
jetson/ros2_ws/src/saltybot_depth_costmap/
CMakeLists.txt, package.xml, plugin.xml
include/saltybot_depth_costmap/depth_costmap_layer.hpp
src/depth_costmap_layer.cpp
jetson/ros2_ws/src/saltybot_bringup/config/nav2_params.yaml (updated)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds TeleopWebUI component — a dedicated browser-based remote control
panel combining live video and joystick teleoperation in one view:
- Live camera stream (front/rear/left/right) via rosbridge CompressedImage
- Virtual joystick (canvas-based, touch + mouse, 10% deadzone)
- WASD / arrow-key keyboard fallback, Space for quick stop
- Speed presets: SLOW (20%), NORMAL (50%), FAST (100%)
- Latching E-stop button with pulsing visual indicator
- Real-time linear/angular velocity display
- Mobile-responsive: stacks vertically on small screens, side-by-side on lg+
- Added TELEOP tab group → Drive tab in App.jsx
Topics: /camera/<name>/image_raw/compressed (subscribe)
/cmd_vel geometry_msgs/Twist (publish)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three bugs prevented mpu6000_is_calibrated() from returning true,
blocking arming and balance mode:
1. WHO_AM_I single-attempt: one SPI glitch returning 0x00 caused
icm42688_init() to return -128, skipping mpu6000_calibrate()
entirely. Fix: retry WHO_AM_I up to 3 times with 10ms gaps.
2. icm42688_read() rx[15] uninitialized: if HAL_SPI_TransmitReceive()
failed, garbage stack data was accumulated as gyro bias. Fix: zero-
init rx[15] so failed transfers produce zero data.
3. mpu6000_calibrate() raw uninitialized: UB if icm42688_read() is
a no-op (imu_type mismatch). Fix: zero-init raw each iteration.
Also add SCB_InvalidateDCache_by_Addr() on SPI rx buffers in rreg()
and icm42688_read() for DCache coherency. Currently a no-op (DCache
is not enabled), but required if SCB_EnableDCache() is added — stack
buffers in SRAM2 are in the cacheable memory region on STM32F7.
Fix misleading DCache comment in icm42688.c (claimed DCache was
disabled by main.c; actually SCB_EnableDCache() is never called).
Build: 59904 bytes Flash (+512), 17100 bytes RAM — SUCCESS
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- esc_hoverboard.c: huart2 static in production; non-static only under
#ifdef DEBUG_MOTOR_TEST (needed by R command in jetson_uart.c)
- esc_hoverboard.c: UART5 diagnostic in hoverboard_backend_init() and
per-packet printf in hoverboard_backend_send() guarded by same flag
- esc_hoverboard.c: #include <stdio.h> also guarded (not needed in production)
- jetson_uart.c: R (baud sweep) and X (GPIO test) commands guarded by
#ifdef DEBUG_MOTOR_TEST — not compiled into production firmware
Production build: no debug output, static huart2, no R/X commands.
Debug build: define DEBUG_MOTOR_TEST to re-enable all diagnostics.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add saltybot_motor_daemon ROS2 package — Python daemon that subscribes
to /cmd_vel and drives the FC via W<speed>,<steer>\n over /dev/ttyTHS1
at 921600 baud.
- motor_daemon_node.py: 50 Hz fixed-rate TX, 200ms safety watchdog,
Twist→ESC conversion (±1000 range), FC ack parsing (W:<s>,<st>),
periodic ? status query, /diagnostics publisher, auto-reconnect
- config/motor_daemon_params.yaml: all tunable params with comments
- launch/motor_daemon.launch.py: parameterised launch file
- test/test_motor_daemon.py: 25 unit tests (all passing)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
USART1 IDLE interrupt (DMA circular RX) was calling HAL_UART_IRQHandler
mid-frame during polling HAL_UART_Transmit, resetting gState and causing
leading nulls / truncated frames on the Jetson telemetry link at 921600 baud.
Fix: introduce jlink_tx_locked() which disables USART1_IRQn around every
blocking HAL_UART_Transmit call, preventing IRQHandler from corrupting
gState while the TX loop is running. A s_tx_busy flag drops any
re-entrant caller (ESC debug, future USART6/VESC paths).
Both jlink_send_telemetry (50 Hz) and jlink_send_power_telemetry (1 Hz)
now use jlink_tx_locked(). Also correct the stale config.h comment that
misidentified the Jetson link as USART6 (it moved to USART1 in Issue #120).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add saltybot_motor_daemon ROS2 package — Python daemon that subscribes
to /cmd_vel and drives the FC via W<speed>,<steer>\n over /dev/ttyTHS1
at 921600 baud.
- motor_daemon_node.py: 50 Hz fixed-rate TX, 200ms safety watchdog,
Twist→ESC conversion (±1000 range), FC ack parsing (W:<s>,<st>),
periodic ? status query, /diagnostics publisher, auto-reconnect
- config/motor_daemon_params.yaml: all tunable params with comments
- launch/motor_daemon.launch.py: parameterised launch file
- test/test_motor_daemon.py: 25 unit tests (all passing)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Root causes confirmed from code audit:
1. DCache coherency: USB OTG FS reads physical SRAM while CPU writes through
DCache. Fix: MPU Region 0 marks 512B aligned USB buffer struct non-cacheable
(TEX=1, C=0, B=0) before HAL_PCD_Init(). DCache stays enabled globally.
2. IWDG ordering: safety_init() (IWDG start) deferred after all peripheral inits
to avoid watchdog reset during mpu6000_calibrate() (~510ms blocking).
DMA conflicts, GPIO conflicts, clock tree, and interrupt priorities all ruled out
with evidence. Full findings documented in USB_CDC_BUG.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implement automatic mission logging with bag recorder:
- Auto-records to ~/.saltybot-data/bags/ with 30min rotation
- Records mission-critical topics: /scan, /cmd_vel, /odom, /tf, /camera/color/image_raw/compressed, /saltybot/diagnostics
- MCAP format (preferred) with fallback to sqlite3 with zstd compression
- Services: /saltybot/save_bag, /saltybot/start_recording, /saltybot/stop_recording
- FIFO 20GB disk limit with automatic cleanup of oldest bags
- Auto-starts on launch, auto-saves on graceful shutdown
Changes:
- Updated bag_recorder_node.py with new parameters and services
- Changed default bag_dir to ~/.saltybot-data/bags/
- Set max_storage_gb to 20 (FIFO limit)
- Changed storage_format to MCAP by default
- Added start/stop recording service callbacks
- Updated package.xml description for mission logging
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Integrate saltybot_docking package into full_stack.launch.py
- Auto-trigger docking when battery drops to 20% (configurable via battery_low_pct)
- Launch docking at t=7s (after sensors, before Nav2)
- Add /saltybot/docking_state publisher (std_msgs/String) for state monitoring
- Update docking_params.yaml:
- battery_low_pct: 15% → 20% per Issue #489
- Add references to Issue #475 for conservative FC+hoverboard speeds
- Docking behavior includes:
- ArUco marker or IR beacon detection for dock location
- Nav2-based approach to pre-dock pose (~1m away)
- Visual servoing final alignment with contact detection
- Auto-undocking on full charge (80%) or command
- Integration with power management for mission interruption/resumption
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Integrate saltybot_docking package into full_stack.launch.py
- Auto-trigger docking when battery drops to 20% (configurable via battery_low_pct)
- Launch docking at t=7s (after sensors, before Nav2)
- Add /saltybot/docking_state publisher (std_msgs/String) for state monitoring
- Update docking_params.yaml:
- battery_low_pct: 15% → 20% per Issue #489
- Add references to Issue #475 for conservative FC+hoverboard speeds
- Docking behavior includes:
- ArUco marker or IR beacon detection for dock location
- Nav2-based approach to pre-dock pose (~1m away)
- Visual servoing final alignment with contact detection
- Auto-undocking on full charge (80%) or command
- Integration with power management for mission interruption/resumption
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Natural language voice command routing with fuzzy matching for speech variations.
Supported Commands:
- Follow me / Come with me
- Stop / Halt / Freeze
- Go home / Return to dock / Charge
- Patrol / Autonomous mode
- Come here / Approach
- Sit / Sit down
- Spin / Rotate / Turn around
- Dance / Groove
- Take photo / Picture / Smile
- What's that / Identify / Recognize
- Battery status / Battery level
Features:
- Fuzzy matching (rapidfuzz token_set_ratio) with 75% threshold
- Multiple pattern support per command for natural variations
- Three routing types: velocity (/cmd_vel), actions (/saltybot/action_command), services
- Command monitoring via /saltybot/voice_command
- Graceful handling of unrecognized speech
Architecture:
- Input: /saltybot/speech/transcribed_text (lowercase text)
- Fuzzy match against 11 command groups with 40+ patterns
- Route to: /cmd_vel (velocity), /saltybot/action_command (actions), or services
Files:
- saltybot_voice_router_node.py: Main router with fuzzy matching
- launch/voice_router.launch.py: Launch configuration
- VOICE_ROUTER_README.md: Usage documentation
Dependencies:
- rapidfuzz: Fuzzy string matching for natural speech handling
- rclpy, std_msgs, geometry_msgs: ROS2 core
Performance: <100ms per command (fuzzy matching + routing)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Update max_vel_x to 0.3 m/s (conservative for FC + hoverboard ESC)
- Update max_vel_theta to 0.5 rad/s (conservative for FC + hoverboard ESC)
- Set robot_radius to 0.22 m for 0.4m x 0.4m footprint
- Configure velocity smoother with conservative limits
- Both DWB local planner and velocity smoother updated for consistency
- RPLIDAR (/scan) + depth_to_laserscan (/depth_scan) costmap layers enabled
- NavFn global planner, DWB local planner configured
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implement automatic map serialization and persistence for slam_toolbox:
- New SlamToolboxPersistenceNode with auto-save every 5 minutes
- Auto-load most recent map on startup
- Services: /saltybot/save_map, /saltybot/load_map, /saltybot/list_maps
- Export to Nav2-compatible YAML + PGM format
- Stores maps in ~/.saltybot-data/maps/ with .posegraph format
- Integrates with slam_toolbox serialize/deserialize services
Changes:
- Created saltybot_mapping/slam_toolbox_persistence.py
- Added slam_toolbox_persistence.launch.py
- Updated slam.launch.py to include persistence service
- Updated CMakeLists.txt to install new executable
- Added slam_toolbox dependency to package.xml
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- New ROS2 node: power_supervisor_node for battery state monitoring
- Battery thresholds: 30% warning, 20% dock search, 10% graceful shutdown, 5% force kill
- Charge cycle tracking and battery health estimation
- CSV logging to battery_log.csv for external analysis
- Publishes /saltybot/power_state for MQTT relay
- Graceful shutdown cascade: save state, stop motors, disarm on critical low battery
- Replaces/extends Issue #125 battery_node with supervisor-level power management
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- saltybot_object_detection_msgs: DetectedObject, DetectedObjectArray, QueryObjects.srv
- saltybot_object_detection: YOLOv8n TensorRT FP16 node with depth projection
- Message filters for RGB-depth sync, TF2 transform to base_link
- Configurable confidence and class filtering (COCO 80 classes)
- Query service for voice integration ("whats in front of you")
- TensorRT build script with ONNX fallback
- Launch file with parameter configuration
- Full stack integration at t=6s (30 FPS target alongside person tracker)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implements hand and body gesture recognition via MediaPipe on Jetson Orin GPU.
- MediaPipe Hands (21-point hand landmarks) + Pose (33-point body landmarks)
- Recognizes: wave, point, stop_palm, thumbs_up, come_here, arms_up, arms_spread
- GestureArray publishing at 10–15 fps on Jetson Orin
- Confidence threshold: 0.7 (configurable)
- Range: 2–5 meters optimal
- GPU acceleration via Jetson Tensor RT
- Integrates with voice command router for multimodal interaction
- Temporal smoothing: history-based motion detection (wave, beckon)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implements hand and body gesture recognition via MediaPipe on Jetson Orin GPU.
- MediaPipe Hands (21-point hand landmarks) + Pose (33-point body landmarks)
- Recognizes: wave, point, stop_palm, thumbs_up, come_here, arms_up, arms_spread
- GestureArray publishing at 10–15 fps on Jetson Orin
- Confidence threshold: 0.7 (configurable)
- Range: 2–5 meters optimal
- GPU acceleration via Jetson Tensor RT
- Integrates with voice command router for multimodal interaction
- Temporal smoothing: history-based motion detection (wave, beckon)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implement GPS-based geofence safety system:
- Subscribe to /phone/gps and /odom for position tracking
- Support circle (default 50m radius) or polygon geofence
- Three zones: SAFE, WARNING (2m buffer), VIOLATION
- WARNING zone: slow + concern emotion + TTS warning + amber LED
- VIOLATION zone: stop + auto-return + red LED + TTS alert
- Publish /saltybot/geofence_state (UInt8: 0=SAFE, 1=WARNING, 2=VIOLATION)
- LED feedback colors: off (safe), amber (warning), red (violation)
- Auto-return to safe zone when breaching boundary
- Configurable via geofence_config.yaml (type, radius, warning distance, etc)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Automatic night mode detection and stealth operation:
- Ambient light detection: auto-switch at 50 lux threshold
- Motor speed reduction: 50% speed in stealth mode
- LED control: dim to 5% minimum brightness with slow blue fade
- Face-only mode: disable TTS speaker, show text on face
- IR-based tracking: use IR cameras only (RGB disabled)
- Face brightness: reduce to 30% for low-light visibility
- Manual override: voice commands and gamepad toggle (Y button)
- Smooth transitions: 1-second fade between modes with ramps
Features:
- Hysteresis: 5 lux band prevents mode flickering
- Light sensor smoothing: 5-sample averaging for stability
- Transition manager: smooth motor ramp (2s), LED fade (0.5s)
- Multiple sensor support: RealSense IR, phone ambient sensor
- Stealth LED pattern: slow breathing dim blue (0.3 Hz)
Configuration:
- YAML-based threshold and behavior settings
- Per-subsystem transition timing
- Tracking parameter tuning for IR mode
- Face control with contrast boost
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Add autonomous patrol mode with idle behaviors and social integration:
Patrol Node:
- Nav2 /navigate_to_pose integration for autonomous navigation
- Idle behaviors at each waypoint (10-30s pan-tilt sweep + greetings)
- Person detection → pause patrol → social mode → resume
- Battery monitoring: automatic return to dock at <25%
- Teleop override: can be interrupted by joystick control
- Voice commands: 'patrol start/stop' via speech
- Randomized waypoint order for variety
- State publishing to /saltybot/patrol_state
Features:
- Waypoint loading from patrol_params.yaml (x, y, yaw coordinates)
- Random idle durations (configurable 10-30s)
- Pan-tilt sweep and greeting at each waypoint
- Person detection pause with timeout
- Automatic dock return when battery low
- Polling loop for state management
- Voice control integration
State Machine:
IDLE ↔ NAVIGATE → IDLE_BEHAVIOR → (next waypoint)
↓
[Person Detected] → PAUSE_PERSON
↓
[Battery Low] → RETURN_TO_DOCK
↓
[Teleop/Voice] → IDLE
Configuration:
- Waypoints with name, x, y, yaw
- Idle time range (min/max)
- Battery dock threshold (default 25%)
- Person detection pause timeout
Topics:
- /saltybot/patrol_state (String)
- /saltybot/speech_text (String)
- /saltybot/pan_tilt_cmd (String)
- /saltybot/person_detections (Detection2DArray)
- /saltybot/teleop_cmd (String)
- /saltybot/voice_cmd (String)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
NeoPixel status indicators with animated patterns:
- Breathing blue (idle state)
- Green pulse (follow mode)
- Red flash (error state)
- Rainbow animation (celebrate)
- Amber pulse (low battery <20%)
- White chase (search mode)
Features:
- Smooth transitions between states
- Configurable LED count (default 30) and GPIO pin (default GPIO18)
- Auto-dim brightness control
- Subscribes to battery, balance, social, emotion, health topics
- Publishes LED state JSON to /saltybot/led_state
- 30Hz update frequency with multiple animation patterns
Configuration:
- YAML-based hardware and pattern settings
- Per-pattern speed and color customization
- State priority system for concurrent status indicators
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
NeoPixel status indicators with animated patterns:
- Breathing blue (idle state)
- Green pulse (follow mode)
- Red flash (error state)
- Rainbow animation (celebrate)
- Amber pulse (low battery <20%)
- White chase (search mode)
Features:
- Smooth transitions between states
- Configurable LED count (default 30) and GPIO pin (default GPIO18)
- Auto-dim brightness control
- Subscribes to battery, balance, social, emotion, health topics
- Publishes LED state JSON to /saltybot/led_state
- 30Hz update frequency with multiple animation patterns
Configuration:
- YAML-based hardware and pattern settings
- Per-pattern speed and color customization
- State priority system for concurrent status indicators
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implement centralized health monitoring node that:
- Subscribes to /saltybot/<node>/heartbeat from all tracked nodes
- Tracks expected nodes from YAML configuration
- Marks nodes DEAD if silent >5 seconds
- Triggers auto-restart via ros2 launch when nodes fail
- Publishes /saltybot/system_health JSON with full status
- Alerts face display on critical node failures
Features:
- Configurable heartbeat timeout (default 5s)
- Automatic dead node detection and restart
- System health JSON publishing (timestamp, uptime, node status, critical alerts)
- Face alert system for critical failures
- Rate-limited alerting to avoid spam
- Comprehensive monitoring config with critical/important node tiers
Package structure:
- saltybot_health_monitor: Main health monitoring node
- health_config.yaml: Configurable list of monitored nodes
- health_monitor.launch.py: Launch file with parameters
- Unit tests for heartbeat parsing and health status generation
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implement multi-person tracking with:
- Track up to 10 people with persistent unique IDs
- Target priority: wake-word speaker > closest known > largest bbox
- Occlusion handoff with 3-second grace period
- Re-ID via face embedding (cosine similarity) + HSV color histogram
- Group detection and centroid calculation
- Lost target behavior: stop + rotate + SEARCHING state
- 15+ fps on Jetson Orin Nano Super
- PersonArray message publishing with active target tracking
- Configurable similarity thresholds and grace periods
- Unit tests for tracking, matching, priority, and re-ID
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Central ROS2 TTS node using Piper (offline ONNX speech synthesis)
- Subscribe to /saltybot/tts_request (String messages) for TTS requests
- Priority queue management with interrupt capability
- Audio output to Jabra device via ALSA/PulseAudio
- Configurable voice, speed, pitch, and volume parameters
- Publish /saltybot/tts_state (idle/synthesizing/playing) for status tracking
- Preload Piper model on startup for faster synthesis
- Queue management with configurable max size (default 16)
- Non-blocking async playback via worker thread
- Complete ROS2 package with launch file and tests
- Update battery_low_pct from 15% to 20% for auto-dock trigger (Issue #410 requirement)
- Add social mood publisher for charging animation display
- Trigger face charging animation (happy mood) when entering CHARGING state
- ArUco marker ID 42 detection via RealSense D435i (already configured)
- Approach sequence: detect → align → slow → contact → verify charging
- 360° search then expand capability (DETECTING state handles search)
- Safety abort timeouts already implemented in FSM state machine
- IR beacon fallback detection also available
- ROS2 node for balance mode PID parameter management via pyvesc UART
- Tilt safety kill switch: ±45° pitch > 500ms triggers motor cutoff
- Startup ramp: gradual acceleration from 0 to full output over configurable duration
- IMU integration: subscribe to /imu/data for pitch/roll angle computation
- State publishing: /saltybot/balance_state with tilt angles, PID values, motor telemetry
- Data logging: /saltybot/balance_log publishes CSV-formatted IMU + motor data
- Configurable parameters: PID gains, tilt thresholds, ramp duration, control frequency
- Test suite: quaternion to Euler conversion, tilt safety checks, startup ramp
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- ROS2 node for balance mode PID parameter management via pyvesc UART
- Tilt safety kill switch: ±45° pitch > 500ms triggers motor cutoff
- Startup ramp: gradual acceleration from 0 to full output over configurable duration
- IMU integration: subscribe to /imu/data for pitch/roll angle computation
- State publishing: /saltybot/balance_state with tilt angles, PID values, motor telemetry
- Data logging: /saltybot/balance_log publishes CSV-formatted IMU + motor data
- Configurable parameters: PID gains, tilt thresholds, ramp duration, control frequency
- Test suite: quaternion to Euler conversion, tilt safety checks, startup ramp
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implement circular buffer bag recorder with:
- Configurable topics recording
- Last N minutes circular buffer (default 30min)
- Manual save trigger via /saltybot/save_bag service
- Auto-save on crash with signal handlers
- Storage management (7-day TTL, 50GB quota)
- Compression via zstd
- Optional rsync to NAS for backup
- Periodic maintenance (cleanup expired, enforce quota)
Saves to /home/seb/rosbags/ by default.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implement EncounterSyncService ROS2 node for managing offline-first
encounter data syncing. Features:
- Monitors /home/seb/encounter-queue/ for JSON files
- Uploads to configurable cloud API when connectivity detected
- Exponential backoff retry with max 5 attempts
- Moves synced files to /home/seb/encounter-queue/synced/
- Publishes status on /social/encounter_sync_status topic
- Connectivity check via HTTP ping (configurable URL)
- Handles offline operation gracefully
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implement EncounterSyncService ROS2 node for managing offline-first
encounter data syncing. Features:
- Monitors /home/seb/encounter-queue/ for JSON files
- Uploads to configurable cloud API when connectivity detected
- Exponential backoff retry with max 5 attempts
- Moves synced files to /home/seb/encounter-queue/synced/
- Publishes status on /social/encounter_sync_status topic
- Connectivity check via HTTP ping (configurable URL)
- Handles offline operation gracefully
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implement state machine for detecting and enrolling unknown persons.
Manages workflow: DETECT → GREET → ASK_NAME → SMALL_TALK → ENROLL → FAREWELL
Features:
- Subscribes to /saltybot/person_tracker for unknown face detection
- Unknown person threshold configurable (default: 30% confidence)
- State machine with Piper TTS triggers for each state
- Captures STT responses for name and conversation context
- Publishes /social/orchestrator/state for coordination with other nodes
- Handles person interruptions gracefully (walks away)
- Auto-enrolls person to face gallery (configurable)
- Stores encounter data as JSON in /home/seb/encounter-queue/
- Tracks duration, responses, interests, and enrollment success
Encounter data structure:
{
person_id, timestamp, state, name, context, greeting_response,
interests[], enrollment_success, duration_sec, notes
}
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Creates log-mel spectrogram template for 'hey salty' wake word detection
using synthetic speech generation. Template generated from 5 synthetic
audio samples with varying pitch to improve robustness.
- generate_wake_word_template.py: Script to synthesize and generate template
- hey_salty.npy: 40-band log-mel template (40, 61) shape
- wake_word_params.yaml: Updated template_path
- README.md: Documentation for template usage and retraining procedures
The template is used by wake_word_node.py via cosine similarity matching
against incoming audio. Configurable sensitivity via match_threshold.
Future work: Collect real training recordings to improve accuracy.
- Detect if MCU was reset by IWDG watchdog timeout at startup
- Log watchdog reset events to debug terminal (USB CDC)
- Store watchdog reset flag for status reporting to Jetson
- Watchdog timer configured with 2-second timeout in safety_init()
- Main loop calls safety_refresh() to kick the watchdog every iteration
The IWDG (Independent Watchdog) resets the MCU if the main loop
hangs and fails to call safety_refresh() within the timeout window.
This provides hardware-enforced detection of software failures.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Adds #include "bno055.h" to src/main.c to resolve implicit declaration
warnings for bno055_read(), bno055_calib_status(), and bno055_temperature().
Functions were properly implemented but header was missing from includes.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implement SCS/STS serial protocol driver for Waveshare ST3215 servos
at 1Mbps daisy-chain configuration. Pan and tilt servos on single UART.
Features:
- SCSServoBus class: Low-level protocol handler with packet construction
- Position write commands with configurable speed (0-1000)
- Position and temperature readback from servos
- PanTiltNode: ROS2 node with target tracking control loop
- Subscribes to /saltybot/target_track for centroid position
- Proportional control to keep target centered in D435i FOV
- Publishes /pan_tilt/state with angles and temperatures
- Publishes /pan_tilt/command for servo position monitoring
- 30 Hz control loop, 1 Hz telemetry loop
- Configurable servo limits and speeds
Servos: 0.24° resolution, 0-4095 position range
Camera: 87° × 58° field of view
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Add coulomb counter for accurate SoC estimation independent of load:
- New coulomb_counter module: integrate current over time to track Ah consumed
* coulomb_counter_init(capacity_mah) initializes with battery capacity
* coulomb_counter_accumulate(current_ma) integrates current at 100 Hz
* coulomb_counter_get_soc_pct() returns SoC 0-100% (255 = invalid)
* coulomb_counter_reset() for charge-complete reset
- Battery module integration:
* battery_accumulate_coulombs() reads motor INA219 currents and accumulates
* battery_get_soc_coulomb() returns coulomb-based SoC with fallback to voltage
* Initialize coulomb counter at startup with DEFAULT_BATTERY_CAPACITY_MAH
- Telemetry updates:
* JLink STATUS: use coulomb SoC if available, fallback to voltage-based
* CRSF battery frame: now includes remaining capacity in mAh (from coulomb counter)
* CRSF capacity field was always 0; now reflects actual remaining mAh
- Mainloop integration:
* Call battery_accumulate_coulombs() every tick for continuous integration
* INA219 motor currents + 200 mA subsystem baseline = total battery draw
Motor current sources (INA219 addresses 0x40/0x41) provide most power draw;
Jetson ROS2 battery_node already prioritizes coulomb-based soc_pct from STATUS frame.
Default capacity: 2200 mAh (typical lab 3S LiPo); configurable via firmware parameter.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Replace safety.c's direct IWDG initialization with watchdog module API
- Use watchdog_init(2000) for ~2s timeout in safety_init()
- Use watchdog_kick() in safety_refresh() to feed the watchdog
- Remove unused watchdog_get_divider() helper function
- Watchdog now configured with automatic prescaler selection
The watchdog module provides a clean, flexible IWDG interface that:
- Automatically calculates prescaler and reload values
- Detects watchdog-triggered resets via watchdog_was_reset_by_watchdog()
- Supports timeout range of ~1ms to ~32 seconds
- Integrates seamlessly with existing safety system
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Polls /proc/stat (CPU delta), /proc/meminfo (RAM), os.statvfs (disk),
/sys/devices/gpu.0/load (GPU), and thermal zone sysfs paths; publishes
JSON payload on /saltybot/system_resources at 1 Hz.
Pure helpers (parse_proc_stat, cpu_percent_from_stats, parse_meminfo,
compute_ram_stats, read_disk_usage, read_gpu_load, read_thermal_zones)
are all unit-tested offline. Injectable I/O on SysmonNode allows full
node tick tests without /proc or /sys. 67/67 tests passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move camera viewer from TELEMETRY to new CAMERAS tab group (rose color).
Reorganizes tab structure to separate media capture from system telemetry.
CameraViewer.jsx already provides comprehensive MJPEG stream support:
- Multi-camera switching (7 total: front/left/rear/right CSI, D435i RGB/depth, panoramic)
- FPS counter per camera with quality badge (FULL/GOOD/LOW/NO SIGNAL)
- Resolution and camera info display
- Detection overlays (faces, gestures, scene objects)
- Picture-in-picture support (up to 3 pinned cameras)
- Video recording (MP4/WebM) and snapshot capture
- 360° panoramic viewer with mouse drag pan
- Color-coded quality indicators based on FPS
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Features:
- Subscribes to /saltybot/hands (21 landmarks per hand - MediaPipe format)
- Subscribes to /saltybot/hand_gesture (String gesture label)
- Canvas-based hand skeleton rendering with bone connections
- Support for dual hand tracking (left and right)
- Handedness indicators with color coding
* Left hand: green
* Right hand: yellow
- Real-time gesture display with confidence indicator
- Per-landmark confidence visualization
- Bone connections between all 21 joints
Hand Skeleton Features:
- 21 MediaPipe landmarks per hand
* Wrist (1)
* Thumb (4)
* Index finger (4)
* Middle finger (4)
* Ring finger (4)
* Pinky finger (4)
- 20 bone connections between joints
- Confidence-based rendering (only show high-confidence points)
- Scaling and normalization for viewport
- Joint type indicators (tips with ring outline)
- Glow effects around landmarks
Gesture Recognition:
- Real-time gesture label display
- Confidence percentage (0-100%)
- Color-coded confidence:
* Green: >80% (high confidence)
* Yellow: 50-80% (medium confidence)
* Blue: <50% (detecting)
Hand Status Display:
- Live detection status for both hands
- Visual indicators (✓ detected / ◯ not detected)
- Dual-hand canvas rendering
- Gesture info panel with confidence bar
Integration:
- Added to SOCIAL tab group as "Hands" tab
- Positioned after "Faces" tab
- Uses subscribe hook for real-time updates
- Dark theme with color-coded hands
- Canvas-based rendering for smooth visualization
Build: 125 modules, no errors
Main bundle: 270.08 KB
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Features:
- Subscribes to /diagnostics (diagnostic_msgs/DiagnosticArray)
- Hardware status cards per subsystem with color-coded health
- Real-time error and warning counts
- Expandable diagnostic cards with detailed key-value pairs
- Diagnostic status timeline with timestamps
- Aggregated system health summary
- Status indicators: OK (green), WARNING (yellow), ERROR (red), STALE (gray)
- Hardware Status Display
* Per-subsystem diagnostic cards
* Status level with color coding
* Expandable details with key values
* Hardware ID tracking
* Name and message display
- Health Summary Card
* Total diagnostic count
* OK/WARNING/ERROR/STALE breakdowns
* Overall system health status
* Visual status indicator
- Timeline and History
* Recent status timeline (10 latest events)
* Timestamp tracking
* Status transitions
* Scrollable history
- Status Legend
* Color-coded reference guide
* Status descriptions
* Quick status lookup
Integration:
- Added to MONITORING tab group as first tab (highest priority)
- Uses subscribe hook for real-time updates
- Dark theme with comprehensive status visualization
- Max 100 diagnostic events in history
Build: 124 modules, no errors
Main bundle: 264.31 KB
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implements saltybot_pure_pursuit package:
- Pure pursuit algorithm for path following with configurable parameters
- Lookahead distance (0.5m default) for target point on path
- Goal tolerance (0.1m) for goal detection
- Heading error correction to reduce speed when misaligned with path
- Publishes Twist commands on /cmd_vel_tracked for Nav2 integration
- Subscribes to /odom (odometry) and /path (Path trajectory)
- Tracks and publishes cross-track error for monitoring
Pure pursuit geometry:
- Finds closest point on path to robot current position
- Looks ahead specified distance along path from closest point
- Computes steering angle to follow circular arc to lookahead point
- Reduces linear velocity when heading error is large (with correction enabled)
- Clamps velocities to configurable maximums
Configuration parameters:
- lookahead_distance: 0.5m (typical range: 0.1-1.0m)
- goal_tolerance: 0.1m (distance to goal before stopping)
- heading_tolerance: 0.1 rad (unused but can support in future)
- max_linear_velocity: 1.0 m/s
- max_angular_velocity: 1.57 rad/s
- use_heading_correction: true (reduces speed on large heading errors)
Comprehensive test suite: 20+ tests covering:
- Geometric calculations (distance, quaternion conversions)
- Path following logic (empty path, straight/curved/spiral paths)
- Steering calculations (heading errors, velocity limits)
- Edge cases and realistic scenarios
- Control loop integration
- Parameter variations
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Critical bug fix:
- Added missing StatusHeader import (used in JSX line 215)
- Added missing LogViewer import (used in JSX line 291)
- Added missing MotorCurrentGraph import (used in JSX line 264)
These imports were referenced in JSX but not imported, causing dashboard crashes on load.
Build verification:
- 122 modules, all compiled successfully
- No errors or warnings
- Bundle: 255.38 KB main
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Polls /dev/video* at 2 Hz, drives a three-state machine
(connected/disconnected/restarting) and publishes to
/saltybot/camera_status (std_msgs/String). Reconnects within
restart_grace_s (5 s) → 'restarting' held for restart_hold_s (2 s)
to signal downstream capture pipelines to restart. Scan function
is injected for offline testing. 82/82 tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Features:
- Virtual dual-stick gamepad interface
* Left stick: linear velocity (forward/backward)
* Right stick: angular velocity (turn left/right)
- Canvas-based rendering with real-time visual feedback
- Velocity vector visualization on sticks
- 10% deadzone on both axes for dead-stick detection
- WASD keyboard fallback for manual driving
* W/S: forward/backward
* A/D: turn left/right
* Smooth blending between gamepad and keyboard input
- Speed limiter slider (0-100%)
* Real-time control of max velocity
* Disabled during e-stop state
- Emergency stop button (e-stop)
* Immediate velocity zeroing
* Visual status indicator (red when active)
* Prevents accidental motion during e-stop
- ROS Integration
* Publishes geometry_msgs/Twist to /cmd_vel
* Max linear velocity: 0.5 m/s
* Max angular velocity: 1.0 rad/s
* 50ms update rate
- UI Features
* Real-time velocity display (m/s and rad/s)
* Status bar with control mode indicator
* Speed limit percentage display
* Control info with keyboard fallback guide
Integration:
- Replaces previous split-view control layout
- Enhanced TELEMETRY/control tab
- Comprehensive gamepad + keyboard control system
Build: 120 modules, no errors
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Energy-gated log-mel + cosine-similarity wake-word node. Subscribes to
/social/speech/audio_raw (PCM-16 UInt8MultiArray), maintains a 1.5 s
sliding ring buffer, runs detection every 100 ms; fires Bool(True) on
/saltybot/wake_word_detected with 2 s cooldown. Template loaded from
.npy file; passive (no detections) when template_path is empty.
91/91 tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Features:
- Enumerate active ROS nodes from /rosout activity
- Real-time node status tracking (alive/dead based on heartbeat)
- Heartbeat timeout: 5 seconds without updates = dead
- Display node name, status, uptime, and last seen timestamp
- Color-coded status indicators (green=alive, gray=dead)
- Sortable table with node statistics
- Summary card showing alive/dead node counts
- Periodic status polling every 2 seconds
Integration:
- Added to MONITORING tab group as 'Nodes' tab
- Subscribes to /rosout (rcl_interfaces/Log) to detect active nodes
- Real-time updates with smooth transitions
Build: 119 modules, no errors
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Features:
- Circular gauge visualization for CPU and GPU temperatures
- Color-coded temperature zones: green <60°C, yellow 60-75°C, red >75°C
- Real-time needle pointer animation
- Fan speed percentage display for each sensor
- Peak temperature tracking and max reached indicator
- Thermal alert status (Normal/Caution/Critical)
- ROS subscription to /saltybot/thermal_status
- Integrated into TELEMETRY tab group
Build: 118 modules, no errors
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implements ROS2 adaptive PID gain scheduler for SaltyBot with:
- Subscribes to /saltybot/speed_scale for speed conditions
- Subscribes to /saltybot/terrain_roughness for surface conditions
- Adjusts PID gains dynamically:
* P gain increases with terrain roughness (better response on rough)
* D gain decreases at low speed (prevent oscillation when slow)
* I gain scales with both conditions for stability
- Publishes Float32MultiArray [Kp, Ki, Kd] on /saltybot/pid_gains
- Configurable scaling factors for each gain modulation
- Includes 18+ unit tests for gain scheduling logic
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Real-time motor current visualization with:
- Subscribes to /saltybot/motor_currents for dual-motor current data
- Rolling 60-second history window with automatic data culling
- Dual-axis line chart for left (cyan) and right (amber) motor amps
- Canvas-based rendering for performance
- Thermal warning threshold line (25A, configurable)
- Real-time statistics:
* Current draw for left and right motors
* Peak current tracking over 60-second window
* Average current calculation
* Thermal status indicator with warning badge
- Color-coded thermal alerts:
* Red background when threshold exceeded
* Warning indicator and message
- Grid overlay, axis labels, time labels, legend
- Takes absolute value of currents (handles reverse direction)
Integrated into TELEMETRY tab group as 'Motor Current' tab.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Real-time motor current visualization with:
- Subscribes to /saltybot/motor_currents for dual-motor current data
- Rolling 60-second history window with automatic data culling
- Dual-axis line chart:
* Left motor current (cyan) in Amps
* Right motor current (amber) in Amps
- Canvas-based rendering for performance
- Thermal warning threshold line (25A, configurable):
* Dashed red line overlay on chart
* Configurable threshold constant for motor specs
* Auto-detects thermal warnings when threshold exceeded
- Real-time statistics display:
* Current draw for left and right motors (real-time)
* Peak current tracking over 60-second window
* Average current calculation
* Thermal status indicator with warning badge
- Color-coded thermal alerts:
* Red background when threshold exceeded
* Warning indicator with icon
* Left/right motor temperature monitoring
- Grid overlay, axis labels, time labels, legend
- Takes absolute value of currents (handles reverse direction)
Integrated into TELEMETRY tab group as 'Motor Current' tab.
Follows established canvas rendering patterns from BatteryChart.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implements ROS2 geofence enforcer for SaltyBot with:
- Loads polygon geofence from params (list of x/y vertices)
- Subscribes to /odom for real-time robot position
- Point-in-polygon ray casting algorithm for boundary checking
- Publishes Bool on /saltybot/geofence_breach on boundary violation
- Optional enforcement flag for cmd_vel zeroing
- Configurable safety margin
- Includes 20+ unit tests for geometry and breach detection
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implements STM32F722 driver for WS2812 NeoPixel 8-LED ring with finite state machine.
Features:
- 8 operational states with animations:
* BOOT: Blue pulse (0.5 Hz)
* IDLE: Green breathe (0.5 Hz)
* ARMED: Solid green
* NAV: Cyan spin (1 Hz)
* ERROR: Red flash (2 Hz)
* LOW_BATT: Orange blink (1 Hz)
* CHARGING: Green fill (1 Hz)
* ESTOP: Red solid
- Non-blocking tick-based animation system
- State transitions via API
- PWM control on PB4 (TIM3_CH1) at 800 kHz
- Color interpolation for smooth effects
All 25 unit tests passing covering state transitions, animations, timing, and edge cases.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Topic bandwidth tracking and visualization with:
- Tracks message rates for all subscribed ROS topics
- Estimates bandwidth based on message size and frequency
- Message size estimated from JSON serialization
- Updates every 1 second with rolling 30-second history window
- Sortable table display:
* Topic name with truncation for long names
* Message rate (messages per second)
* Average message size (bytes)
* Bandwidth estimate (B/s, KB/s, or MB/s)
* Sparkline mini-chart showing bandwidth trend
- Total bandwidth summary at top
- Click column headers to sort (ascending/descending toggle)
- Visual indicators with color-coded columns
Integrated into MONITORING tab group as 'Bandwidth' tab.
Component provides window.__trackRosMessage() hook for optional
bandwidth tracking integration with ROS bridge.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implements ROS2 cliff detector for SaltyBot with:
- Subscribes to /saltybot/cliff_sensors (IR range array)
- Threshold-based detection (default 0.5m)
- Debouncing (3 consecutive frames) for robustness
- Majority voting (min 2 sensors) for safety
- Publishes Bool on /saltybot/cliff_detected
- Emergency stop trigger on cliff/drop-off detection
- Includes 15+ unit tests for detection logic
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Adds face_track_servo_node to saltybot_social:
- Subscribes /social/faces/detected (FaceDetectionArray)
- Picks closest face by largest bbox area (proximity proxy)
- Computes pan/tilt error from bbox centre vs image centre using
configurable FOV (fov_h_deg=60°, fov_v_deg=45°)
- Independent PID controllers for pan and tilt (velocity/incremental
output with anti-windup); servo position integrates velocity*dt
- Clamps commands to ±pan_limit_deg / ±tilt_limit_deg
- Returns to centre at return_rate_deg_s when face lost >lost_timeout_s
- Dead zone suppresses jitter for small errors
- Publishes Float32 on /saltybot/head_pan and /saltybot/head_tilt
- 81/81 tests passing
Closes#279
Real-time battery history visualization with:
- Subscribes to /saltybot/battery_state for continuous battery data
- Rolling 30-minute history window with automatic data culling
- Dual-axis line chart: voltage (left, cyan) and percentage (right, amber)
- Canvas-based rendering for performance
- Charge/discharge rate calculation (last 5-minute average):
* Voltage rate in mV/min with up/down/stable indicator
* Percentage rate in %/min with up/down/stable indicator
- Grid overlay, axis labels, time labels, and legend
- Current stats display: voltage, percentage, rates
- Responsive canvas sizing
Integrated into TELEMETRY tab group as 'Battery History' tab.
Follows established canvas rendering and data subscription patterns.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implements ROS2 IMU gyro + accel calibration node with:
- Service-triggered calibration via /saltybot/calibrate_imu
- Optional auto-calibration on startup (configurable)
- Collects N stationary samples (default 100)
- Computes mean bias offsets for gyro and accel
- Publishes bias-corrected IMU on /imu/calibrated
- Includes 10+ unit tests for calibration logic
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Real-time ROS log stream viewer with:
- Subscribes to /rosout (rcl_interfaces/Log)
- Severity-based color coding:
DEBUG=grey | INFO=white | WARN=yellow | ERROR=red | FATAL=magenta
- Filter by severity level (multi-select toggle)
- Filter by node name (text input)
- Auto-scroll to latest logs
- Max 500 logs in history (configurable)
- Scrolling log output in monospace font
- Proper timestamp formatting (HH:MM:SS)
Integrated into MONITORING tab group as 'Logs' tab alongside 'Events'.
Follows established React/Tailwind patterns from other dashboard components.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Adds greeting_trigger_node to saltybot_social:
- Subscribes to /social/faces/detected (FaceDetectionArray) for face arrivals
- Subscribes to /social/person_states (PersonStateArray) to cache face_id→distance
- Fires greeting when face_id is within proximity_m (default 2m) and
not in per-face_id cooldown window (default 300s)
- Publishes JSON on /saltybot/greeting_trigger:
{face_id, person_name, distance_m, ts}
- unknown_distance param controls assumed distance for faces with no PersonState yet
- Thread-safe distance cache and greeted map
- 50/50 tests passing
Closes#270
Adds multi-pass spatial-Gaussian hole filler for D435i depth images.
Each pass replaces zero/NaN pixels with the Gaussian-weighted mean of valid
neighbours in a growing kernel (×1, ×2.5, ×6 default); original valid
pixels are never modified. Handles uint16 mm → float32 m conversion,
border pixels via BORDER_REFLECT, and above-d_max pixels as holes.
Publishes filled float32 depth on /camera/depth/filled at camera rate.
37/37 pure-Python tests pass (no ROS2 required).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Persistent top bar showing real-time robot health indicators:
- Battery percentage and voltage (color-coded: green >60%, amber 30-60%, red <30%)
- WiFi signal strength (RSSI dBm with quality assessment)
- Motor status and current draw in Amperes
- Emergency state indicator (red highlight when active)
- System uptime in hours and minutes
- Current operational mode (idle/nav/social/docking)
- Connection status indicator
Component subscribes to relevant ROS topics and displays in compact
flex layout matching dashboard dark theme.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implements STM32F722 driver for brushless cooling fan on PA9 using TIM1_CH2 PWM.
Features:
- Temperature-based speed curve: off <40°C, 30% at 50°C, 100% at 70°C
- Smooth speed ramp transitions with configurable rate (default 0.05%/ms)
- Linear interpolation between curve points
- PWM duty cycle control (0-100%)
- State transitions and edge case handling
All 51 unit tests passing:
- Temperature curve verification (6 test zones)
- Speed boundaries and transitions
- Ramp timing and rate control
- PWM duty cycle calculation
- Temperature extremes and boundary conditions
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Interactive waypoint editor for Nav2 goal-based navigation:
- Click on map display to place waypoints
- Drag waypoints in list to reorder navigation sequence
- Right-click waypoints to delete them
- Visual waypoint overlay on map with numbering
- Robot position indicator at center
- Waypoint list sidebar with selection and ordering
- Send Nav2 goal to individual selected waypoint
- Execute all waypoints in sequence with automatic progression
- Clear all waypoints button
- Real-time coordinate display and robot pose tracking
- Integrated into new NAVIGATION tab group
- Uses /navigate_to_pose service for goal publishing
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Adds sliding-window drift detector that compares cumulative path lengths
of visual odom and wheel odom over a configurable window (default 10 s).
Drift = |vo_path − wheel_path|; flagged when ≥ 0.5 m (configurable).
OdomBuffer handles per-source rolling storage with automatic age eviction.
Publishes Bool on /saltybot/vo_drift_detected and Float32 on
/saltybot/vo_drift_magnitude at 2 Hz. 27/27 pure-Python tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds ambient_sound_node to saltybot_social:
- Accumulates 1 s of PCM-16 audio from /social/speech/audio_raw
- Extracts mel-spectrogram feature vector (energy_db, zcr, mel_centroid,
mel_flatness, low_ratio, high_ratio) using pure numpy (no torch/onnx)
- Priority-cascade classifier: silence → music → speech → crowd → outdoor → alarm
- Publishes label as std_msgs/String on /saltybot/ambient_sound on each buffer fill
- All 11 thresholds exposed as ROS parameters (yaml + launch file)
- numpy-free energy-only fallback for edge environments
- 77/77 tests passing
Closes#252
Add vad_node to saltybot_social: subscribes to /social/speech/audio_raw
(UInt8MultiArray PCM-16), computes RMS energy (dBFS) and zero-crossing
rate per chunk, applies onset/offset hysteresis (VadStateMachine), and
publishes /social/speech/is_speaking (Bool) and /social/speech/energy
(Float32 linear RMS). All thresholds configurable via ROS params:
rms_threshold_db=-35.0, zcr_min=0.01, zcr_max=0.40, onset_frames=2,
offset_frames=8, audio_topic. 69/69 tests passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements a chat-style conversation viewer that subscribes to
/social/conversation_text and displays user speech (STT) and robot
responses (TTS) with timestamps and speaker labels. Includes auto-scroll
to latest message, manual scroll detection, and message history limiting.
- New component: ConversationHistory.jsx (chat-style message bubbles)
- User messages in blue, robot responses in green
- Auto-scrolling with manual scroll detection toggle
- Timestamp formatting (HH:MM:SS)
- Message history limiting (max 100 messages)
- Clear history button
- Integrated into SOCIAL tab group as "History" tab
- Subscribes to /social/conversation_text topic
(saltybot_social_msgs/ConversationMessage)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Adds gap-based Euclidean distance clustering of /scan LaserScan points.
Each cluster is published as a labelled semi-transparent CUBE + TEXT marker
in /saltybot/lidar_clusters (MarkerArray), sorted nearest-first. Stale
markers from shrinking cluster counts are explicitly deleted each cycle.
22/22 pure-Python tests pass (no ROS2 required).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add real-time audio level visualization with VU-style meter:
- Responsive VU bar with color gradient (silent to clipping)
- Peak hold indicator with exponential decay
- Speech activity detection from /social/speech/is_speaking
- Color-coded audio levels with visual feedback
- Grid markers for level reference (25%, 50%, 75%)
- Comprehensive audio statistics (average, max, peak count)
Features:
- Dynamic color coding: Gray (silent) → Red (clipping)
- Level thresholds: Silent, Low, Moderate, Good, Loud, Clipping
- Peak hold with 1-second hold time + 5% decay per 50ms
- Speech activity indicator with pulsing animation
- 100-sample rolling average for statistics
- Real-time metric updates
Visual Elements:
- Main VU bar with smooth fill animation
- Separate peak hold display with glow effect
- Color reference legend (all 6 levels)
- Statistics panel (average, max, peak holds)
- Grid-based scale (0-100%)
- Speech status badge (SPEAKING/SILENT)
Integration:
- Added to SOCIAL tab as new "Audio" tab
- Subscribes to /saltybot/audio_level and /social/speech/is_speaking
- Properly formatted topic info footer
- Responsive design matching dashboard theme
Build: ✓ Passing (113 modules, 202.67 KB main bundle)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Adds cv2.QRCodeDetector-based QR reader that subscribes to all four IMX219
CSI camera streams, deduplicates detections with a 2 s per-payload cooldown,
and publishes /saltybot/qr_codes (QRDetectionArray) at 10 Hz. New
QRDetection / QRDetectionArray messages added to saltybot_scene_msgs.
16/16 pure-Python tests pass (no ROS2 required).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add real-time audio level visualization with VU-style meter:
- Responsive VU bar with color gradient (silent to clipping)
- Peak hold indicator with exponential decay
- Speech activity detection from /social/speech/is_speaking
- Color-coded audio levels with visual feedback
- Grid markers for level reference (25%, 50%, 75%)
- Comprehensive audio statistics (average, max, peak count)
Features:
- Dynamic color coding: Gray (silent) → Red (clipping)
- Level thresholds: Silent, Low, Moderate, Good, Loud, Clipping
- Peak hold with 1-second hold time + 5% decay per 50ms
- Speech activity indicator with pulsing animation
- 100-sample rolling average for statistics
- Real-time metric updates
Visual Elements:
- Main VU bar with smooth fill animation
- Separate peak hold display with glow effect
- Color reference legend (all 6 levels)
- Statistics panel (average, max, peak holds)
- Grid-based scale (0-100%)
- Speech status badge (SPEAKING/SILENT)
Integration:
- Added to SOCIAL tab as new "Audio" tab
- Subscribes to /saltybot/audio_level and /social/speech/is_speaking
- Properly formatted topic info footer
- Responsive design matching dashboard theme
Build: ✓ Passing (113 modules, 202.67 KB main bundle)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Adds per-person constant-velocity Kalman filter that smooths raw 68-point
facial landmarks and republishes on /social/faces/landmarks_smooth at input
rate. New FaceLandmarks / FaceLandmarksArray messages added to
saltybot_social_msgs. 21/21 pure-Python tests pass (no ROS2 required).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace basic IMU panel with interactive Three.js 3D pose viewer:
- Real-time robot box model visualization
- Rotation controlled by /saltybot/imu quaternion data
- 30-second position trail from /odom navigation
- Trail visualization with point history
- Reset button to clear trail history
- Grid ground plane and directional indicator arrow
- Interactive 3D scene with proper lighting and shadows
Features:
- Three.js scene with dynamic box model (0.3x0.3x0.5m)
- Forward direction indicator (amber cone)
- Ambient and directional lighting with shadows
- Grid helper for spatial reference
- Trail line geometry with dynamic point updates
- Automatic window resize handling
- Proper cleanup of WebGL resources and subscriptions
Integration:
- Replaces ImuPanel in TELEMETRY tab (IMU view)
- Added to flex layout for proper 3D viewport sizing
- Subscribes to /saltybot/imu and /odom topics
- Updates trail points every 100ms (max 30-second history)
Build: ✓ Passing (112 modules, 196.69 KB main bundle)
Three.js vendor: 473.57 KB
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
saltybot_social_msgs:
- Add PointingTarget.msg: origin (INDEX_MCP), direction (unit vec), target,
range_m, person_id, confidence, coarse_direction, is_active
- Register in CMakeLists.txt
saltybot_social:
- _pointing_ray.py (pure Python, no rclpy): unproject(), sample_depth()
(median with outlier rejection), compute_pointing_ray() — reprojects
INDEX_MCP and INDEX_TIP into 3-D using D435i depth; falls back to image-
plane direction when both depths are equal; gracefully handles one-sided
missing depth
- pointing_node.py: subscribes /social/gestures + synced D435i colour+depth;
re-runs MediaPipe Hands when a 'point' gesture is cached (within
gesture_timeout_s); picks closest hand to gesture anchor; publishes
PointingTarget on /saltybot/pointing_target at 5 Hz
- setup.py: adds pointing_node entry point
- 18/18 unit tests pass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements complete I2C1 driver for TI INA219 power monitoring IC supporting:
- Dual sensors on I2C1 (left motor @ 0x40, right motor @ 0x41)
- Auto-calibration for 5A max current, 0.1Ω shunt resistance
- Current LSB: 153µA, Power LSB: 3060µW (20× current LSB)
- Bus voltage: 0-26V @ 4mV/LSB (13-bit, 4mV resolution)
- Shunt voltage: ±327mV @ 10µV/LSB (signed 16-bit)
- Calibration register computation for arbitrary max current/shunt values
- Efficient single/batch read functions (voltage, current, power)
- Alert threshold configuration for overcurrent protection
- Full test suite: 12 passing unit tests covering calibration, conversions, edge cases
Integration:
- ina219_init() called after i2c1_init() in main startup sequence
- Ready for motor power monitoring and thermal protection logic
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Two files added to saltybot_bringup:
- _scan_height_filter.py: pure-Python helpers (no rclpy) —
filter_scan_by_height() projects each LIDAR ray to world-frame height
using pitch/roll from the IMU and filters ground/ceiling returns;
pitch_roll_from_accel() uses convention-agnostic atan2 formula;
AttitudeEstimator low-pass filters the accelerometer attitude.
- scan_height_filter_node.py: subscribes /scan + /camera/imu, publishes
/scan_filtered (LaserScan) for Nav2 at source rate (up to 20 Hz).
setup.py: adds scan_height_filter entry point.
18/18 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements TIM4 PWM driver for 2-servo camera mount with:
- 50 Hz PWM frequency (standard servo control)
- CH1 (PB6) pan servo, CH2 (PB7) tilt servo
- 0-180° angle range → 500-2500 µs pulse width mapping
- Non-blocking servo_set_angle() for immediate positioning
- servo_sweep() for smooth pan-tilt animation (linear interpolation)
- Independent sweep control per servo (pan and tilt move simultaneously)
- 15 comprehensive unit tests covering all scenarios
Integration:
- servo_init() called at startup after power_mgmt_init()
- servo_tick(now_ms) called every 1ms in main loop
- Ready for camera/gimbal control automation
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Two new packages:
- saltybot_person_reid_msgs: PersonAppearance + PersonAppearanceArray msgs
- saltybot_person_reid: MobileNetV2 torso-crop embedder (128-dim L2-norm)
with 128-bin HSV histogram fallback, cosine-similarity gallery with EMA
identity updates and configurable age-based pruning, ROS2 node publishing
PersonAppearanceArray on /saltybot/person_reid at 5 Hz.
Pure-Python helpers (_embedding_model, _reid_gallery) importable without
rclpy — 18/18 unit tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New saltybot_thermal package with thermal_node: reads all
/sys/class/thermal/thermal_zone* sysfs entries (millidegrees→°C),
publishes /saltybot/thermal JSON at 1 Hz with zones[], max_temp_c,
warn, and throttled flags. Logs ROS2 WARN at ≥75°C, ERROR at ≥85°C.
thermal_root param allows sysfs override for offline testing.
50/50 tests passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements node watchdog ROS2 node that monitors heartbeats from critical
systems and triggers safety fallback when motor driver is lost >2s.
Features:
- Monitor heartbeats from: balance, motor_driver, emergency, docking
- Alert on /saltybot/node_watchdog (JSON) if heartbeat lost >1s
- Safety fallback: zero cmd_vel if motor driver lost >2s
- Republish cmd_vel on /saltybot/cmd_vel_safe with safety checks
- 20Hz monitoring and publishing frequency
- Configurable heartbeat timeout thresholds
Heartbeat Topics:
- /saltybot/balance_heartbeat (std_msgs/UInt32)
- /saltybot/motor_driver_heartbeat (std_msgs/UInt32)
- /saltybot/emergency_heartbeat (std_msgs/UInt32)
- /saltybot/docking_heartbeat (std_msgs/UInt32)
- /cmd_vel (geometry_msgs/Twist)
Published Topics:
- /saltybot/node_watchdog (std_msgs/String) - JSON status
- /saltybot/cmd_vel_safe (geometry_msgs/Twist) - Safety-checked velocity
Package: saltybot_node_watchdog
Entry point: node_watchdog_node
Launch file: node_watchdog.launch.py
Tests: 20+ unit tests covering:
- Heartbeat reception and timeout detection
- Motor driver critical timeout (>2s)
- Safety fallback logic
- cmd_vel zeroing on motor driver loss
- Health status JSON serialization
- Multi-node failure scenarios
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
New MeshPeer.msg (1 Hz DDS heartbeat: robot_id, social_state, active persons,
greeted names) and MeshHandoff.msg (person context transfer on STATE_LEAVING).
mesh_comms_node subscribes to person_states and orchestrator/state, publishes
announce heartbeat, triggers handoff on LEAVING, tracks peers with timeout
cleanup, and propagates mesh-wide greeting deduplication via /social/mesh/greeted.
73/73 tests passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add SpeechTranscript.language (BCP-47), ConversationResponse.language fields
- speech_pipeline_node: whisper_language param (""=auto-detect via Whisper LID);
detected language published in every transcript
- conversation_node: track per-speaker language; inject "[Please respond in X.]"
hint for non-English speakers; propagate language to ConversationResponse.
_LANG_NAMES: 24 BCP-47 codes -> English names. Also adds Issue #161 emotion
context plumbing (co-located in same branch for clean merge)
- tts_node: voice_map_json param (JSON BCP-47->ONNX path); lazy voice loading
per language; playback queue now carries (text, lang) tuples for voice routing
- speech_params.yaml, tts_params.yaml: new language params with docs
- 47/47 tests pass (test_multilang.py)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds STM32F7 STOP-mode power management with <10ms wake latency:
- power_mgmt.c: state machine (ACTIVE→SLEEP_PENDING→SLEEPING→WAKING),
30s idle timeout (PM_IDLE_TIMEOUT_MS), 3s LED fade before STOP,
gate SPI3/I2S3+SPI2+USART6+UART5 on sleep (clock-only, state preserved),
EXTI1(PA1/CRSF)+EXTI7(PB7/JLink)+EXTI4(PC4/IMU) wake sources,
PLL restore after STOP (PLLM=8/N=216/P=2 → 216MHz), uwTick save/restore
- Peripheral gating: I2S3, SPI2(OSD), USART6, UART5 disabled during STOP;
SPI1(IMU), UART4(CRSF), USART1(JLink), I2C1 remain active as wake sources
- Sleep LED: triangle-wave pulse (2s period) on LED1 during SLEEP_PENDING,
software PWM in main loop (1-bit, pm_pwm_phase vs brightness)
- IWDG: fed just before WFI; <10ms wake << 50ms WATCHDOG_TIMEOUT_MS
- JLink: JLINK_CMD_SLEEP=0x09, JLINK_TLM_POWER=0x81 (11-byte power frame
at 1Hz: power_state, est_total_ma, est_audio_ma, est_osd_ma, idle_ms)
- main.c: power_mgmt_init(), activity() on CRSF/JLink/armed, tick() when
disarmed, sleep_req handler, LED PWM, JLINK_TLM_POWER telemetry
- config.h: PM_* constants, PM_CURRENT_*_MA estimates, PM_TLM_HZ
- test_power_mgmt.py: 72 tests passing (state machine, LED, gating,
current estimates, JLink protocol, wake latency, hardware constants)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
i%40==39 fired the first IWDG refresh only after 40ms of calibration.
Combined with ~10ms of main loop overhead before entering calibrate(),
total elapsed since last refresh could exceed the 50ms IWDG window.
Change to i%40==0: first refresh fires at i=0 (<1ms after entry),
subsequent refreshes every 40ms — safely within the 50ms window.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 22:04:27 -05:00
1750 changed files with 184595 additions and 3473 deletions
# Issue #469: Terrain Classification Implementation Plan
## Context
SaltyBot currently has good sensor infrastructure (IMU, cameras, RealSense) and a robust velocity control system with the `VelocityRamp` class. However, it lacks terrain awareness for surface type detection and speed adaptation. This feature will enable:
1. **Surface detection** via IMU vibration analysis and camera texture analysis
2. **Automatic speed adaptation** based on terrain type and roughness
3. **Terrain logging** for mapping and future learning
4. **Improved robot safety** by reducing speed on rough/unstable terrain
## Architecture Overview
The implementation follows the existing ROS2 patterns:
- The existing velocity ramp (`velocity_ramp_node.py`) processes `/cmd_vel_smooth` or `/cmd_vel`
- Optionally, update cmd_vel_bridge to use `/cmd_vel_terrain` if available, else fall back to `/cmd_vel`
- Terrain classification runs independently at 5 Hz (much slower than velocity ramping at 50 Hz)
### 7. **Future CNN Enhancement**
The current implementation uses rule-based classification with IMU FFT and camera edge detection. A future enhancement could add a lightweight CNN for texture classification (e.g., MobileNet) by:
1. Creating a `terrain_classifier_cnn.py` with TensorFlow/ONNX model
2. Replacing decision logic in `terrain_classifier_node.py` with CNN inference
3. Maintaining same message interface
## Implementation Tasks
1. ✅ **Phase 1: Message Definition**
- Create `TerrainState.msg` in saltybot_social_msgs
- Update `CMakeLists.txt`
2. ✅ **Phase 2: Terrain Classifier Node**
- Implement `terrain_classifier_node.py` with IMU FFT analysis
- Implement camera texture analysis
- Decision logic for classification
3. ✅ **Phase 3: Speed Adapter Node**
- Implement `terrain_speed_adapter_node.py`
- Velocity command adaptation
4. ✅ **Phase 4: Terrain Mapper Node**
- Implement `terrain_mapper_node.py` for logging
5. ✅ **Phase 5: Integration**
- Update `full_stack.launch.py` with new nodes
- Update `setup.py` with entry points
- Test integration
## Testing & Verification
**Unit Tests:**
- Test IMU FFT feature extraction with synthetic vibration data
- Test terrain classification decision logic
- Test speed ratio application
- Test CSV logging format
**Integration Tests:**
- Run full stack with simulated IMU/camera data
- Verify terrain messages published at 5 Hz
- Verify cmd_vel_terrain adapts speeds correctly
- Check terrain log file is created and properly formatted
**Manual Testing:**
- Drive robot on different surfaces
- Verify terrain detection changes appropriately
- Verify speed adaptation is smooth (no jerks from ramping)
- Computed from FFT power spectral density in 10-50 Hz band
- Reflects mechanical vibration from surface contact
2. `cnn_texture_features` = 1280-dim feature vector from MobileNetV2
- Pre-trained features capture texture, edge, and surface characteristics
- Reduced to 2-3 principal components via PCA or simple aggregation
3. `accel_magnitude` = RMS of total acceleration (m/s²)
- Helps distinguish stationary (9.81 m/s²) vs. moving
**Classification approach (Version 1):**
- Simple decision tree with IMU-dominant logic + CNN support:
```
if imu_roughness <0.2andaccel_magnitude<9.8:
terrain = PAVEMENT (confidence boosted if CNN agrees)
elif imu_roughness <0.35andcnn_grainy_score<0.4:
terrain = GRASS
elif imu_roughness > 0.45 and cnn_granular_score > 0.5:
terrain = GRAVEL
elif cnn_sand_texture_score > 0.6 and imu_roughness > 0.3:
terrain = SAND
else:
terrain = INDOOR
```
- Confidence: weighted combination of IMU and CNN agreement
- Roughness metric: `0.0 = smooth, 1.0 = very rough` derived from IMU FFT energy ratio
**Speed recommendations:**
- Pavement: 1.0 (full speed)
- Grass: 0.8 (20% slower)
- Gravel: 0.5 (50% slower)
- Sand: 0.4 (60% slower)
- Indoor: 0.7 (30% slower by default)
**Future improvement:** Replace decision tree with trained classifier (Random Forest, SVM, or small Dense net) on labeled terrain dataset once collected.
---
This plan follows SaltyBot's established patterns:
- Pure Python libraries for core logic (_terrain_analysis.py)
- ROS2 node wrappers for integration
- Parameter-based configuration in YAML
- Message-based pub/sub architecture
- Integration with existing velocity control pipeline
The robot can now be armed and operated autonomously from the Jetson without requiring an RC transmitter. The RC receiver (ELRS) is now optional and serves as an override/kill-switch rather than a requirement.
## Arming Sources
### Jetson Autonomous Arming
- Command: `A\n` (single byte 'A' followed by newline)
<<<<<<<HEAD
- Sent via USB CDC to the ESP32 BALANCE firmware
=======
- Sent via USB Serial (CH343) to the ESP32-S3 firmware
>>>>>>> 291dd68 (feat: remove all STM32/Mamba/BlackPill references — ESP32-S3 only)
- Robot arms after ARMING_HOLD_MS (~500ms) safety hold period
- Works even when RC is not connected or not armed
### RC Arming (Optional Override)
- Command: CH5 switch on ELRS transmitter
- When RC is connected and armed, robot can be armed via RC
- RC and Jetson can both request arming independently
## Safety Features
### Maintained from Original Design
1. **Arming Hold Timer** — 500ms hold before motors enable (prevents accidental arming)
2. **Tilt Safety** — Robot must be within ±10° level to arm
3. **IMU Calibration** — Gyro must be calibrated before arming
4. **Remote E-Stop Override** — `safety_remote_estop_active()` blocks all arming
### New for Autonomous Operation
1. **RC Kill Switch** (CH5 OFF when RC connected)
- Triggers emergency stop (motor cutoff) instead of disarm
- Allows Jetson-armed robots to remain armed when RC disconnects
- Maintains safety of kill switch for emergency situations
2. **RC Failsafe**
- If RC signal is lost after being established, robot disarms (500ms timeout)
- Prevents runaway if RC connection drops during flight
- USB-only mode (no RC ever connected) is unaffected
3. **Jetson Timeout** (200ms heartbeat)
- Jetson must send heartbeat (H command) every 500ms
- Prevents autonomous runaway if Jetson crashes/loses connection
- Handled by `jetson_cmd_is_active()` checks
## Command Protocol
<<<<<<<HEAD
### From Jetson to ESP32 BALANCE (USB CDC)
=======
### From Jetson to ESP32-S3 (USB Serial (CH343))
>>>>>>> 291dd68 (feat: remove all STM32/Mamba/BlackPill references — ESP32-S3 only)
```
A — Request arm (triggers safety hold, then motors enable)
D — Request disarm (immediate motor stop)
E — Emergency stop (immediate motor cutoff, latched)
Z — Clear emergency stop latch
H — Heartbeat (refresh timeout timer, every 500ms)
- **Firmware:** ESP-IDF/PlatformIO target; legacy `src/` STM32 HAL archived
- **Comms:** UART 460800 baud inter-board; CANable2 USB→CAN for Orin; CAN 500 kbps to VESCs (L:68 / R:56)
=======
Self-balancing two-wheeled robot using a drone ESP32-S3 BALANCE (ESP32-S3), hoverboard hub motors, and eventually a Jetson Orin Nano Super for AI/SLAM.
## Current Status
- **Hardware:** Assembled — FC, motors, ESC, IMU, battery, RC all on hand
- **Firmware:** Balance PID + hoverboard ESC protocol written, but blocked by USB CDC bug
- **Blocker:** USB CDC TX stops working when peripheral inits (SPI/UART/GPIO) are added alongside USB OTG FS — see `USB_CDC_BUG.md`
- **Firmware:** Balance PID + hoverboard ESC protocol written, but blocked by USB Serial (CH343) bug
- **Blocker:** USB Serial (CH343) TX stops working when peripheral inits (SPI/UART/GPIO) are added alongside USB on ESP32-S3 — see `legacy/stm32/USB_CDC_BUG.md` for historical context
>>>>>>> 291dd68 (feat: remove all STM32/Mamba/BlackPill references — ESP32-S3 only)
---
@ -14,18 +24,30 @@ Self-balancing two-wheeled robot using a drone flight controller (STM32F722), ho
### 1. Embedded Firmware Engineer (Lead)
**Must-have:**
- Deep STM32 HAL experience (F7 series specifically)
<<<<<<<HEAD
- Deep ESP32 (Arduino/ESP-IDF) or STM32 HAL experience
**Why:** The immediate blocker is a USB peripheral conflict. Need someone who's debugged STM32 USB issues before — this is not a software logic bug, it's a hardware peripheral interaction issue.
<<<<<<<HEAD
**Why:** The immediate blocker is a USB peripheral conflict. Need someone who's debugged STM32 USB issues before — ESP32 firmware for the balance loop and I/O needs to be written from scratch.
=======
**Why:** The immediate blocker is a USB peripheral conflict on ESP32-S3. Need someone who's debugged ESP32-S3 USB Serial (CH343) issues before — this is not a software logic bug, it's a hardware peripheral interaction issue.
>>>>>>> 291dd68 (feat: remove all STM32/Mamba/BlackPill references — ESP32-S3 only)
### 2. Control Systems / Robotics Engineer
**Must-have:**
@ -43,7 +65,7 @@ Self-balancing two-wheeled robot using a drone flight controller (STM32F722), ho
### 3. Perception / SLAM Engineer (Phase 2)
**Must-have:**
- Jetson Nano / NVIDIA Jetson platform
- Jetson Orin Nano Super / NVIDIA Jetson platform
- Intel RealSense D435i depth camera
- RPLIDAR integration
- SLAM (ORB-SLAM3, RTAB-Map, or similar)
@ -54,19 +76,23 @@ Self-balancing two-wheeled robot using a drone flight controller (STM32F722), ho
- Obstacle avoidance
- Nav2 stack
**Why:** Phase 2 goal is autonomous navigation. Jetson Nano with RealSense + RPLIDAR for indoor mapping and person following.
**Why:** Phase 2 goal is autonomous navigation. Jetson Orin Nano Super with RealSense + RPLIDAR for indoor mapping and person following.
---
## Hardware Reference
| Component | Details |
|-----------|---------|
| FC | MAMBA F722S (STM32F722RET6, MPU6000) |
<<<<<<<HEAD
| FC | ESP32 BALANCE (ESP32RET6, MPU6000) |
=======
| FC | ESP32-S3 BALANCE (ESP32-S3RET6, QMI8658) |
>>>>>>> 291dd68 (feat: remove all STM32/Mamba/BlackPill references — ESP32-S3 only)
- Likely culprit: **peripheral init (SPI/UART/GPIO) is interfering with USB OTG FS**
## Suspected Root Cause
One of the additional peripheral inits (SPI1 for IMU, USART2 for hoverboard ESC, or GPIO for status LEDs) is likely conflicting with the USB OTG FS peripheral — either a clock conflict, GPIO pin conflict, or interrupt priority issue.
## Hardware
- MAMBA F722S FC (STM32F722RET6)
- Betaflight target: DIAT-MAMBAF722_2022B
- IMU: MPU6000 on SPI1 (PA4/PA5/PA6/PA7)
- USB: OTG FS (PA11/PA12)
- Hoverboard ESC: USART2 (PA2/PA3)
- LEDs: PC14, PC15
- Buzzer: PB2
## Files
- PlatformIO project: `~/Projects/saltylab-firmware/` on mbpm4 (192.168.87.40)
- Working test: was in src/main.c (replaced with balance code)
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.