seb/saltylab-firmware
1
0
Fork 0
Code Issues Pull Requests 5 Actions Packages Projects Releases Wiki Activity

feat(social): voice command NLU — 30+ intents with confirmation flow (Issue #137) #148

Merged
sl-jetson merged 1 commits from sl-jetson/issue-137-voice-commands into main 2026-03-02 09:51:14 -05:00
Conversation 0 Commits 1 Files Changed 8 +1722
sl-jetson commented 2026-03-02 09:38:38 -05:00
Collaborator
Copy Link

Summary

  • VoiceCommand.msg: new ROS2 message — intent, entities[] (key=value), confidence, requires_confirmation, confirmation_token
  • voice_command_parser.py: pure-Python regex NLU, zero ML/ROS2 deps, < 1 ms/call
  • voice_command_node.py: subscribes /social/speech/transcript, publishes /social/voice_command, handles confirmation flow

Intent taxonomy (30+ named commands)

Namespace Intents
nav.* go_to, go_home, follow_me, stop, wait, come_here, patrol, set_mode (6 sub-modes), teach_route, stop_teaching, replay_route
social.* remember_me, forget_me ⚠️, whats_my_name, tell_joke
system.* battery_status, map_status, shutdown ⚠️, restart ⚠️, volume_up, volume_down, volume_set
config.* personality, sass_level, follow_mode
confirm.* yes, no
— fallback → LLM conversation engine

⚠️ = requires_confirmation (10 s timeout, UUID token for matching)

Confirmation flow

User: "shut down"
 → VoiceCommand(intent="system.shutdown", requires_confirmation=True, token=UUID)
 → TTS: "Are you sure?"
User: "yes"
 → VoiceCommand(intent="system.shutdown", requires_confirmation=False, token=same UUID)

Files

File Lines
saltybot_social_msgs/msg/VoiceCommand.msg 24
voice_command_parser.py 339
voice_command_node.py 193
test_voice_command_parser.py 459
config/voice_command_params.yaml 14
launch/voice_command.launch.py 43

Test plan

  • python -m pytest test/test_voice_command_parser.py -v → 191/191 pass (no ROS2 runtime)
  • colcon build --packages-select saltybot_social_msgs saltybot_social
  • ros2 run saltybot_social voice_command_node starts without errors
  • Echo test: publish fake SpeechTranscript, verify /social/voice_command output

Closes #137

🤖 Generated with Claude Code

## Summary - **`VoiceCommand.msg`**: new ROS2 message — `intent`, `entities[]` (key=value), `confidence`, `requires_confirmation`, `confirmation_token` - **`voice_command_parser.py`**: pure-Python regex NLU, zero ML/ROS2 deps, < 1 ms/call - **`voice_command_node.py`**: subscribes `/social/speech/transcript`, publishes `/social/voice_command`, handles confirmation flow ## Intent taxonomy (30+ named commands) | Namespace | Intents | |-----------|----------| | `nav.*` | `go_to`, `go_home`, `follow_me`, `stop`, `wait`, `come_here`, `patrol`, `set_mode` (6 sub-modes), `teach_route`, `stop_teaching`, `replay_route` | | `social.*` | `remember_me`, `forget_me` ⚠️, `whats_my_name`, `tell_joke` | | `system.*` | `battery_status`, `map_status`, `shutdown` ⚠️, `restart` ⚠️, `volume_up`, `volume_down`, `volume_set` | | `config.*` | `personality`, `sass_level`, `follow_mode` | | `confirm.*` | `yes`, `no` | | — | `fallback` → LLM conversation engine | ⚠️ = requires_confirmation (10 s timeout, UUID token for matching) ## Confirmation flow ``` User: "shut down" → VoiceCommand(intent="system.shutdown", requires_confirmation=True, token=UUID) → TTS: "Are you sure?" User: "yes" → VoiceCommand(intent="system.shutdown", requires_confirmation=False, token=same UUID) ``` ## Files | File | Lines | |------|-------| | `saltybot_social_msgs/msg/VoiceCommand.msg` | 24 | | `voice_command_parser.py` | 339 | | `voice_command_node.py` | 193 | | `test_voice_command_parser.py` | 459 | | `config/voice_command_params.yaml` | 14 | | `launch/voice_command.launch.py` | 43 | ## Test plan - [x] `python -m pytest test/test_voice_command_parser.py -v` → **191/191 pass** (no ROS2 runtime) - [ ] `colcon build --packages-select saltybot_social_msgs saltybot_social` - [ ] `ros2 run saltybot_social voice_command_node` starts without errors - [ ] Echo test: publish fake SpeechTranscript, verify `/social/voice_command` output Closes #137 🤖 Generated with [Claude Code](https://claude.com/claude-code)
sl-jetson added 1 commit 2026-03-02 09:38:39 -05:00
feat(social): voice command NLU — 30+ intents, confirmation flow (Issue #137)
Some checks failed
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 4s
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
Details
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 3s
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
Details
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Details
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
Details
fabb988e72 
## New files
- saltybot_social_msgs/msg/VoiceCommand.msg
    intent + entities[] + confidence + confirmation_token + requires_confirmation
- saltybot_social/voice_command_parser.py
    Pure-Python regex NLU, zero ROS2/ML deps, < 1 ms/call
    30+ named intents across nav.*, social.*, system.*, config.*, confirm.*
    Entity extraction: location, name, mode, level, route name
    Dangerous-command flag: system.shutdown, system.restart, social.forget_me
- saltybot_social/voice_command_node.py
    Subscribes /social/speech/transcript, publishes /social/voice_command
    Confirmation flow with UUID token + 10 s timeout
    Below-threshold → intent=fallback → LLM conversation engine
- saltybot_social/test/test_voice_command_parser.py
    191 unit tests (all pass), no ROS2 runtime required
- saltybot_social/config/voice_command_params.yaml
- saltybot_social/launch/voice_command.launch.py

## Intent taxonomy
nav:    go_to, go_home, follow_me, stop, wait, come_here, patrol,
        set_mode (shadow/lead/side/orbit/loose/tight),
        teach_route, stop_teaching, replay_route
social: remember_me, forget_me [CONFIRM], whats_my_name, tell_joke
system: battery_status, map_status, shutdown [CONFIRM], restart [CONFIRM],
        volume_up, volume_down, volume_set
config: personality, sass_level, follow_mode

## Updated
- saltybot_social_msgs/CMakeLists.txt: register VoiceCommand.msg
- saltybot_social/setup.py: add voice_command_node entry point

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
sl-jetson merged commit 4cda5d0f19 into main 2026-03-02 09:51:13 -05:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
shaytac sl-controls sl-firmware sl-jetson sl-mechanical sl-perception sl-uwb sl-webui seb
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: seb/saltylab-firmware#148
No description provided.
Delete Branch "sl-jetson/issue-137-voice-commands"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?