seb/saltylab-firmware
1
0
Fork 0
Code Issues Pull Requests 5 Actions Packages Projects Releases Wiki Activity

feat(social): ambient sound classifier via mel-spectrogram (Issue #252) #258

Merged
sl-jetson merged 1 commits from sl-jetson/issue-252-ambient-sound into main 2026-03-02 14:13:33 -05:00
Conversation 0 Commits 1 Files Changed 5 +835
sl-webui commented 2026-03-02 12:55:56 -05:00
Collaborator
Copy Link

Summary

  • Adds ambient_sound_node to saltybot_social
  • Accumulates 1 s of PCM-16 audio from /social/speech/audio_raw
  • Extracts mel-spectrogram feature vector (energy_db, zcr, mel_centroid, mel_flatness, low_ratio, high_ratio) using pure numpy — no torch/onnx dependency
  • Priority-cascade classifier: silence → music → speech → crowd → outdoor → alarm
  • Publishes label as std_msgs/String on /saltybot/ambient_sound on each buffer fill (~1 Hz)
  • All 11 thresholds exposed as ROS parameters (ambient_sound_params.yaml + launch arg overrides)
  • numpy-free energy-only fallback for edge environments
  • 77/77 tests passing

Files

  • saltybot_social/ambient_sound_node.py — mel DSP + classifier + ROS2 node
  • config/ambient_sound_params.yaml — default thresholds
  • launch/ambient_sound.launch.py — launch file with window_s, n_mels, silence_db args
  • test/test_ambient_sound.py — 77 tests
  • setup.py — ambient_sound_node entry point added

Closes #252

## Summary - Adds `ambient_sound_node` to `saltybot_social` - Accumulates 1 s of PCM-16 audio from `/social/speech/audio_raw` - Extracts mel-spectrogram feature vector (energy_db, zcr, mel_centroid, mel_flatness, low_ratio, high_ratio) using pure numpy — no torch/onnx dependency - Priority-cascade classifier: silence → music → speech → crowd → outdoor → alarm - Publishes label as `std_msgs/String` on `/saltybot/ambient_sound` on each buffer fill (~1 Hz) - All 11 thresholds exposed as ROS parameters (`ambient_sound_params.yaml` + launch arg overrides) - numpy-free energy-only fallback for edge environments - 77/77 tests passing ## Files - `saltybot_social/ambient_sound_node.py` — mel DSP + classifier + ROS2 node - `config/ambient_sound_params.yaml` — default thresholds - `launch/ambient_sound.launch.py` — launch file with `window_s`, `n_mels`, `silence_db` args - `test/test_ambient_sound.py` — 77 tests - `setup.py` — `ambient_sound_node` entry point added Closes #252
sl-webui added 1 commit 2026-03-02 12:55:56 -05:00
feat(social): ambient sound classifier via mel-spectrogram — Issue #252
Some checks failed
social-bot integration tests / Lint (flake8 + pep257) (push) Failing after 8s
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (push) Has been skipped
Details
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 8s
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
Details
social-bot integration tests / Latency profiling (GPU, Orin) (push) Has been cancelled
Details
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Details
b2d76b434b 
Adds ambient_sound_node to saltybot_social:
- Accumulates 1 s of PCM-16 audio from /social/speech/audio_raw
- Extracts mel-spectrogram feature vector (energy_db, zcr, mel_centroid,
  mel_flatness, low_ratio, high_ratio) using pure numpy (no torch/onnx)
- Priority-cascade classifier: silence → music → speech → crowd → outdoor → alarm
- Publishes label as std_msgs/String on /saltybot/ambient_sound on each buffer fill
- All 11 thresholds exposed as ROS parameters (yaml + launch file)
- numpy-free energy-only fallback for edge environments
- 77/77 tests passing

Closes #252
sl-webui force-pushed sl-jetson/issue-252-ambient-sound from b2d76b434b to 3cd9faeed9 2026-03-02 13:22:45 -05:00 Compare
sl-jetson merged commit 50636de5a9 into main 2026-03-02 14:13:33 -05:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
shaytac sl-controls sl-firmware sl-jetson sl-mechanical sl-perception sl-uwb sl-webui seb
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: seb/saltylab-firmware#258
No description provided.
Delete Branch "sl-jetson/issue-252-ambient-sound"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?