seb/saltylab-firmware
1
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity

feat(social): ambient sound classifier via mel-spectrogram (Issue #252) #258

Merged
sl-jetson merged 1 commits from sl-jetson/issue-252-ambient-sound into main 2026-03-02 14:13:33 -05:00
Conversation 0 Commits 1 Files Changed 5 +835

1 Commits

Author SHA1 Message Date
sl-jetson
3cd9faeed9 feat(social): ambient sound classifier via mel-spectrogram — Issue #252
Some checks failed
social-bot integration tests / Lint (flake8 + pep257) (pull_request) Failing after 2s
Details
social-bot integration tests / Core integration tests (mock sensors, no GPU) (pull_request) Has been skipped
Details
social-bot integration tests / Latency profiling (GPU, Orin) (pull_request) Has been cancelled
Details 
Adds ambient_sound_node to saltybot_social:
- Accumulates 1 s of PCM-16 audio from /social/speech/audio_raw
- Extracts mel-spectrogram feature vector (energy_db, zcr, mel_centroid,
  mel_flatness, low_ratio, high_ratio) using pure numpy (no torch/onnx)
- Priority-cascade classifier: silence → music → speech → crowd → outdoor → alarm
- Publishes label as std_msgs/String on /saltybot/ambient_sound on each buffer fill
- All 11 thresholds exposed as ROS parameters (yaml + launch file)
- numpy-free energy-only fallback for edge environments
- 77/77 tests passing

Closes #252
 2026-03-02 13:22:38 -05:00