sl-jetson
71d6ce610b
Issue #81 — Speech pipeline: - speech_pipeline_node.py: OpenWakeWord "hey_salty" → Silero VAD → faster-whisper STT (Orin GPU, <500ms wake-to-transcript) → ECAPA-TDNN speaker diarization - speech_utils.py: pcm16↔float32, EnergyVad, UtteranceSegmenter (pre-roll, max- duration), cosine speaker identification — all pure Python, no ROS2/GPU needed - Publishes /social/speech/transcript (SpeechTranscript) + /social/speech/vad_state Issue #83 — Conversation engine: - conversation_node.py: llama-cpp-python GGUF (Phi-3-mini Q4_K_M, 20 GPU layers), streaming token output, per-person sliding-window context (4K tokens), summary compression, SOUL.md system prompt, group mode - llm_context.py: PersonContext, ContextStore (JSON persistence), build_llama_prompt (ChatML format), context compression via LLM summarization - Publishes /social/conversation/response (ConversationResponse, partial + final) Issue #85 — Streaming TTS: - tts_node.py: Piper ONNX streaming synthesis, sentence-by-sentence first-chunk streaming (<200ms to first audio), sounddevice USB speaker playback, volume control - tts_utils.py: split_sentences, pcm16_to_wav_bytes, chunk_pcm, apply_volume, strip_ssml Issue #89 — Pipeline orchestrator: - orchestrator_node.py: IDLE→LISTENING→THINKING→SPEAKING state machine, GPU memory watchdog (throttle at <2GB free), rolling latency stats (p50/p95 per stage), VAD watchdog (alert if speech pipeline hangs), /social/orchestrator/state JSON pub - social_bot.launch.py: brings up all 4 nodes with TimerAction delays New messages: SpeechTranscript.msg, VadState.msg, ConversationResponse.msg Config YAMLs: speech_params, conversation_params, tts_params, orchestrator_params Tests: 58 tests (28 speech_utils + 30 llm_context/tts_utils), all passing Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Jetson Nano — AI/SLAM Platform Setup
Self-balancing robot: Jetson Nano dev environment for ROS2 Humble + SLAM stack.
Stack
| Component | Version / Part |
|---|---|
| Platform | Jetson Nano 4GB |
| JetPack | 4.6 (L4T R32.6.1, CUDA 10.2) |
| ROS2 | Humble Hawksbill |
| DDS | CycloneDDS |
| SLAM | slam_toolbox |
| Nav | Nav2 |
| Depth camera | Intel RealSense D435i |
| LiDAR | RPLIDAR A1M8 |
| MCU bridge | STM32F722 (USB CDC @ 921600) |
Quick Start
# 1. Host setup (once, on fresh JetPack 4.6)
sudo bash scripts/setup-jetson.sh
# 2. Build Docker image
bash scripts/build-and-run.sh build
# 3. Start full stack
bash scripts/build-and-run.sh up
# 4. Open ROS2 shell
bash scripts/build-and-run.sh shell
Docs
docs/pinout.md— GPIO/I2C/UART pinout for all peripheralsdocs/power-budget.md— 10W power envelope analysis
Files
jetson/
├── Dockerfile # L4T base + ROS2 Humble + SLAM packages
├── docker-compose.yml # Multi-service stack (ROS2, RPLIDAR, D435i, STM32)
├── README.md # This file
├── docs/
│ ├── pinout.md # GPIO/I2C/UART pinout reference
│ └── power-budget.md # Power budget analysis (10W envelope)
└── scripts/
├── entrypoint.sh # Docker container entrypoint
├── setup-jetson.sh # Host setup (udev, Docker, nvpmodel)
└── build-and-run.sh # Build/run helper
Power Budget (Summary)
| Scenario | Total |
|---|---|
| Idle | 2.9W |
| Nominal (SLAM active) | ~10.2W |
| Peak | 15.4W |
Target: 10W (MAXN nvpmodel). Use RPLIDAR standby + 640p D435i for compliance.
See docs/power-budget.md for full analysis.