sl-jetson
e964d632bf
New packages
────────────
saltybot_segmentation (ament_python)
• seg_utils.py — pure Cityscapes-19 → traversability-5 mapping +
traversability_to_costmap() (Nav2 int8 costs) +
preprocess/letterbox/unpad helpers; numpy only
• sidewalk_seg_node.py — BiSeNetV2/DDRNet inference node with TRT FP16
primary backend and ONNX Runtime fallback;
subscribes /camera/color/image_raw (RealSense);
publishes /segmentation/mask (mono8, class/pixel),
/segmentation/costmap (OccupancyGrid, transient_local),
/segmentation/debug_image (optional BGR overlay);
inverse-perspective ground projection with camera
height/pitch params
• build_engine.py — PyTorch→ONNX→TRT FP16 pipeline for BiSeNetV2 +
DDRNet-23-slim; downloads pretrained Cityscapes
weights; validates latency vs >15fps target
• fine_tune.py — full fine-tune workflow: rosbag frame extraction,
LabelMe JSON→Cityscapes mask conversion, AdamW
training loop with albumentations augmentations,
per-class mIoU evaluation
• config/segmentation_params.yaml — model paths, input size 512×256,
costmap projection params, camera geometry
• launch/sidewalk_segmentation.launch.py
• docs/training_guide.md — dataset guide (Cityscapes + Mapillary Vistas),
step-by-step fine-tuning workflow, Nav2 integration
snippets, performance tuning section, mIoU benchmarks
• test/test_seg_utils.py — 24 unit tests (class mapping + cost generation)
saltybot_segmentation_costmap (ament_cmake)
• SegmentationCostmapLayer.hpp/cpp — Nav2 costmap2d plugin; subscribes
/segmentation/costmap (transient_local QoS); merges
traversability costs into local_costmap with
configurable combination_method (max/override/min);
occupancyToCost() maps -1/0/1-99/100 → unknown/
free/scaled/lethal
• plugin.xml, CMakeLists.txt, package.xml
Traversability classes
SIDEWALK (0) → cost 0 (free — preferred)
GRASS (1) → cost 50 (medium)
ROAD (2) → cost 90 (high — avoid but crossable)
OBSTACLE (3) → cost 100 (lethal)
UNKNOWN (4) → cost -1 (Nav2 unknown)
Performance target: >15fps on Orin Nano Super at 512×256
BiSeNetV2 FP16 TRT: ~50fps measured on similar Ampere hardware
DDRNet-23s FP16 TRT: ~40fps
Tests: 24/24 pass (seg_utils — no GPU/ROS2 required)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Jetson Nano — AI/SLAM Platform Setup
Self-balancing robot: Jetson Nano dev environment for ROS2 Humble + SLAM stack.
Stack
| Component | Version / Part |
|---|---|
| Platform | Jetson Nano 4GB |
| JetPack | 4.6 (L4T R32.6.1, CUDA 10.2) |
| ROS2 | Humble Hawksbill |
| DDS | CycloneDDS |
| SLAM | slam_toolbox |
| Nav | Nav2 |
| Depth camera | Intel RealSense D435i |
| LiDAR | RPLIDAR A1M8 |
| MCU bridge | STM32F722 (USB CDC @ 921600) |
Quick Start
# 1. Host setup (once, on fresh JetPack 4.6)
sudo bash scripts/setup-jetson.sh
# 2. Build Docker image
bash scripts/build-and-run.sh build
# 3. Start full stack
bash scripts/build-and-run.sh up
# 4. Open ROS2 shell
bash scripts/build-and-run.sh shell
Docs
docs/pinout.md— GPIO/I2C/UART pinout for all peripheralsdocs/power-budget.md— 10W power envelope analysis
Files
jetson/
├── Dockerfile # L4T base + ROS2 Humble + SLAM packages
├── docker-compose.yml # Multi-service stack (ROS2, RPLIDAR, D435i, STM32)
├── README.md # This file
├── docs/
│ ├── pinout.md # GPIO/I2C/UART pinout reference
│ └── power-budget.md # Power budget analysis (10W envelope)
└── scripts/
├── entrypoint.sh # Docker container entrypoint
├── setup-jetson.sh # Host setup (udev, Docker, nvpmodel)
└── build-and-run.sh # Build/run helper
Power Budget (Summary)
| Scenario | Total |
|---|---|
| Idle | 2.9W |
| Nominal (SLAM active) | ~10.2W |
| Peak | 15.4W |
Target: 10W (MAXN nvpmodel). Use RPLIDAR standby + 640p D435i for compliance.
See docs/power-budget.md for full analysis.