diff --git a/projects/saltybot/SLAM-SETUP-PLAN.md b/projects/saltybot/SLAM-SETUP-PLAN.md new file mode 100644 index 0000000..e8dd035 --- /dev/null +++ b/projects/saltybot/SLAM-SETUP-PLAN.md @@ -0,0 +1,264 @@ +# bd-wax: Jetson Nano + RealSense D435i + RPLIDAR SLAM Setup — Technical Plan + +**Bead:** bd-wax +**Phase:** 2 (lower priority than Phase 1 balance) +**Owner:** sl-perception +**Hardware:** Jetson Nano 4GB + Intel RealSense D435i + RPLIDAR A1M8 +**Goal:** Indoor SLAM, mapping, and autonomous navigation (person-following) + +--- + +## Hardware Summary + +| Component | Specs | +|-----------|-------| +| Jetson Nano 4GB | Quad-core ARM Cortex-A57 @ 1.43GHz, 128-core Maxwell GPU, 4GB LPDDR4 | +| RealSense D435i | Stereo depth (0.1–10m), 848×480 @ 90fps, BMI055 IMU (accel+gyro) | +| RPLIDAR A1M8 | 360° 2D LIDAR, 12m range, 8000 samples/s, ~5.5Hz scan rate | + +--- + +## 1. OS & ROS2 Environment + +### Recommended: Docker on JetPack 4.6 (Ubuntu 18.04 / L4T 32.x) + +Jetson Nano ships with JetPack 4.6 (Ubuntu 18.04). Native ROS2 Humble requires Ubuntu 22.04, so **Docker is the correct approach**: + +- NVIDIA provides official ROS2 containers for Jetson via [dusty-nv/jetson-containers](https://github.com/dusty-nv/jetson-containers) +- Container: `dustynv/ros:humble-ros-base-l4t-r32.7.1` (arm64, CUDA 10.2) +- Compose file pins container, mounts device nodes (`/dev/video*`, `/dev/ttyUSB*`), and handles GPU access + +**Alternative: JetPack 5.x (Ubuntu 20.04)** allows native ROS2 Foxy but requires flashing newer JetPack — only worth it if we need direct hardware access outside Docker. + +**Decision:** Use JetPack 4.6 + Docker (Humble). Fastest path, avoids full OS reflash, proven for Nano. + +--- + +## 2. SLAM Stack Recommendation + +### Recommendation: RTAB-Map (primary) with ORB-SLAM3 as fallback + +#### Why RTAB-Map + +| Criterion | ORB-SLAM3 | RTAB-Map | +|-----------|-----------|----------| +| Sensor fusion (D435i + RPLIDAR) | D435i only | D435i + RPLIDAR natively | +| Output map type | Sparse 3D point cloud | Dense 2D occupancy + 3D point cloud | +| Nav2 compatibility | Needs wrapper | Direct occupancy map output | +| ATE RMSE accuracy | 0.009m | 0.019m | +| Nano resource usage | Lower | Higher (needs tuning) | +| Loop closure robustness | Good | Excellent | + +RTAB-Map's `subscribe_scan` mode uses the RPLIDAR A1M8 as the primary 2D odometry front-end (fast, low-CPU) with the D435i providing depth for loop closure and 3D reconstruction. This hybrid approach is well-documented for Nano deployments. + +**Note:** Cannot simultaneously use `subscribe_scan` (2D LIDAR) and `subscribe_scan_cloud` (3D point cloud) in RTAB-Map — use 2D mode. + +#### ORB-SLAM3 Role + +Keep ORB-SLAM3 as a lightweight visual odometry alternative if RTAB-Map proves too heavy for real-time operation. Can feed its odometry output into RTAB-Map's map management node. + +--- + +## 3. Software Architecture + +``` +┌─────────────────────────────────────────────────────┐ +│ Jetson Nano (Docker) │ +│ │ +│ ┌───────────────┐ ┌──────────────────────────┐ │ +│ │ realsense2_ │ │ rplidar_ros2 │ │ +│ │ camera node │ │ /scan (LaserScan) │ │ +│ │ │ │ 5.5Hz, 360°, 12m range │ │ +│ │ /depth/image │ └──────────┬───────────────┘ │ +│ │ /color/image │ │ │ +│ │ /imu │ │ │ +│ └──────┬────────┘ │ │ +│ │ │ │ +│ └──────────┬─────────────┘ │ +│ ▼ │ +│ ┌──────────────────┐ │ +│ │ rtabmap_ros │ │ +│ │ │ │ +│ │ subscribe_scan=true (RPLIDAR) │ +│ │ subscribe_rgbd=true (D435i depth) │ +│ │ │ │ +│ │ → /map (OccupancyGrid) │ +│ │ → /rtabmap/odom │ +│ │ → /rtabmap/cloud_map (3D) │ +│ └──────────┬────────┘ │ +│ │ │ +│ ▼ │ +│ ┌──────────────────┐ │ +│ │ nav2 stack │ (Phase 2b) │ +│ │ (reduced freq) │ │ +│ │ 5-10Hz costmap │ │ +│ └──────────────────┘ │ +└─────────────────────────────────────────────────────┘ + │ │ + USB3 (D435i) USB2 (RPLIDAR) +``` + +### ROS2 Node Graph + +| Node | Package | Key Topics | +|------|---------|------------| +| `camera/realsense2_camera_node` | `realsense2_camera` | `/camera/depth/image_rect_raw`, `/camera/color/image_raw`, `/camera/imu` | +| `rplidar_node` | `rplidar_ros` | `/scan` | +| `rtabmap` | `rtabmap_ros` | `/map`, `/rtabmap/odom`, `/rtabmap/cloud_map` | +| `robot_state_publisher` | `robot_state_publisher` | `/tf` (static transforms) | + +--- + +## 4. Implementation Phases + +### Phase 2a — SLAM Bring-up (this bead, bd-wax) + +**Deliverables:** +- [ ] Docker Compose file (`docker/slam/docker-compose.yml`) +- [ ] ROS2 Humble container with RTAB-Map + RealSense + RPLIDAR packages +- [ ] Launch file: `launch/slam.launch.py` (RTAB-Map + both sensor nodes) +- [ ] URDF/static TF: D435i and RPLIDAR positions on robot frame +- [ ] `README.md` for Jetson setup instructions + +**Out of scope (Phase 2b):** +- Nav2 autonomous navigation +- Person-following +- Integration with STM32 motor commands (needs Phase 1 balance complete first) + +### Phase 2b — Nav2 Integration (separate bead) + +- Nav2 stack with pre-built occupancy maps +- 5–10Hz costmap updates (reduced from desktop default 20Hz) +- Velocity command bridge: Nav2 `/cmd_vel` → STM32 via serial/ROS2 bridge +- Person detection (YOLOv5 on Nano, or TensorRT-optimized) + +--- + +## 5. Key Configuration Parameters + +### RTAB-Map Tuning for Jetson Nano + +```yaml +# rtabmap_ros params — power/performance tradeoffs for Nano +rtabmap: + Rtabmap/DetectionRate: "2" # Process keyframes at 2Hz (not every frame) + Rtabmap/TimeThr: "700" # Max processing time 700ms per iteration + Kp/MaxFeatures: "400" # Reduce keypoints from default 1000 + Vis/MaxFeatures: "400" + RGBD/LinearUpdate: "0.1" # Only update map if moved 10cm + RGBD/AngularUpdate: "0.09" # Or rotated ~5 degrees + Mem/STMSize: "30" # Short-term memory limit (saves RAM) + Grid/3D: "false" # Use 2D occupancy only (lighter) + Grid/RangeMin: "0.5" # Ignore points closer than 0.5m + Grid/RangeMax: "5.0" # Limit to 5m (A1M8 is reliable to ~8m) +``` + +### RealSense D435i Launch Params + +```yaml +# realsense2_camera — reduce load on Nano +depth_module.profile: "640x480x15" # 15fps depth (not 90fps) +rgb_camera.profile: "640x480x15" # 15fps color +enable_gyro: true +enable_accel: true +unite_imu_method: 2 # Publish unified /camera/imu topic +align_depth.enable: true +``` + +### RPLIDAR + +```yaml +serial_port: "/dev/ttyUSB0" +serial_baudrate: 115200 +scan_mode: "Standard" # A1M8 only supports Standard mode +frame_id: "laser_frame" +``` + +--- + +## 6. Static TF / URDF Robot Frame + +Robot coordinate frame (`base_link`) transforms needed: + +``` +base_link +├── laser_frame (RPLIDAR A1M8 — top center of robot) +├── camera_link (RealSense D435i — front, ~camera_height above base) +│ ├── camera_depth_frame +│ └── camera_imu_frame +└── imu_link (STM32 IMU — FC board location) +``` + +Transforms will be defined in `urdf/saltybot.urdf.xacro` with measured offsets once hardware is physically mounted. + +--- + +## 7. Power Budget Concern + +Jetson Nano 4GB maximum draw: **10W** (5V/2A barrel jack mode) + +| Component | Estimated Draw | +|-----------|---------------| +| Jetson Nano (under SLAM load) | 7–8W | +| RealSense D435i (USB3) | 1.5W | +| RPLIDAR A1M8 (USB) | 0.5W | +| **Total** | **~10W** | + +**Risk:** Marginal power budget. Recommend: +1. Use 5V/4A supply (requires Jetson J48 header, not USB-C) +2. Disable Jetson display output (`sudo systemctl disable gdm3`) +3. Use `nvpmodel -m 1` (5W mode) during mapping-only tasks +4. External powered USB hub for peripherals + +--- + +## 8. Milestone Checklist + +- [ ] Flash JetPack 4.6 on Nano (or verify existing install) +- [ ] Install Docker + NVIDIA Container Runtime on Nano +- [ ] Pull `dustynv/ros:humble-ros-base-l4t-r32.7.1` container +- [ ] Verify D435i recognized: `realsense-viewer` or `rs-enumerate-devices` +- [ ] Verify RPLIDAR port: `ls /dev/ttyUSB*` +- [ ] Build/pull `realsense2_camera` ROS2 package in container +- [ ] Build/pull `rplidar_ros` ROS2 package in container +- [ ] Build/pull `rtabmap_ros` ROS2 package in container +- [ ] Test individual sensor topics with `ros2 topic echo` +- [ ] Run SLAM launch file — verify `/map` published +- [ ] Record rosbag for offline tuning +- [ ] Document measured TF offsets (physical mount positions) +- [ ] Write Phase 2b bead for Nav2 integration + +--- + +## 9. Repo Structure (to be created) + +``` +saltylab-firmware/ +└── projects/ + └── saltybot/ + ├── SLAM-SETUP-PLAN.md ← this file + ├── SALTYLAB.md ← main design doc (TBD) + └── slam/ + ├── docker/ + │ ├── docker-compose.yml + │ └── Dockerfile.slam + ├── launch/ + │ └── slam.launch.py + ├── config/ + │ ├── rtabmap_params.yaml + │ └── realsense_params.yaml + └── urdf/ + └── saltybot.urdf.xacro +``` + +--- + +## 10. References + +- [dusty-nv/jetson-containers](https://github.com/dusty-nv/jetson-containers) — official NVIDIA ROS2 Docker containers for Jetson +- [rtabmap_ros](https://github.com/introlab/rtabmap_ros) — RTAB-Map ROS2 wrapper +- [realsense-ros](https://github.com/IntelRealSense/realsense-ros) — Intel RealSense ROS2 wrapper +- [rplidar_ros](https://github.com/Slamtec/rplidar_ros) — Slamtec RPLIDAR ROS2 package +- [reedjacobp/JetsonBot](https://github.com/reedjacobp/JetsonBot) — Nano + D435i + RPLIDAR reference implementation +- [RTAB-Map D435 + RPLidar discussion](https://answers.ros.org/question/367019/using-2-d435-cameras-and-an-rplidar-with-rtabmap/) +- [Comparative SLAM evaluation (2024)](https://arxiv.org/html/2401.02816v1)