sl-perception
00d62a621e
Scopes Phase 2 perception/SLAM integration: - RTAB-Map (primary) + ORB-SLAM3 (fallback) recommendation with rationale - Docker/Humble strategy on JetPack 4.6 (avoids OS reflash) - ROS2 node graph, static TF frame, sensor params for Nano power budget - Phased milestones: 2a SLAM bring-up, 2b Nav2 integration (separate bead) - Power budget analysis and mitigation steps for 10W Nano constraint Closes: bd-wax (scoping only — implementation tracking in Phase 2a checklist) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
265 lines
11 KiB
Markdown
265 lines
11 KiB
Markdown
# bd-wax: Jetson Nano + RealSense D435i + RPLIDAR SLAM Setup — Technical Plan
|
||
|
||
**Bead:** bd-wax
|
||
**Phase:** 2 (lower priority than Phase 1 balance)
|
||
**Owner:** sl-perception
|
||
**Hardware:** Jetson Nano 4GB + Intel RealSense D435i + RPLIDAR A1M8
|
||
**Goal:** Indoor SLAM, mapping, and autonomous navigation (person-following)
|
||
|
||
---
|
||
|
||
## Hardware Summary
|
||
|
||
| Component | Specs |
|
||
|-----------|-------|
|
||
| Jetson Nano 4GB | Quad-core ARM Cortex-A57 @ 1.43GHz, 128-core Maxwell GPU, 4GB LPDDR4 |
|
||
| RealSense D435i | Stereo depth (0.1–10m), 848×480 @ 90fps, BMI055 IMU (accel+gyro) |
|
||
| RPLIDAR A1M8 | 360° 2D LIDAR, 12m range, 8000 samples/s, ~5.5Hz scan rate |
|
||
|
||
---
|
||
|
||
## 1. OS & ROS2 Environment
|
||
|
||
### Recommended: Docker on JetPack 4.6 (Ubuntu 18.04 / L4T 32.x)
|
||
|
||
Jetson Nano ships with JetPack 4.6 (Ubuntu 18.04). Native ROS2 Humble requires Ubuntu 22.04, so **Docker is the correct approach**:
|
||
|
||
- NVIDIA provides official ROS2 containers for Jetson via [dusty-nv/jetson-containers](https://github.com/dusty-nv/jetson-containers)
|
||
- Container: `dustynv/ros:humble-ros-base-l4t-r32.7.1` (arm64, CUDA 10.2)
|
||
- Compose file pins container, mounts device nodes (`/dev/video*`, `/dev/ttyUSB*`), and handles GPU access
|
||
|
||
**Alternative: JetPack 5.x (Ubuntu 20.04)** allows native ROS2 Foxy but requires flashing newer JetPack — only worth it if we need direct hardware access outside Docker.
|
||
|
||
**Decision:** Use JetPack 4.6 + Docker (Humble). Fastest path, avoids full OS reflash, proven for Nano.
|
||
|
||
---
|
||
|
||
## 2. SLAM Stack Recommendation
|
||
|
||
### Recommendation: RTAB-Map (primary) with ORB-SLAM3 as fallback
|
||
|
||
#### Why RTAB-Map
|
||
|
||
| Criterion | ORB-SLAM3 | RTAB-Map |
|
||
|-----------|-----------|----------|
|
||
| Sensor fusion (D435i + RPLIDAR) | D435i only | D435i + RPLIDAR natively |
|
||
| Output map type | Sparse 3D point cloud | Dense 2D occupancy + 3D point cloud |
|
||
| Nav2 compatibility | Needs wrapper | Direct occupancy map output |
|
||
| ATE RMSE accuracy | 0.009m | 0.019m |
|
||
| Nano resource usage | Lower | Higher (needs tuning) |
|
||
| Loop closure robustness | Good | Excellent |
|
||
|
||
RTAB-Map's `subscribe_scan` mode uses the RPLIDAR A1M8 as the primary 2D odometry front-end (fast, low-CPU) with the D435i providing depth for loop closure and 3D reconstruction. This hybrid approach is well-documented for Nano deployments.
|
||
|
||
**Note:** Cannot simultaneously use `subscribe_scan` (2D LIDAR) and `subscribe_scan_cloud` (3D point cloud) in RTAB-Map — use 2D mode.
|
||
|
||
#### ORB-SLAM3 Role
|
||
|
||
Keep ORB-SLAM3 as a lightweight visual odometry alternative if RTAB-Map proves too heavy for real-time operation. Can feed its odometry output into RTAB-Map's map management node.
|
||
|
||
---
|
||
|
||
## 3. Software Architecture
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────┐
|
||
│ Jetson Nano (Docker) │
|
||
│ │
|
||
│ ┌───────────────┐ ┌──────────────────────────┐ │
|
||
│ │ realsense2_ │ │ rplidar_ros2 │ │
|
||
│ │ camera node │ │ /scan (LaserScan) │ │
|
||
│ │ │ │ 5.5Hz, 360°, 12m range │ │
|
||
│ │ /depth/image │ └──────────┬───────────────┘ │
|
||
│ │ /color/image │ │ │
|
||
│ │ /imu │ │ │
|
||
│ └──────┬────────┘ │ │
|
||
│ │ │ │
|
||
│ └──────────┬─────────────┘ │
|
||
│ ▼ │
|
||
│ ┌──────────────────┐ │
|
||
│ │ rtabmap_ros │ │
|
||
│ │ │ │
|
||
│ │ subscribe_scan=true (RPLIDAR) │
|
||
│ │ subscribe_rgbd=true (D435i depth) │
|
||
│ │ │ │
|
||
│ │ → /map (OccupancyGrid) │
|
||
│ │ → /rtabmap/odom │
|
||
│ │ → /rtabmap/cloud_map (3D) │
|
||
│ └──────────┬────────┘ │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ ┌──────────────────┐ │
|
||
│ │ nav2 stack │ (Phase 2b) │
|
||
│ │ (reduced freq) │ │
|
||
│ │ 5-10Hz costmap │ │
|
||
│ └──────────────────┘ │
|
||
└─────────────────────────────────────────────────────┘
|
||
│ │
|
||
USB3 (D435i) USB2 (RPLIDAR)
|
||
```
|
||
|
||
### ROS2 Node Graph
|
||
|
||
| Node | Package | Key Topics |
|
||
|------|---------|------------|
|
||
| `camera/realsense2_camera_node` | `realsense2_camera` | `/camera/depth/image_rect_raw`, `/camera/color/image_raw`, `/camera/imu` |
|
||
| `rplidar_node` | `rplidar_ros` | `/scan` |
|
||
| `rtabmap` | `rtabmap_ros` | `/map`, `/rtabmap/odom`, `/rtabmap/cloud_map` |
|
||
| `robot_state_publisher` | `robot_state_publisher` | `/tf` (static transforms) |
|
||
|
||
---
|
||
|
||
## 4. Implementation Phases
|
||
|
||
### Phase 2a — SLAM Bring-up (this bead, bd-wax)
|
||
|
||
**Deliverables:**
|
||
- [ ] Docker Compose file (`docker/slam/docker-compose.yml`)
|
||
- [ ] ROS2 Humble container with RTAB-Map + RealSense + RPLIDAR packages
|
||
- [ ] Launch file: `launch/slam.launch.py` (RTAB-Map + both sensor nodes)
|
||
- [ ] URDF/static TF: D435i and RPLIDAR positions on robot frame
|
||
- [ ] `README.md` for Jetson setup instructions
|
||
|
||
**Out of scope (Phase 2b):**
|
||
- Nav2 autonomous navigation
|
||
- Person-following
|
||
- Integration with STM32 motor commands (needs Phase 1 balance complete first)
|
||
|
||
### Phase 2b — Nav2 Integration (separate bead)
|
||
|
||
- Nav2 stack with pre-built occupancy maps
|
||
- 5–10Hz costmap updates (reduced from desktop default 20Hz)
|
||
- Velocity command bridge: Nav2 `/cmd_vel` → STM32 via serial/ROS2 bridge
|
||
- Person detection (YOLOv5 on Nano, or TensorRT-optimized)
|
||
|
||
---
|
||
|
||
## 5. Key Configuration Parameters
|
||
|
||
### RTAB-Map Tuning for Jetson Nano
|
||
|
||
```yaml
|
||
# rtabmap_ros params — power/performance tradeoffs for Nano
|
||
rtabmap:
|
||
Rtabmap/DetectionRate: "2" # Process keyframes at 2Hz (not every frame)
|
||
Rtabmap/TimeThr: "700" # Max processing time 700ms per iteration
|
||
Kp/MaxFeatures: "400" # Reduce keypoints from default 1000
|
||
Vis/MaxFeatures: "400"
|
||
RGBD/LinearUpdate: "0.1" # Only update map if moved 10cm
|
||
RGBD/AngularUpdate: "0.09" # Or rotated ~5 degrees
|
||
Mem/STMSize: "30" # Short-term memory limit (saves RAM)
|
||
Grid/3D: "false" # Use 2D occupancy only (lighter)
|
||
Grid/RangeMin: "0.5" # Ignore points closer than 0.5m
|
||
Grid/RangeMax: "5.0" # Limit to 5m (A1M8 is reliable to ~8m)
|
||
```
|
||
|
||
### RealSense D435i Launch Params
|
||
|
||
```yaml
|
||
# realsense2_camera — reduce load on Nano
|
||
depth_module.profile: "640x480x15" # 15fps depth (not 90fps)
|
||
rgb_camera.profile: "640x480x15" # 15fps color
|
||
enable_gyro: true
|
||
enable_accel: true
|
||
unite_imu_method: 2 # Publish unified /camera/imu topic
|
||
align_depth.enable: true
|
||
```
|
||
|
||
### RPLIDAR
|
||
|
||
```yaml
|
||
serial_port: "/dev/ttyUSB0"
|
||
serial_baudrate: 115200
|
||
scan_mode: "Standard" # A1M8 only supports Standard mode
|
||
frame_id: "laser_frame"
|
||
```
|
||
|
||
---
|
||
|
||
## 6. Static TF / URDF Robot Frame
|
||
|
||
Robot coordinate frame (`base_link`) transforms needed:
|
||
|
||
```
|
||
base_link
|
||
├── laser_frame (RPLIDAR A1M8 — top center of robot)
|
||
├── camera_link (RealSense D435i — front, ~camera_height above base)
|
||
│ ├── camera_depth_frame
|
||
│ └── camera_imu_frame
|
||
└── imu_link (STM32 IMU — FC board location)
|
||
```
|
||
|
||
Transforms will be defined in `urdf/saltybot.urdf.xacro` with measured offsets once hardware is physically mounted.
|
||
|
||
---
|
||
|
||
## 7. Power Budget Concern
|
||
|
||
Jetson Nano 4GB maximum draw: **10W** (5V/2A barrel jack mode)
|
||
|
||
| Component | Estimated Draw |
|
||
|-----------|---------------|
|
||
| Jetson Nano (under SLAM load) | 7–8W |
|
||
| RealSense D435i (USB3) | 1.5W |
|
||
| RPLIDAR A1M8 (USB) | 0.5W |
|
||
| **Total** | **~10W** |
|
||
|
||
**Risk:** Marginal power budget. Recommend:
|
||
1. Use 5V/4A supply (requires Jetson J48 header, not USB-C)
|
||
2. Disable Jetson display output (`sudo systemctl disable gdm3`)
|
||
3. Use `nvpmodel -m 1` (5W mode) during mapping-only tasks
|
||
4. External powered USB hub for peripherals
|
||
|
||
---
|
||
|
||
## 8. Milestone Checklist
|
||
|
||
- [ ] Flash JetPack 4.6 on Nano (or verify existing install)
|
||
- [ ] Install Docker + NVIDIA Container Runtime on Nano
|
||
- [ ] Pull `dustynv/ros:humble-ros-base-l4t-r32.7.1` container
|
||
- [ ] Verify D435i recognized: `realsense-viewer` or `rs-enumerate-devices`
|
||
- [ ] Verify RPLIDAR port: `ls /dev/ttyUSB*`
|
||
- [ ] Build/pull `realsense2_camera` ROS2 package in container
|
||
- [ ] Build/pull `rplidar_ros` ROS2 package in container
|
||
- [ ] Build/pull `rtabmap_ros` ROS2 package in container
|
||
- [ ] Test individual sensor topics with `ros2 topic echo`
|
||
- [ ] Run SLAM launch file — verify `/map` published
|
||
- [ ] Record rosbag for offline tuning
|
||
- [ ] Document measured TF offsets (physical mount positions)
|
||
- [ ] Write Phase 2b bead for Nav2 integration
|
||
|
||
---
|
||
|
||
## 9. Repo Structure (to be created)
|
||
|
||
```
|
||
saltylab-firmware/
|
||
└── projects/
|
||
└── saltybot/
|
||
├── SLAM-SETUP-PLAN.md ← this file
|
||
├── SALTYLAB.md ← main design doc (TBD)
|
||
└── slam/
|
||
├── docker/
|
||
│ ├── docker-compose.yml
|
||
│ └── Dockerfile.slam
|
||
├── launch/
|
||
│ └── slam.launch.py
|
||
├── config/
|
||
│ ├── rtabmap_params.yaml
|
||
│ └── realsense_params.yaml
|
||
└── urdf/
|
||
└── saltybot.urdf.xacro
|
||
```
|
||
|
||
---
|
||
|
||
## 10. References
|
||
|
||
- [dusty-nv/jetson-containers](https://github.com/dusty-nv/jetson-containers) — official NVIDIA ROS2 Docker containers for Jetson
|
||
- [rtabmap_ros](https://github.com/introlab/rtabmap_ros) — RTAB-Map ROS2 wrapper
|
||
- [realsense-ros](https://github.com/IntelRealSense/realsense-ros) — Intel RealSense ROS2 wrapper
|
||
- [rplidar_ros](https://github.com/Slamtec/rplidar_ros) — Slamtec RPLIDAR ROS2 package
|
||
- [reedjacobp/JetsonBot](https://github.com/reedjacobp/JetsonBot) — Nano + D435i + RPLIDAR reference implementation
|
||
- [RTAB-Map D435 + RPLidar discussion](https://answers.ros.org/question/367019/using-2-d435-cameras-and-an-rplidar-with-rtabmap/)
|
||
- [Comparative SLAM evaluation (2024)](https://arxiv.org/html/2401.02816v1)
|