Add "Vision System"
parent
f77882f814
commit
617520fe04
73
Vision-System.md
Normal file
73
Vision-System.md
Normal file
@ -0,0 +1,73 @@
|
|||||||
|
# Vision System
|
||||||
|
|
||||||
|
## Camera Array — 360° Surround Vision
|
||||||
|
|
||||||
|
### 4× IMX219 160° Wide Angle (CSI)
|
||||||
|
- **Sensor:** Sony IMX219, 8MP
|
||||||
|
- **Resolution:** 3280×2464 (stills), 1080p @ 30fps, 720p @ 60fps (video)
|
||||||
|
- **FOV:** 160° wide angle — 4 cameras = full 360° with overlap
|
||||||
|
- **Interface:** MIPI CSI-2 (15-pin ribbon cable)
|
||||||
|
- **Focus:** Manual focus, 2.96mm lens
|
||||||
|
- **Arriving:** March 1, 2026
|
||||||
|
|
||||||
|
### Mounting Layout
|
||||||
|
```
|
||||||
|
FRONT
|
||||||
|
┌────────┐
|
||||||
|
│ CAM 0 │ → Forward (0°)
|
||||||
|
┌────┤ 160° ├────┐
|
||||||
|
│ └────────┘ │
|
||||||
|
│CAM 3 CAM 1│
|
||||||
|
│ Left Right│
|
||||||
|
│(270°) (90°) │
|
||||||
|
│ ┌────────┐ │
|
||||||
|
└────┤ CAM 2 ├────┘
|
||||||
|
│ 160° │
|
||||||
|
└────────┘
|
||||||
|
REAR (180°)
|
||||||
|
```
|
||||||
|
|
||||||
|
Each camera covers 160° — with 4 cameras at 90° intervals, there's ~70° overlap between adjacent cameras. This overlap is useful for:
|
||||||
|
- Cross-camera object tracking (hand off between views)
|
||||||
|
- Panoramic stitching
|
||||||
|
- Stereo depth estimation from adjacent pairs
|
||||||
|
|
||||||
|
### RealSense D435i (USB3)
|
||||||
|
- **Purpose:** Primary depth sensing + RGB for SLAM/navigation
|
||||||
|
- **Mount:** Forward-facing, mid-height on stem
|
||||||
|
- **Resolution:** 1280×720 depth @ 30fps + 1080p RGB
|
||||||
|
- **Depth range:** 0.2m - 10m
|
||||||
|
- **IMU:** Built-in accelerometer + gyroscope
|
||||||
|
|
||||||
|
### RPLIDAR A1M8 (USB)
|
||||||
|
- **Purpose:** 360° 2D laser scan for SLAM/nav2
|
||||||
|
- **Mount:** Top of sensor tower (under helmet)
|
||||||
|
- **Range:** 12m, 360°, ~5.5Hz scan rate
|
||||||
|
|
||||||
|
## CSI Adapter Required
|
||||||
|
The Orin Nano Dev Kit carrier exposes 2 CSI connectors. To use all 4 cameras simultaneously, need a **quad CSI multiplexer**:
|
||||||
|
- ArduCam Multi-Camera Adapter for Jetson (~$40 USD)
|
||||||
|
- Or Arducam Camarray HAT (hardware sync option)
|
||||||
|
- **TODO:** Confirm Orin Nano compatibility before ordering
|
||||||
|
|
||||||
|
## Processing Pipeline (on Orin Nano @ 67 TOPS)
|
||||||
|
```
|
||||||
|
4× IMX219 (CSI) ──→ Jetson ISP (hardware) ──→ TensorRT inference
|
||||||
|
├── Person detection (all 4 streams)
|
||||||
|
├── Object classification
|
||||||
|
└── Optical flow / motion
|
||||||
|
|
||||||
|
RealSense (USB3) ──→ depth + RGB ──→ SLAM (slam_toolbox)
|
||||||
|
──→ obstacle avoidance (Nav2 costmap)
|
||||||
|
|
||||||
|
RPLIDAR (USB) ──→ 2D scan ──→ SLAM (primary scan source)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Total Sensor Budget
|
||||||
|
| Sensor | Interface | Power | Weight |
|
||||||
|
|--------|-----------|-------|--------|
|
||||||
|
| 4× IMX219 | CSI (0W USB) | ~1W total | ~20g total |
|
||||||
|
| RealSense D435i | USB3 | ~1.5W | 72g |
|
||||||
|
| RPLIDAR A1 | USB | ~2W | 170g |
|
||||||
|
| BME280 | I2C (on FC) | ~0.01W | 2g |
|
||||||
|
| **Total** | | **~4.5W** | **~264g** |
|
||||||
Loading…
x
Reference in New Issue
Block a user