seb/saltylab-firmware
1
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity

Add sensor feeds to SaltyLab HUD (GPS, LIDAR, RealSense) #413

New Issue
Closed
opened 2026-03-04 22:38:29 -05:00 by seb · 1 comment
seb commented 2026-03-04 22:38:29 -05:00
Owner
Copy Link

Summary

The HUD (ui/index.html) currently focuses on FC control (WebSerial, PID tuning, 3D viz). We now have live sensor feeds running on the Jetson Orin that should be integrated into the HUD as additional panels/tabs.

New Sensor Feeds to Add

1. GPS Map (SIM7600A-H)

  • Endpoint: GET /gps on the Jetson sensor server (port 8888)
  • Returns: {lat, lon, alt, speed, sats, fix, time, track[]}
  • Use Leaflet.js with dark CartoDB tiles
  • Show live position marker + track polyline
  • Display stats: coordinates, altitude, speed, satellite count

2. LIDAR 360° Scan (RPLIDAR A1M8)

  • Endpoint: GET /lidar on port 8888
  • Returns array of {a: angle, d: distance_mm, q: quality}
  • Render as 2D polar plot on canvas (like current standalone viz)
  • Color points by distance, show range rings

3. RealSense D435i Camera

  • Color stream: GET /stream/color (MJPEG)
  • Depth stream: GET /stream/depth (MJPEG, JET colormap)
  • Note: RealSense currently has HID/kernel issues on JetPack 6.1 — may need V4L2 fallback. Show placeholder if unavailable.

Architecture

  • Sensor server runs on Jetson at http://<jetson-ip>:8888
  • HUD should auto-discover or be configured with the Jetson IP
  • Sensor panels should be toggleable tabs alongside existing FC panels
  • Consider a 2x2 grid layout: 3D bot viz | GPS map | LIDAR | Camera

Assignee

@sl-webui

Reference

  • Standalone sensor server: saltylab-viz/server.py on the Orin
  • GPS uses NMEA from SIM7600 ttyUSB ports (auto-detected)
  • LIDAR uses RPLIDAR CP210x (auto-detected)
  • Leaflet CDN: https://unpkg.com/leaflet@1.9.4/dist/leaflet.js
## Summary The HUD (`ui/index.html`) currently focuses on FC control (WebSerial, PID tuning, 3D viz). We now have live sensor feeds running on the Jetson Orin that should be integrated into the HUD as additional panels/tabs. ## New Sensor Feeds to Add ### 1. GPS Map (SIM7600A-H) - **Endpoint:** `GET /gps` on the Jetson sensor server (port 8888) - Returns: `{lat, lon, alt, speed, sats, fix, time, track[]}` - Use **Leaflet.js** with dark CartoDB tiles - Show live position marker + track polyline - Display stats: coordinates, altitude, speed, satellite count ### 2. LIDAR 360° Scan (RPLIDAR A1M8) - **Endpoint:** `GET /lidar` on port 8888 - Returns array of `{a: angle, d: distance_mm, q: quality}` - Render as 2D polar plot on canvas (like current standalone viz) - Color points by distance, show range rings ### 3. RealSense D435i Camera - **Color stream:** `GET /stream/color` (MJPEG) - **Depth stream:** `GET /stream/depth` (MJPEG, JET colormap) - Note: RealSense currently has HID/kernel issues on JetPack 6.1 — may need V4L2 fallback. Show placeholder if unavailable. ## Architecture - Sensor server runs on Jetson at `http://<jetson-ip>:8888` - HUD should auto-discover or be configured with the Jetson IP - Sensor panels should be toggleable tabs alongside existing FC panels - Consider a 2x2 grid layout: 3D bot viz | GPS map | LIDAR | Camera ## Assignee @sl-webui ## Reference - Standalone sensor server: `saltylab-viz/server.py` on the Orin - GPS uses NMEA from SIM7600 ttyUSB ports (auto-detected) - LIDAR uses RPLIDAR CP210x (auto-detected) - Leaflet CDN: `https://unpkg.com/leaflet@1.9.4/dist/leaflet.js`
seb commented 2026-03-04 22:42:03 -05:00
Author
Owner
Copy Link

Additional: Two IMX219 CSI Cameras Available

The Orin has two working CSI cameras (IMX219 / RPi Camera v2):

  • /dev/video0 (sensor 9) — up to 3280x2464 @21fps, 1280x720 @60fps
  • /dev/video1 (sensor 10) — same specs

GStreamer pipeline (hardware-accelerated):

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! "video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1" ! nvjpegenc ! filesink location=/tmp/csi.jpg

For MJPEG streaming to the HUD:

import gi
gi.require_version("Gst", "1.0")
# Use nvarguscamerasrc → nvvidconv → nvjpegenc pipeline

These are stereo-capable (two cameras). Could replace RealSense for depth estimation.

Add CSI camera feeds as additional panels alongside the sensor feeds from the parent issue.

## Additional: Two IMX219 CSI Cameras Available The Orin has **two working CSI cameras** (IMX219 / RPi Camera v2): - `/dev/video0` (sensor 9) — up to 3280x2464 @21fps, 1280x720 @60fps - `/dev/video1` (sensor 10) — same specs ### GStreamer pipeline (hardware-accelerated): ``` gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! "video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1" ! nvjpegenc ! filesink location=/tmp/csi.jpg ``` ### For MJPEG streaming to the HUD: ```python import gi gi.require_version("Gst", "1.0") # Use nvarguscamerasrc → nvvidconv → nvjpegenc pipeline ``` These are stereo-capable (two cameras). Could replace RealSense for depth estimation. Add CSI camera feeds as additional panels alongside the sensor feeds from the parent issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
shaytac sl-controls sl-firmware sl-jetson sl-mechanical sl-perception sl-uwb sl-webui seb
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: seb/saltylab-firmware#413
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?