saltylab-firmware/jetson/docs/headscale-vpn-setup.md

Headscale VPN Auto-Connect Setup — Jetson Orin

This document describes the auto-connect VPN setup for the Jetson Orin using Tailscale client connecting to the Headscale server at tailscale.vayrette.com:8180.

Overview

Device Name: saltylab-orin Headscale Server: https://tailscale.vayrette.com:8180 Primary Features:

• Auto-connect on system boot
• Persistent auth key for unattended login
• SSH + HTTP over Tailscale (tailnet)
• WiFi resilience and fallback
• systemd auto-restart on failure

Architecture

Components

1. Tailscale Client (/usr/sbin/tailscaled)

   • VPN daemon running on Jetson
   • Manages WireGuard tunnels
   • Connects to Headscale coordination server

2. systemd Service (tailscale-vpn.service)

   • Auto-starts on boot
   • Restarts on failure
   • Manages lifecycle of tailscaled daemon
   • Logs to journald

3. Auth Key Manager (headscale-auth-helper.sh)

   • Generates and validates auth keys
   • Stores keys securely at /opt/saltybot/tailscale-auth.key
   • Manages revocation

4. Setup Script (setup-tailscale.sh)

   • One-time installation of Tailscale package
   • Configures IP forwarding
   • Sets up persistent state directories

Installation

1. Run Jetson Setup

sudo bash jetson/scripts/setup-jetson.sh

This configures the base system (Docker, NVMe, power mode, etc.).

2. Install Tailscale

sudo bash jetson/scripts/setup-tailscale.sh

This:

• Installs Tailscale from official Ubuntu repository
• Creates state/cache directories at /var/lib/tailscale
• Enables IP forwarding
• Creates environment config at /etc/tailscale/tailscale-env

3. Generate and Store Auth Key

sudo bash jetson/scripts/headscale-auth-helper.sh generate

This prompts you to enter a pre-authorized key (preauthkey) from the Headscale server.

To get a preauthkey from Headscale:

# On Headscale server
sudo headscale preauthkeys create --expiration 2160h --user default
# Copy the generated key (tskey_...)

Then paste into the helper script when prompted.

The key is stored at: /opt/saltybot/tailscale-auth.key

4. Install systemd Services

sudo bash jetson/systemd/install_systemd.sh

This:

• Deploys repo to /opt/saltybot/jetson
• Installs tailscale-vpn.service to /etc/systemd/system/
• Enables service for auto-start

5. Start the VPN Service

sudo systemctl start tailscale-vpn

Check status:

sudo systemctl status tailscale-vpn
sudo journalctl -fu tailscale-vpn
sudo tailscale status

Usage

Check VPN Status

sudo tailscale status

Example output:

  saltylab-orin        100.x.x.x    juno    linux   -
    100.x.x.x    is local IP address
     100.x.x.x  is remote IP address

Access via SSH over Tailnet

From any machine on the tailnet:

ssh <username>@saltylab-orin.tail12345.ts.net

Or use the IP directly:

ssh <username>@100.x.x.x

Access via HTTP

If running a web service (e.g., ROS2 visualization):

http://saltylab-orin.tail12345.ts.net:8080
# or
http://100.x.x.x:8080

View Logs

# Real-time logs
sudo journalctl -fu tailscale-vpn

# Last 50 lines
sudo journalctl -n 50 -u tailscale-vpn

# Since last boot
sudo journalctl -b -u tailscale-vpn

WiFi Resilience

The systemd service is configured with aggressive restart policies to handle WiFi drops:

Restart=always
RestartSec=5s
StartLimitInterval=60s
StartLimitBurst=10

This means:

• Automatically restarts after WiFi drops
• Waits 5 seconds between restart attempts
• Allows up to 10 restarts per 60-second window

The daemon will continuously attempt to reconnect when the primary network is unavailable.

Persistent Storage

Auth Key

Location: /opt/saltybot/tailscale-auth.key Permissions: 600 (readable only by root) Ownership: root:root

If the key file is missing on boot:

1. The systemd service will not auto-authenticate
2. Manual login is required: sudo tailscale login --login-server=https://tailscale.vayrette.com:8180
3. Re-run headscale-auth-helper.sh generate to store the key

State Directory

Location: /var/lib/tailscale/ Contents:

• Machine state and private key
• WireGuard configuration
• Connection state

These are persisted so the device retains its identity and tailnet IP across reboots.

Troubleshooting

Service Won't Start

sudo systemctl status tailscale-vpn
sudo journalctl -u tailscale-vpn -n 30

Check for:

• Missing auth key: ls -la /opt/saltybot/tailscale-auth.key
• Tailscale package: which tailscale
• Permissions: ls -la /var/lib/tailscale

Can't Connect to Headscale Server

sudo tailscale debug derp
curl -v https://tailscale.vayrette.com:8180

Check:

• Network connectivity: ping 8.8.8.8
• DNS: nslookup tailscale.vayrette.com
• Firewall: Port 443 (HTTPS) must be open

Auth Key Expired

If the preauthkey expires:

# Get new key from Headscale server
# Then update:
sudo bash jetson/scripts/headscale-auth-helper.sh revoke
sudo bash jetson/scripts/headscale-auth-helper.sh generate
sudo systemctl restart tailscale-vpn

Can't SSH Over Tailnet

# Verify device is in tailnet:
sudo tailscale status

# Check tailnet connectivity from another machine:
ping <device-ip>

# SSH with verbose output:
ssh -vv <username>@saltylab-orin.tail12345.ts.net

Memory/Resource Issues

Monitor memory with Tailscale:

ps aux | grep tailscaled
sudo systemctl show -p MemoryUsage tailscale-vpn

Tailscale typically uses 30-80 MB of RAM depending on network size.

Integration with Other Services

ROS2 Services

Expose ROS2 services over the tailnet by ensuring your nodes bind to:

• 0.0.0.0 for accepting all interfaces
• Or explicitly bind to the Tailscale IP (100.x.x.x)

Docker Services

If running Docker services that need tailnet access:

# In docker-compose.yml
services:
  app:
    network_mode: host  # Access host's Tailscale interface

Security Considerations

1. Auth Key: Stored in plaintext at /opt/saltybot/tailscale-auth.key

   • Use file permissions (600) to restrict access
   • Consider encryption if sensitive environment

2. State Directory: /var/lib/tailscale/ contains private keys

   • Restricted permissions (700)
   • Only readable by root

3. SSH Over Tailnet: No ACL restrictions by default

   • Configure Headscale ACL rules if needed: headscale acl

4. Key Rotation: Rotate preauthkeys periodically

   • Headscale supports key expiration (set to 2160h = 90 days default)

Maintenance

Update Tailscale

sudo apt-get update
sudo apt-get install --only-upgrade tailscale
sudo systemctl restart tailscale-vpn

Backup

Backup the auth key:

sudo cp /opt/saltybot/tailscale-auth.key ~/tailscale-auth.key.backup

Monitor

Set up log rotation to prevent journal bloat:

# journald auto-rotates, but you can check:
journalctl --disk-usage

MQTT Reporting

The Jetson reports VPN status to the MQTT broker:

saltylab/jetson/vpn/status -> online|offline|connecting
saltylab/jetson/vpn/ip -> 100.x.x.x
saltylab/jetson/vpn/hostname -> saltylab-orin.tail12345.ts.net

This is reported by the cellular/MQTT bridge node on boot and after reconnection.

References

• Headscale Documentation
• Tailscale Documentation
• Tailscale CLI Reference