fix(update): never spawn an interactive polkit prompt when restarting a system-scope gateway (#45145)

When hermes update restarts a hermes-gateway system service as a
non-root user, the systemctl reset-failed/start/restart calls trigger
polkit's org.freedesktop.systemd1.manage-units TTY authentication
agent. That prompt runs inside a captured subprocess with a 10-15s
timeout, so it flashes and dies before the user can answer, and the
resulting TimeoutExpired was swallowed silently by the loop's blanket
except — the restart phase just vanished with no output.

- Resolve a manage-units command prefix up front: plain systemctl as
  root, sudo -n systemctl as non-root (with a targeted reset-failed
  probe so least-privilege sudoers entries scoped to hermes-gateway*
  qualify), or None when no non-interactive privilege path exists.
- Add --no-ask-password to every manage-units call in the update
  restart path so polkit can never prompt inside a captured subprocess.
- When unprivileged: after a graceful drain, rely on systemd's own
  RestartSec auto-restart (needs no privileges) with a message about
  the wait; skip the force-restart fallback with clear manual
  instructions instead of racing a doomed polkit prompt.
- Surface TimeoutExpired in the restart loop instead of passing
  silently, and add sudo to the system-scope recovery hints.
- Docs: headless-VM note recommending user service + enable-linger,
  or sudo updates / a scoped NOPASSWD sudoers entry for system
  services.
This commit is contained in:
Teknium
2026-06-12 12:38:15 -07:00
committed by GitHub
parent fca84fe20b
commit 9b01c4d193
2 changed files with 162 additions and 37 deletions
@@ -388,6 +388,23 @@ journalctl -u hermes-gateway -f
Use the user service on laptops and dev boxes. Use the system service on VPS or headless hosts that should come back at boot without relying on systemd linger.
:::tip Headless VMs: user service + linger avoids root prompts
A system service needs root for every restart — including the automatic gateway restart at the end of `hermes update`. When `hermes update` runs as a non-root user, it tries passwordless `sudo systemctl`; if that's unavailable, it skips the restart and prints the manual `sudo systemctl restart hermes-gateway` command (it never blocks on an interactive password prompt).
For a headless VM you never log into, a **user** service with lingering enabled gives you the same start-at-boot behavior with zero root involvement:
```bash
hermes gateway install # user service
sudo loginctl enable-linger $USER # one-time: start at boot, survive logout
```
After that, `hermes update` can restart the gateway without any privileges. If you prefer to keep the system service, either run updates with `sudo hermes update`, or grant the service account passwordless sudo for systemctl, e.g. in `sudo visudo -f /etc/sudoers.d/hermes-gateway`:
```
hermes ALL=(root) NOPASSWD: /usr/bin/systemctl --no-ask-password reset-failed hermes-gateway*, /usr/bin/systemctl --no-ask-password start hermes-gateway*, /usr/bin/systemctl --no-ask-password restart hermes-gateway*
```
:::
Avoid keeping both the user and system gateway units installed at once unless you really mean to. Hermes will warn if it detects both because start/stop/status behavior gets ambiguous.
:::info Multiple installations