fix(docker): make s6 lifecycle work for the unprivileged hermes user
Resolves the explicit "Known follow-up" left by commit2f8ceeab9and the resulting CI failures in tests/docker/test_dashboard.py and tests/docker/test_s6_profile_gateway_integration.py. The product gap --------------- Every hermes runtime operation inside the container runs as the hermes user (UID 10000) via s6-setuidgid. But s6-supervise — spawned by s6-svscan running as PID 1 — creates each service's supervise/ and top-level event/ directories with mode 0700 owned by its effective UID (root). That left every s6-svc / s6-svstat / s6-svwait call from hermes hitting EACCES on the supervise/control FIFO and supervise/status — i.e. the entire S6ServiceManager lifecycle (register, start, stop, unregister) was inert in production. The2f8ceeab9commit message called this out and deferred the fix. The audit changes that landed alongside it (defaulting docker_exec to -u hermes) made the integration tests reproduce the bug deterministically; the fix below resolves it. The fix: pre-create the supervise/ skeleton hermes-owned ---------------------------------------------------------- Reading s6's source (src/supervision/s6-supervise.c::trymkdir + control_init), the mkdir and mkfifo calls that build the supervise tree are EEXIST-safe: if the directory or FIFO is already present, s6-supervise reuses it and skips the chown/chmod fix-up that would normally make event/ 03730 root:root. So if we lay the skeleton down with hermes ownership before triggering s6-svscanctl -a, s6-supervise inherits our layout and never touches it. The death_tally / lock / status regular files written later by s6-supervise (still as root) land mode 0644 — world-readable — which is all s6-svstat needs. New module-level helper _seed_supervise_skeleton(svc_dir) in hermes_cli/service_manager.py lays down: svc_dir/event/ hermes:hermes 03730 svc_dir/supervise/ hermes:hermes 0755 svc_dir/supervise/event/ hermes:hermes 03730 svc_dir/supervise/control hermes:hermes 0660 (FIFO) svc_dir/log/event/ hermes:hermes 03730 (if log/ present) svc_dir/log/supervise/ hermes:hermes 0755 svc_dir/log/supervise/event/ hermes:hermes 03730 svc_dir/log/supervise/control hermes:hermes 0660 (FIFO) The log/ branch matters because the logger is a second s6-supervise instance — without it, unregister rmtree races on the logger's root-owned supervise dir even after the parent slot's supervise/ is hermes-owned. The helper is idempotent and swallows PermissionError on chown so it works equally well when called from root (cont-init.d) or hermes (runtime register). Wiring ------ 1. S6ServiceManager.register_profile_gateway calls _seed_supervise_skeleton(tmp_dir) just before publishing the slot via Path.replace. Runtime-registered profile gateways are set up by hermes. 2. container_boot._register_service does the same in the cont-init.d reconciliation path so boot-time-restored profile slots inherit the same layout. 3. New cont-init.d/015-supervise-perms script chowns the supervise/ and event/ trees for STATIC s6-rc services (dashboard, main-hermes). These are spawned by s6-rc before cont-init.d gets to run, so the EEXIST-trick doesn't apply; we chown the already-existing tree instead. s6-supervise keeps using the same files; it never re-asserts ownership on a running service. The script skips s6-overlay internal services (s6rc-*, s6-linux-*) so the supervision tree itself stays root-only. 015- slot is intentional: lex-sorts between 01-hermes-setup and 02-reconcile-profiles in the container's C-locale, so the chown finishes before the reconciler walks the scandir. Unregister teardown reordering ------------------------------ S6ServiceManager.unregister_profile_gateway now fires s6-svscanctl -an BEFORE rmtree (with a 200ms grace), so s6-svscan reaps the supervise child and releases its file handles on supervise/lock + supervise/status before we try to remove the directory. Previously rmtree raced s6-supervise on a set of files inside the supervise dir, and even with the parent supervise/ now hermes-owned, the contained files (death_tally, lock, status, written by root) could still be in use. Dashboard down-state redesign ----------------------------- The original PR #30136 review fix wrote a 'down' marker file into /run/service/dashboard/ via cont-init.d/03-dashboard-toggle. That approach was broken in two ways: (a) /run/service/dashboard is a symlink to a TRANSIENT /run/s6-rc:s6-rc-init:<tmpdir>/ directory while s6-rc is mid-transaction; the touch landed in a soon-to-be-discarded tmp. (b) Even when written to the final /run/s6-rc/servicedirs/ location, the 'down' file is only consulted by s6-supervise at slot startup. s6-rc's user-bundle explicitly transitions 'dashboard' to 'up' on every boot, overriding any down marker. The right fix is the canonical s6 pattern: when HERMES_DASHBOARD is unset, the dashboard run script exits 0 and a companion finish script exits 125. Per s6-supervise(8), exit code 125 from the finish script is the 'permanent failure, do not restart' marker — equivalent to s6-svc -O. The slot reports as 'down' to s6-svstat, matching the reality that no dashboard process is running. When HERMES_DASHBOARD IS truthy, finish exits 0 and restart-on-crash semantics apply. 03-dashboard-toggle is removed (its function is now subsumed by the run/finish pair). Tests ----- Adds four unit tests for _seed_supervise_skeleton covering the produced layout, the log/ subservice case, the skip-when-no-log case, and idempotency. The live-container verification continues to live in tests/docker/test_s6_profile_gateway_integration.py and tests/docker/test_dashboard.py — both now pass against the rebuilt image. References ---------- * Skarnet skaware mailing list 2020-02-02 (Laurent Bercot + Guillermo Diaz Hartusch) on unprivileged s6 tool semantics: http://skarnet.org/lists/skaware/1424.html * just-containers/s6-overlay#130 — same EEXIST-preseed pattern, community-validated 2016 onward * https://skarnet.org/software/s6/servicedir.html — exit-code 125 semantics in finish scripts (cherry picked from commitc41f908ad4)
This commit is contained in:
@@ -412,6 +412,115 @@ def test_s6_manager_kind_and_supports_registration() -> None:
|
||||
assert mgr.supports_runtime_registration() is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _seed_supervise_skeleton — unit tests
|
||||
# ---------------------------------------------------------------------------
|
||||
#
|
||||
# The skeleton helper pre-creates the dirs and FIFOs that s6-supervise
|
||||
# would otherwise create as root mode 0700, locking out the
|
||||
# unprivileged hermes user from every lifecycle op. These tests run
|
||||
# against tmp_path and assert the produced layout — the live-container
|
||||
# verification (against real s6-svc / s6-svstat) lives in
|
||||
# tests/docker/test_s6_profile_gateway_integration.py.
|
||||
|
||||
|
||||
def test_seed_supervise_skeleton_creates_expected_layout(tmp_path) -> None:
|
||||
"""Verifies the dirs + FIFO + modes the helper lays down."""
|
||||
import stat
|
||||
|
||||
from hermes_cli.service_manager import _seed_supervise_skeleton
|
||||
|
||||
svc_dir = tmp_path / "gateway-foo"
|
||||
svc_dir.mkdir()
|
||||
|
||||
_seed_supervise_skeleton(svc_dir)
|
||||
|
||||
# Top-level event/ — s6-svlisten1 event subscription dir.
|
||||
event = svc_dir / "event"
|
||||
assert event.is_dir(), "missing top-level event/"
|
||||
assert stat.S_IMODE(event.stat().st_mode) == 0o3730, (
|
||||
f"event/ mode = {oct(event.stat().st_mode)}, want 03730"
|
||||
)
|
||||
|
||||
# supervise/ dir.
|
||||
supervise = svc_dir / "supervise"
|
||||
assert supervise.is_dir(), "missing supervise/"
|
||||
assert stat.S_IMODE(supervise.stat().st_mode) == 0o755
|
||||
|
||||
# supervise/event/.
|
||||
supervise_event = supervise / "event"
|
||||
assert supervise_event.is_dir(), "missing supervise/event/"
|
||||
assert stat.S_IMODE(supervise_event.stat().st_mode) == 0o3730
|
||||
|
||||
# supervise/control FIFO.
|
||||
control = supervise / "control"
|
||||
assert control.exists(), "missing supervise/control FIFO"
|
||||
assert stat.S_ISFIFO(control.stat().st_mode), (
|
||||
"supervise/control must be a FIFO"
|
||||
)
|
||||
assert stat.S_IMODE(control.stat().st_mode) == 0o660
|
||||
|
||||
|
||||
def test_seed_supervise_skeleton_handles_log_subservice(tmp_path) -> None:
|
||||
"""When a log/ subdir exists, its supervise tree also gets seeded.
|
||||
|
||||
Without this, ``unregister_profile_gateway``'s rmtree would EACCES
|
||||
on the logger's root-owned supervise dir even after the parent
|
||||
slot's supervise/ was hermes-owned.
|
||||
"""
|
||||
import stat
|
||||
|
||||
from hermes_cli.service_manager import _seed_supervise_skeleton
|
||||
|
||||
svc_dir = tmp_path / "gateway-foo"
|
||||
svc_dir.mkdir()
|
||||
(svc_dir / "log").mkdir() # logger subdir present
|
||||
|
||||
_seed_supervise_skeleton(svc_dir)
|
||||
|
||||
# Logger's own supervise tree is seeded the same way.
|
||||
log_event = svc_dir / "log" / "event"
|
||||
log_supervise = svc_dir / "log" / "supervise"
|
||||
log_supervise_event = log_supervise / "event"
|
||||
log_control = log_supervise / "control"
|
||||
|
||||
assert log_event.is_dir()
|
||||
assert stat.S_IMODE(log_event.stat().st_mode) == 0o3730
|
||||
assert log_supervise.is_dir()
|
||||
assert log_supervise_event.is_dir()
|
||||
assert log_control.exists() and stat.S_ISFIFO(log_control.stat().st_mode)
|
||||
|
||||
|
||||
def test_seed_supervise_skeleton_skips_when_no_log_subservice(tmp_path) -> None:
|
||||
"""If log/ isn't present, no logger skeleton is created."""
|
||||
from hermes_cli.service_manager import _seed_supervise_skeleton
|
||||
|
||||
svc_dir = tmp_path / "gateway-foo"
|
||||
svc_dir.mkdir()
|
||||
|
||||
_seed_supervise_skeleton(svc_dir)
|
||||
|
||||
assert not (svc_dir / "log").exists(), (
|
||||
"helper must not synthesize a log/ subdir on its own"
|
||||
)
|
||||
|
||||
|
||||
def test_seed_supervise_skeleton_is_idempotent(tmp_path) -> None:
|
||||
"""Calling the helper twice on the same dir is a no-op the second time.
|
||||
|
||||
Important because s6-supervise may have already opened the FIFO
|
||||
when a re-register / reconcile happens; double-creation would
|
||||
error out. The helper short-circuits on existence.
|
||||
"""
|
||||
from hermes_cli.service_manager import _seed_supervise_skeleton
|
||||
|
||||
svc_dir = tmp_path / "gateway-foo"
|
||||
svc_dir.mkdir()
|
||||
|
||||
_seed_supervise_skeleton(svc_dir)
|
||||
_seed_supervise_skeleton(svc_dir) # must not raise
|
||||
|
||||
|
||||
def test_s6_register_creates_service_dir_and_triggers_scan(
|
||||
s6_scandir, fake_subprocess_run,
|
||||
) -> None:
|
||||
|
||||
Reference in New Issue
Block a user