fix(kanban): bridge worker runtime activity to board heartbeat (#31752)

The dispatcher watchdog (release_stale_claims) reads tasks.last_heartbeat_at
to decide whether to reclaim a running task. The agent maintains its own
in-process `_last_activity_ts` for every chunk/tool result, but those
liveness ticks never reach the board unless the model explicitly calls
the `kanban_heartbeat` tool — so a worker actively executing a long run
without tool-level heartbeats can be reclaimed mid-flight as 'stale',
returning the task to ready and orphaning the in-flight worker's progress.

Fix: in `_touch_activity` (the canonical 'we just did work' hook in
run_agent.py), call a new `heartbeat_current_worker_from_env` helper
in `tools/kanban_tools.py` that:

- No-ops outside dispatcher-spawned worker context (no HERMES_KANBAN_TASK).
- Rate-limited to one DB write per 60s (runtime activity ticks too often
  to faithfully mirror; we just need the watchdog to see liveness).
- Best-effort: never raises. heartbeat_claim + heartbeat_worker calls are
  individually try/except'd; any DB error logs at debug and returns.
- Uses worker env identity: HERMES_KANBAN_TASK + HERMES_KANBAN_RUN_ID +
  HERMES_KANBAN_CLAIM_LOCK (all pinned by the dispatcher at spawn time).
- No durable note on auto-heartbeats — that's reserved for the explicit
  `kanban_heartbeat` tool which carries a model-supplied note.

The explicit `kanban_heartbeat` tool stays available unchanged for
workers that want to attach a note or pre-emptively extend a claim
across a known-long single tool call.

Co-authored-by: faisfamilytravel <223516181+faisfamilytravel@users.noreply.github.com>
This commit is contained in:
teknium1
2026-05-29 00:05:58 -07:00
committed by Teknium
co-authored by faisfamilytravel
parent 40217aa194
commit bc31ee5cf8
3 changed files with 103 additions and 1 deletions
+18 -1
View File
@@ -2144,9 +2144,26 @@ class AIAgent:
return apply_pending_steer_to_tool_results(self, messages, num_tool_msgs)
def _touch_activity(self, desc: str) -> None:
"""Update the last-activity timestamp and description (thread-safe)."""
"""Update the last-activity timestamp and description (thread-safe).
Also bridges to the kanban board's heartbeat fields when this
process is a dispatcher-spawned worker (HERMES_KANBAN_TASK set),
so the dispatcher watchdog doesn't reclaim an actively-running
worker as stale (#31752). Bridge is rate-limited (60s) and
best-effort — it never raises into the agent loop.
"""
self._last_activity_ts = time.time()
self._last_activity_desc = desc
if os.environ.get("HERMES_KANBAN_TASK"):
try:
from tools.kanban_tools import heartbeat_current_worker_from_env
heartbeat_current_worker_from_env()
except Exception:
# Never let the bridge break the agent loop. The function
# already swallows exceptions internally; this outer guard
# covers import-time failures (kanban_tools unavailable,
# etc.) on niche deployment surfaces.
pass
def _capture_rate_limits(self, http_response: Any) -> None:
"""Parse x-ratelimit-* headers from an HTTP response and cache the state.