Reconciles the "pluginify everything" refactor (moving provider adapters,
platform adapters, and terminal backends out of core into plugins/) with
267 commits of main. main had 515 edits across 635 files; our branch moved
98 files into plugins/. The dangerous class was main editing files at OLD
paths that we relocated — handled below.
STRUCTURAL DECISIONS
- ai-gateway + vercel_sandbox: honored main's PR #33067 deletion. Dropped our
orphaned plugins/terminals/vercel/ dir and all 3 stale vercel refs that
survived in pyproject (workspace member, [all] entry, uv source).
- tools/lazy_deps.py: kept main's version. Our branch deleted it incompletely
(WIP) while main has 30 live callers + active maintenance. main's intent won.
- agent/agent_runtime_helpers.py: COMBINED — main's atomic swap+rollback
snapshot envelope wraps our pluginified registries-based anthropic imports.
- agent/credential_pool.py: took our plugin-hook indirection AND ported main's
api_key_path_explicit OAuth-masquerade security fix INTO the anthropic
credential_pool_hook (it was pluginified from a pre-fix base and would have
been silently dropped otherwise).
- hermes_cli/model_switch.py, hermes_cli/models.py: took main's logic
(cached_provider_model_ids, AI-gateway removal) then re-applied our pluginify
import rewrites (agent.bedrock_adapter/anthropic_adapter -> registries
namespace lookups) since those modules moved to plugins.
- tools/terminal_tool.py: took main's version (vercel_sandbox deletion) then
restored our lazy registry resolution for modal + daytona environments
(both moved to plugins; main's direct imports would crash at runtime).
- nix/checks.nix: kept both — our hermetic-boundary checks + main's
messaging-variant discord.py guard.
- pyproject.toml: plugin extras as workspace members (ours) + main's new wecom
extra; vercel removed throughout.
- uv.lock: regenerated with `uv lock` (not hand-merged); 233 packages.
VERIFICATION
- Zero conflict markers anywhere.
- Exhaustive 17-moved-module grep: no dead imports of relocated modules in core.
- Import smoke test: all hot core modules import clean.
- Targeted tests (21 files incl. credential_pool, anthropic plugin,
run_agent, project_metadata): pass.
- 3 credential_pool security tests rewired to register the real plugin hook
(core tests don't trigger plugin discovery): pass.
- Full suite: remaining failures are pre-existing on premerge-oh-god (telegram
collection error: setUpModule MagicMock __code__) or environmental (matrix
DNS/e2ee-deps, network) — NOT introduced by this merge. See PR notes.
The long-lived prefix-cache layout split the system prompt into stable/
context/volatile blocks and re-derived them on every API call. The
volatile tier (timestamp + memory snapshot + USER profile) ticks per
turn, so the system message bytes mutated mid-conversation and broke
upstream prompt caches (OpenRouter, Nous Portal, Anthropic).
Diagnosed via live wire-format diffing: an 8-turn conversation showed
OLD layout flipping system block[1] sha mid-session at the minute
boundary, dropping cached_tokens to 0 on that turn (cumulative
66.6% vs 83.3% for the single-block layout). Hermes invariant:
history (system + all but the last 1-2 messages) must be static.
Fix: drop the long-lived layout entirely. Single layout everywhere —
system_and_3 with one cached system string built once on first turn,
replayed verbatim on every subsequent turn. Loses cross-session 1h
prefix caching for Claude (the feature that motivated the split), but
within-session caching now actually works on every provider.
Removed:
- run_agent.py: _use_long_lived_prefix_cache flag, _long_lived_cache_ttl,
_supports_long_lived_anthropic_cache method, the long-lived branch in
run_conversation, mark_tools_for_long_lived_cache call site
- agent/prompt_caching.py: apply_anthropic_cache_control_long_lived,
mark_tools_for_long_lived_cache, _mark_system_stable_block helper
- hermes_cli/config.py: prompt_caching.long_lived_prefix and
prompt_caching.long_lived_ttl config keys
- tests/agent/test_prompt_caching_live.py (entire file)
- tests/agent/test_prompt_caching.py: TestMarkToolsForLongLivedCache,
TestApplyAnthropicCacheControlLongLived
- tests/run_agent/test_anthropic_prompt_cache_policy.py:
TestSupportsLongLivedAnthropicCache
Targeted tests: 62/62 pass.
When the API returns "max_tokens too large given prompt" (input tokens
are within the context window, but input + requested output > window),
the old code incorrectly routed through the same handler as "prompt too
long" errors, calling get_next_probe_tier() and permanently halving
context_length. This made things worse: the window was fine, only the
requested output size needed trimming for that one call.
Two distinct error classes now handled separately:
Prompt too long — input itself exceeds context window.
Fix: compress history + halve context_length (existing behaviour,
unchanged).
Output cap too large — input OK, but input + max_tokens > window.
Fix: parse available_tokens from the error message, set a one-shot
_ephemeral_max_output_tokens override for the retry, and leave
context_length completely untouched.
Changes:
- agent/model_metadata.py: add parse_available_output_tokens_from_error()
that detects Anthropic's "available_tokens: N" error format and returns
the available output budget, or None for all other error types.
- run_agent.py: call the new parser first in the is_context_length_error
block; if it fires, set _ephemeral_max_output_tokens (with a 64-token
safety margin) and break to retry without touching context_length.
_build_api_kwargs consumes the ephemeral value exactly once then clears
it so subsequent calls use self.max_tokens normally.
- agent/anthropic_adapter.py: expand build_anthropic_kwargs docstring to
clearly document the max_tokens (output cap) vs context_length (total
window) distinction, which is a persistent source of confusion due to
the OpenAI-inherited "max_tokens" name.
- cli-config.yaml.example: add inline comments explaining both keys side
by side where users are most likely to look.
- website/docs/integrations/providers.md: add a callout box at the top
of "Context Length Detection" and clarify the troubleshooting entry.
- tests/test_ctx_halving_fix.py: 24 tests across four classes covering
the parser, build_anthropic_kwargs clamping, ephemeral one-shot
consumption, and the invariant that context_length is never mutated
on output-cap errors.
