Compare commits

...
Author SHA1 Message Date
Teknium 2e48537cf1 chore(release): map xxxigm author email 2026-06-13 14:58:19 -07:00
xxxigm af1477d812 fix(codex): bound leaked tool-call scan to prefix window 2026-06-13 14:58:19 -07:00
Teknium bf8effad02 fix(utils): copy fallback for atomic replace across devices (#43852)
Fallback from `os.replace` on EXDEV/EBUSY using copy+fsync+unlink while preserving symlink target semantics and metadata.
2026-06-13 14:50:05 -07:00
Teknium 817f392311 feat(read): extract notebook and office documents (#37082)
Add stdlib-only extraction for `.ipynb`, `.docx`, and `.xlsx` in read_file with lazy integration and malformed-document fallback.
2026-06-13 14:42:51 -07:00
Teknium 2b67e96aec fix(approval): gate in-place edits to sensitive user files
Cover sed, perl, and ruby in-place mutations against shell rc, SSH, and credential files so terminal approvals pair the redirection and copy guards.
2026-06-13 14:35:27 -07:00
helix4u abd69b8117 fix(approval): detect absolute home shell rc writes 2026-06-13 14:35:27 -07:00
briandevans da28d5d113 fix(security): gate cp/mv/install into ~/.ssh, credential, and shell-rc files
tools/approval.py already denies tee/redirection writes to every
_SENSITIVE_WRITE_TARGET (~/.ssh/*, ~/.netrc/.pgpass/.npmrc/.pypirc, shell
rc files, ~/.hermes/config.yaml/.env) via the DANGEROUS_PATTERNS tee/`>`
rules, but cp/mv/install were only paired for _SYSTEM_CONFIG_PATH (/etc) and
the project-relative env/config target. So `cp evil ~/.ssh/authorized_keys`
(SSH-key implant / persistence), `cp creds ~/.netrc`, and `cp evil ~/.bashrc`
(login-time command injection) auto-approved while the equivalent tee/`>`
forms were denied — an unpaired write deny is theater (same rationale as
#14639 / commit 4e9d886d, which paired the terminal side for
~/.hermes/config.yaml writes but did not touch these cp/mv/install verbs on
the broader sensitive set).

Add one (cp|mv|install) DANGEROUS_PATTERNS entry reusing the existing
_SENSITIVE_WRITE_TARGET fragment, anchored via _COMMAND_TAIL so it fires on
the destination (last arg) only: reading OUT of a sensitive path
(`cp ~/.ssh/config /tmp/x`) stays auto-approved. Description differs from the
system-config cp entry so the two keep distinct approval keys (no silent
cross-approval). Additive — does not subsume the /etc or project-config rules.

Adds TestSensitiveCopyMovePattern: 5 positive cases (ssh authorized_keys,
ssh private key via mv, netrc via install, bashrc, ~/.hermes/config.yaml) +
2 negative guards (copy FROM ssh, unrelated copy). The ssh/netrc/bashrc
positives fail on main and pass on this branch; the negatives stay green
both ways.
2026-06-13 14:35:27 -07:00
Teknium 1fa761f8de fix(search): keep partial results on search timeout (#36142)
Treat search command budget timeouts as soft truncation so partial results survive, while real search failures still return structured errors.
2026-06-13 14:35:21 -07:00
Teknium 069bfd6545 fix(agent): keep Codex reasoning replay on Codex path 2026-06-13 14:35:00 -07:00
briandevans 1d584a301e fix(agent): treat Codex reasoning items as thinking-only 2026-06-13 14:35:00 -07:00
ITheEqualizer 57c2a55be4 fix(telegram): harden rich message fallback handling
Carry forward focused follow-ups from PR #45741: treat PTB's raw Bot API 10.1 response shapes safely, recognize real missing-endpoint errors, preserve link preview settings on rich sends, and lock the rich limit to Telegram's character-based cap.
2026-06-13 14:34:53 -07:00
brooklyn! 0a865e5948 fix(desktop): bypass Chromium editing pipeline for large paste & select-delete (#45812)
Large paste and Ctrl+A → Delete froze the composer for seconds — both routed
through Chromium's contenteditable editing pipeline (~O(n²) on multiline DOM).

- insertPlainTextAtCaret: Range + text/<br> fragment (paste path)
- deleteSelectionInEditor: range.deleteContents for non-collapsed Backspace/Delete
- Shared composerSelectionRange helper; both flush via flushEditorToDraft

Profiled live (47 KB / 122 paragraphs): paste 4474 ms → 13 ms; select-delete
1304 ms → 4 ms. Collapsed-caret deletes still native.
2026-06-13 20:49:58 +00:00
Teknium c8e5f34f24 fix(gemini): strip native self prefixes before generateContent (#36141)
Strip `google/` and `gemini/` self-prefixes before native Gemini generateContent calls, and keep provider-normalization expectations aligned.
2026-06-13 13:47:08 -07:00
briandevans 7d11fa4e9e fix(codex-responses): let final_answer complete top-level incomplete responses 2026-06-13 13:45:29 -07:00
ITheEqualizer 7c0605bf22 fix(telegram): preserve rich formatting on stream final 2026-06-13 13:44:45 -07:00
achaljhawar 819def44c7 fix(agent): scope Nous tags to Nous auxiliary calls 2026-06-13 13:24:40 -07:00
Teknium 08890d77e6 fix(plugins): normalize browser-pasted GitHub repo URLs (#33539)
Accept common GitHub web URLs in `hermes plugins install` by normalizing repository views back to cloneable `.git` URLs, with focused parser coverage.
2026-06-13 13:23:59 -07:00
brooklyn! 425e777f54 fix(desktop): polish slash command completion (space/tab/click + typed args) (#45760)
* fix(desktop): accept slash command on space at command stage

Pressing space on a no-arg slash command (e.g. /hermes-agent) fell
through to the arg-completion stage and dead-ended on "No matches"
instead of inserting the directive. Space now mirrors Tab/Enter while
the command name is still being typed: no-arg commands commit the chip,
arg-taking commands expand to their options step.

* fix(desktop): suppress arg popover for no-arg slash commands

Committing a no-arg command (`/hermes-agent `) re-detected the chip+space
as an arg query and re-opened the popover on "No matches". The arg-stage
menu now only opens when the command actually takes args.

* fix(desktop): polish slash arg completion (space/tab/click + typed args)

Unify Enter/Tab/Space accept of the highlighted item at both the command
and arg stages: no-arg commands commit a chip, arg commands expand to
options, and an arg option commits the full `/cmd arg` chip. A fully-typed
arg (which the backend completer drops from suggestions) now commits on
Space/Tab via the verbatim text instead of dead-ending, and the "No
matches" empty state is suppressed past a command's name. Space stays
slash-only so @ mentions keep a literal space.
2026-06-13 18:43:52 +00:00
kshitij 7be22e37e1 Merge pull request #45753 from kshitijk4poor/salvage/gateway-auto-resume-duplicate-agent
fix(gateway): claim session slot before auto-resume task to prevent duplicate agents (#45456)
2026-06-13 23:46:17 +05:30
kshitijk4poor 28902dc890 chore: map liuhao1024 contributor email for attribution 2026-06-13 23:39:49 +05:30
kshitijk4poor 63097ee0d7 test(gateway): cover auto-resume full-path no-regression; clarify guard docstring
The salvaged fix's two regression tests mock adapter.handle_message, so
they only assert the pre-claimed sentinel is set/cleaned around a stub —
they never drive the real dispatch chain. Add a full-path test that
exercises _schedule_resume_pending_sessions -> _guarded_handle_message ->
adapter.handle_message -> _process_message_background -> _handle_message
and asserts the resumed session's agent runs EXACTLY ONCE: not zero (the
pre-claim must not self-bounce the resume into a queued no-op) and not
twice (the duplicate-agent bug #45456 the fix targets). Also assert no
leaked sentinel and no orphaned pending event after the drain settles.

Tighten the _guarded_handle_message docstring: on current main the real
sentinel is taken over inside _handle_message (not _process_message_background),
and note the `is _AGENT_PENDING_SENTINEL` guard only releases the slot we
ourselves placed, never one a live run owns.
2026-06-13 23:39:35 +05:30
liuhao1024 6e2fd955ca fix(gateway): claim session slot before auto-resume task to prevent duplicate agents
When the gateway restarts and auto-resumes an interrupted session, an
inbound message arriving in the window between `asyncio.create_task()`
and the task's first await could spin up a second AIAgent for the same
session.  Both agents would then process messages concurrently,
producing interleaved duplicate responses (#45456).

Fix: set `_AGENT_PENDING_SENTINEL` in `_running_agents` immediately
after the "already running" check, before creating the task.  This
closes the race window — any inbound message sees the slot as occupied
and queues behind the auto-resume.

A `_guarded_handle_message` wrapper ensures the pre-claimed sentinel is
always released, even if `handle_message` raises before reaching
`_process_message_background` (whose `finally` block handles normal
cleanup).

(cherry picked from commit 85150c976b)
2026-06-13 23:36:51 +05:30
helix4u 78c11d99e3 fix(update): stop Windows gateways before mutating install 2026-06-13 10:46:08 -07:00
ashishpatel26andTranquil-Flow 957a8ffa88 fix(bedrock): omit sampling params for restricted Claude models
Bedrock Converse rejects non-default sampling parameters for Opus 4.7 and 4.8 with a ValidationException. Reuse the Anthropic-native sampling-param guard in the Bedrock kwargs builder so those models omit temperature/topP while older Claude and non-Claude models keep existing behavior.

Includes the stop-sequence regression from the parallel fix to ensure stopSequences still pass through for restricted Opus models.

Co-authored-by: Tranquil-Flow <tranquil_flow@protonmail.com>
2026-06-13 10:45:56 -07:00
Teknium cc14b74718 docs(profile): update clone-from references 2026-06-13 07:33:58 -07:00
Teknium 9b5f7b63c6 fix(profile): make clone-from a full source selector 2026-06-13 07:33:58 -07:00
Teknium d146b85173 chore(release): map WompaJango author 2026-06-13 07:33:58 -07:00
WompaJango 28bf8fb47d feat(dashboard): clone profiles from any source 2026-06-13 07:33:58 -07:00
Que0x 3380563d94 fix(security): stop /api/status leaking host paths and PID on gated binds
The dashboard's public /api/status liveness endpoint is in PUBLIC_API_PATHS
and bypasses dashboard auth, yet it returned absolute hermes_home,
config_path, env_path, the gateway PID, and the internal gateway health URL.
That exceeds the shape its own allowlist documents as public ("version,
gateway state, active session count, and the dashboard auth-gate shape. No
bodies, no session content, no secrets"), leaking deployment recon to any
unauthenticated caller on a network-exposed (gated) bind.

Withhold host-local detail unless the bind is loopback / --insecure, where
the dashboard is local-only and the caller is already inside the trust
envelope -- the same split should_require_auth draws. The NAS liveness probe
and the auth-gate badge are unaffected.

Adds invariant tests for both modes (gated withholds, loopback keeps).
2026-06-13 07:18:59 -07:00
Teknium ad7436a5d9 fix(gateway): preserve WeCom per-group sender allowlists
Keep the own-policy fail-closed hardening from PR #45444, but still trust WeCom groups.<id>.allow_from because the adapter already checked that sender allowlist before dispatching to gateway auth.
2026-06-13 07:18:54 -07:00
Que0x fc46354580 fix(security): fail closed when an own-policy gateway adapter has no allowlist
Own-policy adapters (WhatsApp, WeCom, Weixin, QQBot, Yuanbao) default dm_policy/group_policy to "open", which forwards every sender. The gateway's adapter-trust shortcut in _is_user_authorized blanket-trusted those platforms when no env allowlist was set, so an operator who enabled one with only credentials authorized the entire external network -- the fail-open SECURITY.md section 2.6 forbids ("an allowlist is required for every enabled network-exposed adapter").

Trust the adapter only when its effective policy for the chat type is an actual "allowlist" restriction (the case #34515 was protecting). "open"/"pairing"/anything else falls through to default-deny, where {PLATFORM}_ALLOW_ALL_USERS / GATEWAY_ALLOW_ALL_USERS and the pairing flow remain the explicit opt-ins.
2026-06-13 07:18:54 -07:00
Teknium 1185dfd773 test: cover legacy Office document extensions 2026-06-13 07:18:37 -07:00
Clayton Chew f82cb48120 fix(platform): add .xls, .doc, .ppt to SUPPORTED_DOCUMENT_TYPES
Old Office formats (.xls, .doc, .ppt) were missing from the
SUPPORTED_DOCUMENT_TYPES dict in gateway/platforms/base.py while their
newer counterparts (.xlsx, .docx, .pptx) were included.

Sending an .xls file via Telegram triggers 'Unsupported document type'
and the file is silently dropped instead of being cached and forwarded
to the agent.

Add the three legacy MIME types so these files are handled the same way
as their modern equivalents.
2026-06-13 07:18:37 -07:00
Tranquil-Flow 4fd9397ae3 fix(codex): drop extra_headers for chatgpt.com backend 2026-06-13 07:13:24 -07:00
Sarvesh 45f9099e51 fix(matrix): preserve markdown table structure 2026-06-13 06:57:08 -07:00
Teknium 4373e802a1 fix(docs): reuse healthy skills index during Pages deploys (#45616) 2026-06-13 06:46:07 -07:00
Teknium d206e1f51d fix(dashboard): keep local file browser on home 2026-06-13 06:39:38 -07:00
95 changed files with 3685 additions and 361 deletions
+83 -16
View File
@@ -11,8 +11,20 @@ on:
- 'optional-skills/**'
- '.github/workflows/deploy-site.yml'
workflow_dispatch:
inputs:
skills_index_run_id:
description: 'Optional Build Skills Index run ID whose skills-index artifact should be deployed'
required: false
type: string
rebuild_skills_index:
description: 'Force a fresh multi-source crawl instead of reusing the latest healthy index'
required: false
default: false
type: boolean
permissions:
contents: read
actions: read
pages: write
id-token: write
@@ -55,26 +67,81 @@ jobs:
- name: Install PyYAML for skill extraction
run: pip install pyyaml==6.0.2 httpx==0.28.1
- name: Build skills index (unified multi-source catalog)
- name: Prepare skills index (unified multi-source catalog)
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_TOKEN: ${{ github.token }}
GITHUB_TOKEN: ${{ github.token }}
SKILLS_INDEX_RUN_ID: ${{ github.event.inputs.skills_index_run_id || '' }}
REBUILD_SKILLS_INDEX: ${{ github.event.inputs.rebuild_skills_index || 'false' }}
run: |
# Rebuild the unified catalog. The file is gitignored, so a fresh
# checkout starts without it and we want the freshest crawl in
# every deploy.
# The unified external catalog is expensive to crawl and can burn
# through the repository installation's GitHub API quota when several
# docs deploys land close together. Normal docs deploys therefore
# reuse the latest healthy catalog: first the artifact from a
# scheduled skills-index run, then the currently live index. Only a
# manual force rebuild does a fresh crawl here.
#
# This MUST be fatal. build_skills_index.py runs a health check and
# exits non-zero WITHOUT writing the output file when a source
# collapses (e.g. a GitHub API rate limit zeroes the github /
# claude-marketplace / well-known taps all at once). Letting the
# deploy continue would either (a) ship a degenerate index missing
# whole hubs — the June 2026 regression where OpenAI/Anthropic/
# HuggingFace/NVIDIA tabs vanished — or (b) fall through to a
# local-only catalog. Failing here keeps the last good deployment
# live (GitHub Pages serves the previous build) instead of
# publishing a broken catalog. Re-run the workflow once the
# transient rate limit clears.
# If we do crawl, the build remains fatal. build_skills_index.py runs
# the health check BEFORE writing and exits non-zero on source
# collapse, keeping the last good Pages deployment live instead of
# publishing a degenerate catalog.
set -euo pipefail
INDEX_PATH="website/static/api/skills-index.json"
mkdir -p "$(dirname "$INDEX_PATH")"
validate_index() {
python3 - "$INDEX_PATH" <<'PY'
import json
import sys
from pathlib import Path
path = Path(sys.argv[1])
try:
data = json.loads(path.read_text(encoding="utf-8"))
except Exception as exc:
print(f"invalid skills index JSON: {exc}", file=sys.stderr)
sys.exit(1)
skills = data.get("skills")
if not isinstance(skills, list) or len(skills) < 1500:
count = len(skills) if isinstance(skills, list) else "missing"
print(f"skills index too small: {count}", file=sys.stderr)
sys.exit(1)
print(f"skills index ready: {len(skills)} skills")
PY
}
if [ "$REBUILD_SKILLS_INDEX" = "true" ]; then
python3 scripts/build_skills_index.py
validate_index
exit 0
fi
if [ -n "$SKILLS_INDEX_RUN_ID" ]; then
tmpdir="$(mktemp -d)"
echo "Downloading skills-index artifact from run $SKILLS_INDEX_RUN_ID"
if gh run download "$SKILLS_INDEX_RUN_ID" --name skills-index --dir "$tmpdir"; then
candidate="$(find "$tmpdir" -name skills-index.json -type f | head -n 1 || true)"
if [ -n "$candidate" ]; then
cp "$candidate" "$INDEX_PATH"
if validate_index; then
exit 0
fi
fi
fi
echo "::warning::Could not use skills-index artifact from run $SKILLS_INDEX_RUN_ID; trying live index"
fi
echo "Downloading currently live skills index"
if curl -fsSL --retry 3 --retry-delay 5 \
"https://hermes-agent.nousresearch.com/docs/api/skills-index.json" \
-o "$INDEX_PATH" && validate_index; then
exit 0
fi
echo "::warning::Live skills index unavailable or unhealthy; falling back to a fresh crawl"
rm -f "$INDEX_PATH"
python3 scripts/build_skills_index.py
validate_index
- name: Extract skill metadata for dashboard
run: python3 website/scripts/extract-skills.py
+1 -1
View File
@@ -53,4 +53,4 @@ jobs:
- name: Trigger Deploy Site workflow
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: gh workflow run deploy-site.yml --repo ${{ github.repository }}
run: gh workflow run deploy-site.yml --repo ${{ github.repository }} -f skills_index_run_id=${{ github.run_id }}
+9 -1
View File
@@ -881,6 +881,8 @@ def try_recover_primary_transport(
def drop_thinking_only_and_merge_users(
messages: List[Dict[str, Any]],
*,
drop_codex_reasoning_items: bool = True,
) -> List[Dict[str, Any]]:
"""Drop thinking-only assistant turns; merge any adjacent user messages left behind.
@@ -902,7 +904,13 @@ def drop_thinking_only_and_merge_users(
return messages
# Pass 1: drop thinking-only assistant turns.
kept = [m for m in messages if not _ra().AIAgent._is_thinking_only_assistant(m)]
kept = [
m for m in messages
if not _ra().AIAgent._is_thinking_only_assistant(
m,
drop_codex_reasoning_items=drop_codex_reasoning_items,
)
]
dropped = len(messages) - len(kept)
if dropped == 0:
return messages
+1 -1
View File
@@ -5004,7 +5004,7 @@ def _build_call_kwargs(
# Provider-specific extra_body
merged_extra = dict(extra_body or {})
if provider == "nous" or auxiliary_is_nous:
if provider == "nous":
merged_extra.setdefault("tags", []).extend(_nous_portal_tags())
if merged_extra:
kwargs["extra_body"] = merged_extra
+7 -4
View File
@@ -935,11 +935,14 @@ def build_converse_kwargs(
if system_prompt:
kwargs["system"] = system_prompt
if temperature is not None:
kwargs["inferenceConfig"]["temperature"] = temperature
from agent.anthropic_adapter import _forbids_sampling_params
if top_p is not None:
kwargs["inferenceConfig"]["topP"] = top_p
if not _forbids_sampling_params(model):
if temperature is not None:
kwargs["inferenceConfig"]["temperature"] = temperature
if top_p is not None:
kwargs["inferenceConfig"]["topP"] = top_p
if stop_sequences:
kwargs["inferenceConfig"]["stopSequences"] = stop_sequences
+21 -2
View File
@@ -70,6 +70,21 @@ _TOOL_CALL_LEAK_PATTERN = re.compile(
r"(?:^|[\s>|])to=functions\.[A-Za-z_][\w.]*",
re.IGNORECASE,
)
_TOOL_CALL_LEAK_SCAN_LIMIT = 8192
def _scan_for_leaked_tool_call(text: str) -> bool:
"""Return True if Codex leaked a Harmony tool-call marker near the start.
Real leaked tool-call serializations begin with the Harmony marker. Bound
the regex to a prefix window so multi-megabyte successful assistant output
never spends unbounded GIL time proving it does not contain a leak.
"""
return bool(
_TOOL_CALL_LEAK_PATTERN.search(
text[:_TOOL_CALL_LEAK_SCAN_LIMIT + 64]
)
)
# ---------------------------------------------------------------------------
@@ -1081,6 +1096,7 @@ def _normalize_codex_response(
message_items_raw: List[Dict[str, Any]] = []
tool_calls: List[Any] = []
has_incomplete_items = response_status in {"queued", "in_progress", "incomplete"}
saw_streaming_or_item_incomplete = response_status in {"queued", "in_progress"}
saw_commentary_phase = False
saw_final_answer_phase = False
saw_reasoning_item = False
@@ -1095,6 +1111,7 @@ def _normalize_codex_response(
if item_status in {"queued", "in_progress", "incomplete"}:
has_incomplete_items = True
saw_streaming_or_item_incomplete = True
if item_type == "message":
item_phase = getattr(item, "phase", None)
@@ -1225,7 +1242,7 @@ def _normalize_codex_response(
# ``function_call`` item. The existing loop already handles message
# append, dedup, and retry budget.
leaked_tool_call_text = False
if final_text and not tool_calls and _TOOL_CALL_LEAK_PATTERN.search(final_text):
if final_text and not tool_calls and _scan_for_leaked_tool_call(final_text):
leaked_tool_call_text = True
logger.warning(
"Codex response contains leaked tool-call text in assistant content "
@@ -1252,7 +1269,9 @@ def _normalize_codex_response(
finish_reason = "tool_calls"
elif leaked_tool_call_text:
finish_reason = "incomplete"
elif has_incomplete_items or (saw_commentary_phase and not saw_final_answer_phase):
elif saw_streaming_or_item_incomplete:
finish_reason = "incomplete"
elif (has_incomplete_items or saw_commentary_phase) and not saw_final_answer_phase:
finish_reason = "incomplete"
elif (reasoning_items_raw or reasoning_parts or saw_reasoning_item) and not final_text:
# Response contains only reasoning (encrypted thinking state and/or
+4 -1
View File
@@ -707,7 +707,10 @@ def run_conversation(
# a thinking-only turn. Runs on the per-call copy only — the
# stored conversation history keeps the reasoning block for the
# UI transcript and session persistence.
api_messages = agent._drop_thinking_only_and_merge_users(api_messages)
api_messages = agent._drop_thinking_only_and_merge_users(
api_messages,
drop_codex_reasoning_items=agent.api_mode != "codex_responses",
)
# Normalize message whitespace and tool-call JSON for consistent
# prefix matching. Ensures bit-perfect prefixes across turns,
+11
View File
@@ -41,6 +41,16 @@ DEFAULT_GEMINI_BASE_URL = "https://generativelanguage.googleapis.com/v1beta"
GEMINI_DEFAULT_MAX_OUTPUT_TOKENS = 65535
def bare_gemini_model_id(model: str) -> str:
"""Strip Gemini's own provider prefix from an aggregator-style model id."""
name = (model or "").strip()
lowered = name.lower()
for prefix in ("google/", "gemini/"):
if lowered.startswith(prefix):
return name[len(prefix):].strip() or name
return name
def is_native_gemini_base_url(base_url: str) -> bool:
"""Return True when the endpoint speaks Gemini's native REST API."""
normalized = str(base_url or "").strip().rstrip("/").lower()
@@ -914,6 +924,7 @@ class GeminiNativeClient:
thinking_config=thinking_config,
)
model = bare_gemini_model_id(model)
if stream:
return self._stream_completion(model=model, request=request, timeout=timeout)
+13 -7
View File
@@ -511,13 +511,19 @@ PLATFORM_HINTS = {
"Standard Markdown is automatically converted to Telegram formatting. "
"Supported: **bold**, *italic*, ~~strikethrough~~, ||spoiler||, "
"`inline code`, ```code blocks```, [links](url), and ## headers. "
"Telegram supports rich Markdown, so when it improves clarity you may "
"use headings, tables (pipe `| col | col |` syntax), task lists "
"(`- [ ]` / `- [x]`), nested blockquotes, collapsible details, "
"footnotes/references, math/formulas (`$...$`, `$$...$$`), underline, "
"subscript/superscript, marked (highlighted) text, and anchors. Prefer "
"real Markdown tables and task lists over hand-built bullet substitutes "
"when presenting structured data. "
"Telegram now supports rich Markdown, so lean into it: whenever it "
"makes the answer clearer or easier to scan, actively reach for real "
"Markdown tables (pipe `| col | col |` syntax), bullet and numbered "
"lists, task lists (`- [ ]` / `- [x]`), headings, nested blockquotes, "
"collapsible details, footnotes/references, math/formulas (`$...$`, "
"`$$...$$`), underline, subscript/superscript, marked (highlighted) "
"text, and anchors. Default to structured formatting over dense "
"paragraphs for any comparison, set of steps, key/value summary, or "
"tabular data. Prefer real Markdown tables and task lists over "
"hand-built bullet substitutes when presenting structured data; these "
"degrade gracefully (tables become readable bullet groups) when rich "
"rendering is unavailable, but advanced constructs like math and "
"collapsible details may render as plain source text in that case. "
"You can send media files natively: to deliver a file to the user, "
"include MEDIA:/absolute/path/to/file in your response. Images "
"(.png, .jpg, .webp) appear as photos, audio (.ogg) sends as voice "
+4 -16
View File
@@ -218,22 +218,10 @@ class ResponsesApiTransport(ProviderTransport):
kwargs.pop("timeout", None)
if is_codex_backend:
prompt_cache_key = kwargs.get("prompt_cache_key")
cache_scope_id = str(prompt_cache_key or session_id or "").strip()
if cache_scope_id:
existing_extra_headers = kwargs.get("extra_headers")
merged_extra_headers: Dict[str, str] = {}
if isinstance(existing_extra_headers, dict):
merged_extra_headers.update(
{
str(key): str(value)
for key, value in existing_extra_headers.items()
if key and value is not None
}
)
merged_extra_headers["session_id"] = cache_scope_id
merged_extra_headers["x-client-request-id"] = cache_scope_id
kwargs["extra_headers"] = merged_extra_headers
# chatgpt.com/backend-api/codex rejects body-level
# ``extra_headers`` with HTTP 400. Correlation/cache routing for
# this backend must not be sent through the Responses payload.
kwargs.pop("extra_headers", None)
max_tokens = params.get("max_tokens")
if max_tokens is not None and not is_codex_backend:
+122 -45
View File
@@ -85,6 +85,8 @@ import {
import { QueuePanel } from './queue-panel'
import {
composerPlainText,
deleteSelectionInEditor,
insertPlainTextAtCaret,
normalizeComposerEditorDom,
placeCaretEnd,
refChipElement,
@@ -135,6 +137,12 @@ function slashChipKindForItem(item: Unstable_TriggerItem): SlashChipKind {
return 'command'
}
/** A `/` query is at its arg stage once it's past the command name. */
const slashArgStage = (query: string) => query.includes(' ')
/** The `/command` token of a slash query (`personality x` → `/personality`). */
const slashCommandToken = (query: string) => `/${query.split(/\s+/, 1)[0]?.toLowerCase() ?? ''}`
interface QueueEditState {
attachments: ComposerAttachment[]
draft: string
@@ -532,48 +540,6 @@ export function ChatBar({
})
}, [])
const handlePaste = (event: ClipboardEvent<HTMLDivElement>) => {
const imageBlobs = extractClipboardImageBlobs(event.clipboardData)
if (imageBlobs.length > 0) {
event.preventDefault()
if (onAttachImageBlob) {
triggerHaptic('selection')
for (const blob of imageBlobs) {
void onAttachImageBlob(blob)
}
}
return
}
// Trim surrounding whitespace so a copy that dragged along leading/trailing
// blank lines (common when selecting from terminals, code blocks, web pages)
// doesn't dump multiline padding into the composer. Internal newlines are
// preserved — only the edges are cleaned up.
const pastedText = event.clipboardData.getData('text').trim()
if (!pastedText) {
event.preventDefault()
return
}
if (DATA_IMAGE_URL_RE.test(pastedText)) {
event.preventDefault()
return
}
event.preventDefault()
document.execCommand('insertText', false, pastedText)
const nextDraft = composerPlainText(event.currentTarget)
draftRef.current = nextDraft
aui.composer().setText(nextDraft)
}
const [trigger, setTrigger] = useState<TriggerState | null>(null)
const [triggerActive, setTriggerActive] = useState(0)
const [triggerItems, setTriggerItems] = useState<readonly Unstable_TriggerItem[]>([])
@@ -610,7 +576,15 @@ export function ChatBar({
}
const before = textBeforeCaret(editor)
const detected = detectTrigger(before ?? composerPlainText(editor))
const found = detectTrigger(before ?? composerPlainText(editor))
// The arg-stage popover is only useful for commands with an options screen.
// For a no-arg command it would dead-end on "No matches", so drop it — the
// directive is already complete.
const detected =
found?.kind === '/' && slashArgStage(found.query) && !desktopSlashCommandTakesArgs(slashCommandToken(found.query))
? null
: found
setTrigger(detected)
@@ -650,6 +624,46 @@ export function ChatBar({
flushEditorToDraft(event.currentTarget)
}
const handlePaste = (event: ClipboardEvent<HTMLDivElement>) => {
const imageBlobs = extractClipboardImageBlobs(event.clipboardData)
if (imageBlobs.length > 0) {
event.preventDefault()
if (onAttachImageBlob) {
triggerHaptic('selection')
for (const blob of imageBlobs) {
void onAttachImageBlob(blob)
}
}
return
}
// Trim surrounding whitespace so a copy that dragged along leading/trailing
// blank lines (common when selecting from terminals, code blocks, web pages)
// doesn't dump multiline padding into the composer. Internal newlines are
// preserved — only the edges are cleaned up.
const pastedText = event.clipboardData.getData('text').trim()
if (!pastedText) {
event.preventDefault()
return
}
if (DATA_IMAGE_URL_RE.test(pastedText)) {
event.preventDefault()
return
}
event.preventDefault()
insertPlainTextAtCaret(event.currentTarget, pastedText)
flushEditorToDraft(event.currentTarget)
}
const triggerAdapter: Unstable_TriggerAdapter | null =
trigger?.kind === '@' ? at.adapter : trigger?.kind === '/' ? slash.adapter : null
@@ -665,6 +679,12 @@ export function ChatBar({
const triggerLoading = trigger?.kind === '@' ? at.loading : trigger?.kind === '/' ? slash.loading : false
// Suppress the "No matches" empty state once a slash command is past its name:
// a no-arg command has nothing to offer, and a fully-typed arg commits on
// Space/Tab — neither should dead-end on a popover.
const argStageEmpty =
trigger?.kind === '/' && slashArgStage(trigger.query) && !triggerLoading && !triggerItems.length
const closeTrigger = () => {
setTrigger(null)
setTriggerItems([])
@@ -675,6 +695,25 @@ export function ChatBar({
setTriggerActive(idx => Math.min(idx, Math.max(0, triggerItems.length - 1)))
}, [triggerItems.length])
// Commit the literally-typed `/command arg` as a directive chip — used when
// the completion list is empty because the arg is already fully typed (the
// backend completer drops exact matches). Reuses the chip path via a
// synthetic item whose serialized form is the verbatim text.
const commitTypedSlashDirective = () => {
if (trigger?.kind !== '/') {
return
}
const text = `/${trigger.query.trimEnd()}`
replaceTriggerWithChip({
id: text,
type: 'slash',
label: text.slice(1),
metadata: { command: slashCommandToken(trigger.query), display: text, meta: '', group: '', action: '', rawText: text }
})
}
const replaceTriggerWithChip = (item: Unstable_TriggerItem) => {
const editor = editorRef.current
@@ -793,6 +832,18 @@ export function ChatBar({
return
}
// Non-collapsed Backspace/Delete: native selection-delete is ~O(n²) on large
// drafts (Ctrl+A → Delete froze ~1.3s). Collapsed carets fall through.
if (
(event.key === 'Backspace' || event.key === 'Delete') &&
deleteSelectionInEditor(event.currentTarget)
) {
event.preventDefault()
flushEditorToDraft(event.currentTarget)
return
}
// Cmd/Ctrl+Shift+K drains the next queued message. Plain Cmd/Ctrl+K is
// reserved for the global command palette.
if ((event.metaKey || event.ctrlKey) && !event.altKey && event.shiftKey && event.key.toLowerCase() === 'k') {
@@ -822,7 +873,15 @@ export function ChatBar({
return
}
if (event.key === 'Enter' || event.key === 'Tab') {
// Enter / Tab / Space all accept the highlighted item: a no-arg command
// commits its directive chip, an arg-taking command expands to its
// options step, and an arg option commits the full `/cmd arg` chip. Space
// is slash-only (an `@` mention takes a literal space) and gated to a
// non-empty query so a bare `/ ` still types a space.
const acceptOnSpace = event.key === ' ' && trigger.kind === '/' && Boolean(trigger.query.trim())
const accept = event.key === 'Enter' || event.key === 'Tab' || acceptOnSpace
if (accept) {
event.preventDefault()
triggerKeyConsumedRef.current = true
const item = triggerItems[triggerActive]
@@ -843,6 +902,24 @@ export function ChatBar({
}
}
// Arg stage with nothing left to suggest — a fully-typed arg the backend
// completer no longer echoes (it drops the exact match), e.g.
// `/personality creative`. Space/Tab still commit what's typed as a single
// directive chip; Enter falls through to submit (send it as-is).
if (
trigger?.kind === '/' &&
!triggerItems.length &&
(event.key === ' ' || event.key === 'Tab') &&
slashArgStage(trigger.query) &&
trigger.query.trim()
) {
event.preventDefault()
triggerKeyConsumedRef.current = true
commitTypedSlashDirective()
return
}
// ArrowUp/ArrowDown navigate, in priority order: the queue (edit entries in
// place) then sent-message history. The history ring is derived from live
// session messages each press — single source of truth, no mirror.
@@ -1765,7 +1842,7 @@ export function ChatBar({
ref={composerRef}
>
{showHelpHint && <HelpHint />}
{trigger && (
{trigger && !argStageEmpty && (
<ComposerTriggerPopover
activeIndex={triggerActive}
items={triggerItems}
@@ -3,12 +3,24 @@ import { describe, expect, it } from 'vitest'
import { insertInlineRefsIntoEditor } from './inline-refs'
import {
composerPlainText,
deleteSelectionInEditor,
insertPlainTextAtCaret,
normalizeComposerEditorDom,
refChipElement,
renderComposerContents,
RICH_INPUT_SLOT
} from './rich-editor'
const caretIn = (editor: HTMLElement) => {
const range = document.createRange()
const selection = window.getSelection()!
range.selectNodeContents(editor)
range.collapse(false)
selection.removeAllRanges()
selection.addRange(range)
}
describe('renderComposerContents', () => {
it('renders refs and raw text without interpreting user text as HTML', () => {
const editor = document.createElement('div')
@@ -59,3 +71,64 @@ describe('insertInlineRefsIntoEditor', () => {
expect(composerPlainText(editor)).toBe('@file:`src/foo.ts` ')
})
})
describe('insertPlainTextAtCaret', () => {
it('inserts multiline text as text nodes + br', () => {
const editor = document.createElement('div')
editor.dataset.slot = RICH_INPUT_SLOT
document.body.append(editor)
caretIn(editor)
insertPlainTextAtCaret(editor, 'one\ntwo\nthree')
expect(editor.querySelectorAll('br').length).toBe(2)
expect(composerPlainText(editor)).toBe('one\ntwo\nthree')
editor.remove()
})
it('replaces the selected span', () => {
const editor = document.createElement('div')
editor.dataset.slot = RICH_INPUT_SLOT
editor.textContent = 'abXYef'
document.body.append(editor)
const text = editor.firstChild!
const selection = window.getSelection()!
const range = document.createRange()
range.setStart(text, 2)
range.setEnd(text, 4)
selection.removeAllRanges()
selection.addRange(range)
insertPlainTextAtCaret(editor, 'cd')
expect(composerPlainText(editor)).toBe('abcdef')
editor.remove()
})
})
describe('deleteSelectionInEditor', () => {
it('clears a non-collapsed range and leaves a collapsed caret', () => {
const editor = document.createElement('div')
editor.dataset.slot = RICH_INPUT_SLOT
editor.textContent = 'hello world'
document.body.append(editor)
const selection = window.getSelection()!
const range = document.createRange()
range.selectNodeContents(editor)
selection.removeAllRanges()
selection.addRange(range)
expect(deleteSelectionInEditor(editor)).toBe(true)
expect(composerPlainText(editor)).toBe('')
expect(selection.getRangeAt(0).collapsed).toBe(true)
expect(deleteSelectionInEditor(editor)).toBe(false)
editor.remove()
})
})
@@ -132,6 +132,63 @@ export function renderComposerContents(target: HTMLElement, text: string) {
appendComposerContents(target, text)
}
/** Caret range when the selection lives inside `editor`; else null. */
function composerSelectionRange(editor: HTMLElement) {
const selection = window.getSelection()
const range = selection?.rangeCount ? selection.getRangeAt(0) : null
if (!selection || !range || !editor.contains(range.commonAncestorContainer)) {
return null
}
return { range, selection }
}
/** Insert plain text at the caret (replacing any selection). Pastes use this
* instead of `execCommand('insertText')` Chromium's editing pipeline is
* ~O(n²) on large multiline blobs. */
export function insertPlainTextAtCaret(editor: HTMLElement, text: string) {
const hit = composerSelectionRange(editor)
const fragment = document.createDocumentFragment()
appendTextWithBreaks(fragment, text)
const tail = fragment.lastChild
if (hit) {
hit.range.deleteContents()
hit.range.insertNode(fragment)
} else {
editor.append(fragment)
}
if (tail) {
const caret = document.createRange()
caret.setStartAfter(tail)
caret.collapse(true)
const selection = hit?.selection ?? window.getSelection()
selection?.removeAllRanges()
selection?.addRange(caret)
}
}
/** Remove a non-collapsed selection in-editor. Skips collapsed carets so word/
* line delete (Opt/Cmd+Backspace) stays native. Returns whether anything ran. */
export function deleteSelectionInEditor(editor: HTMLElement) {
const hit = composerSelectionRange(editor)
if (!hit || hit.range.collapsed) {
return false
}
hit.range.deleteContents()
hit.range.collapse(true)
hit.selection.removeAllRanges()
hit.selection.addRange(hit.range)
return true
}
/** Serialize a draft string into chip-HTML for the contenteditable surface. */
export function composerHtml(text: string) {
let cursor = 0
@@ -284,6 +284,7 @@ export function ProfileRail() {
selectProfile(name)
}}
open={createOpen}
profiles={profiles}
/>
<RenameProfileDialog
@@ -2,14 +2,15 @@ import { useEffect, useState } from 'react'
import { ActionStatus } from '@/components/ui/action-status'
import { Button } from '@/components/ui/button'
import { Checkbox } from '@/components/ui/checkbox'
import { Dialog, DialogContent, DialogDescription, DialogFooter, DialogHeader, DialogTitle } from '@/components/ui/dialog'
import { Input } from '@/components/ui/input'
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from '@/components/ui/select'
import { Textarea } from '@/components/ui/textarea'
import { createProfile, updateProfileSoul } from '@/hermes'
import { useI18n } from '@/i18n'
import { AlertTriangle } from '@/lib/icons'
import { cn } from '@/lib/utils'
import type { ProfileInfo } from '@/types/hermes'
const PROFILE_NAME_RE = /^[a-z0-9][a-z0-9_-]{0,63}$/
@@ -23,16 +24,18 @@ export function isValidProfileName(name: string): boolean {
export function CreateProfileDialog({
onClose,
onCreated,
open
open,
profiles = []
}: {
onClose: () => void
onCreated?: (name: string) => Promise<void> | void
open: boolean
profiles?: ProfileInfo[]
}) {
const { t } = useI18n()
const p = t.profiles
const [name, setName] = useState('')
const [cloneFromDefault, setCloneFromDefault] = useState(true)
const [cloneFrom, setCloneFrom] = useState<null | string>('default')
const [soul, setSoul] = useState('')
const [status, setStatus] = useState<'done' | 'idle' | 'saving'>('idle')
const [error, setError] = useState<null | string>(null)
@@ -43,7 +46,7 @@ export function CreateProfileDialog({
}
setName('')
setCloneFromDefault(true)
setCloneFrom('default')
setSoul('')
setError(null)
setStatus('idle')
@@ -66,7 +69,7 @@ export function CreateProfileDialog({
setError(null)
try {
await createProfile({ name: trimmed, clone_from_default: cloneFromDefault })
await createProfile({ name: trimmed, clone_from: cloneFrom })
if (soul.trim()) {
await updateProfileSoul(trimmed, soul)
@@ -107,17 +110,25 @@ export function CreateProfileDialog({
</p>
</div>
<label className="flex cursor-pointer select-none items-start gap-2.5 px-0.5 py-1">
<Checkbox
checked={cloneFromDefault}
className="mt-0.5 shrink-0"
onCheckedChange={checked => setCloneFromDefault(checked === true)}
/>
<span className="grid gap-0.5 leading-snug">
<span className="text-sm font-medium">{p.cloneFromDefault}</span>
<span className="text-xs text-muted-foreground">{p.cloneFromDefaultDesc}</span>
</span>
</label>
<div className="grid gap-1.5">
<label className="text-xs font-medium" htmlFor="new-profile-clone-from">
{p.cloneFrom}
</label>
<Select onValueChange={value => setCloneFrom(value === '__none__' ? null : value)} value={cloneFrom ?? '__none__'}>
<SelectTrigger className="h-9 rounded-md" id="new-profile-clone-from">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="__none__">{p.cloneFromNone}</SelectItem>
{profiles.map(profile => (
<SelectItem key={profile.name} value={profile.name}>
{profile.name}
</SelectItem>
))}
</SelectContent>
</Select>
<p className="text-xs text-muted-foreground">{p.cloneFromDesc}</p>
</div>
<div className="grid gap-1.5">
<label className="text-xs font-medium" htmlFor="new-profile-soul">
@@ -127,7 +138,7 @@ export function CreateProfileDialog({
className="min-h-28 font-mono text-xs leading-5"
id="new-profile-soul"
onChange={event => setSoul(event.target.value)}
placeholder={p.soulPlaceholder(cloneFromDefault ? p.soulPlaceholderCloned : p.soulPlaceholderEmpty)}
placeholder={p.soulPlaceholder(cloneFrom ? p.soulPlaceholderCloned : p.soulPlaceholderEmpty)}
value={soul}
/>
</div>
+31 -20
View File
@@ -12,6 +12,7 @@ import {
DialogTitle
} from '@/components/ui/dialog'
import { Input } from '@/components/ui/input'
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from '@/components/ui/select'
import { Textarea } from '@/components/ui/textarea'
import {
createProfile,
@@ -82,14 +83,14 @@ export function ProfilesView({ onClose }: ProfilesViewProps) {
}, [profiles, selectedName])
const handleCreate = useCallback(
async (name: string, cloneFromDefault: boolean) => {
async (name: string, cloneFrom: null | string) => {
const trimmed = name.trim()
if (!isValidProfileName(trimmed)) {
throw new Error(p.nameHint)
}
await createProfile({ name: trimmed, clone_from_default: cloneFromDefault })
await createProfile({ name: trimmed, clone_from: cloneFrom })
notify({ kind: 'success', title: p.created, message: trimmed })
setSelectedName(trimmed)
await refresh()
@@ -180,8 +181,9 @@ export function ProfilesView({ onClose }: ProfilesViewProps) {
<CreateProfileDialog
onClose={() => setCreateOpen(false)}
onCreate={async (name, cloneFromDefault) => handleCreate(name, cloneFromDefault)}
onCreate={async (name, cloneFrom) => handleCreate(name, cloneFrom)}
open={createOpen}
profiles={profiles ?? []}
/>
<Dialog onOpenChange={open => !open && !deleting && setPendingDelete(null)} open={pendingDelete !== null}>
@@ -453,16 +455,18 @@ function SoulEditor({ profileName }: { profileName: string }) {
function CreateProfileDialog({
onClose,
onCreate,
open
open,
profiles
}: {
onClose: () => void
onCreate: (name: string, cloneFromDefault: boolean) => Promise<void>
onCreate: (name: string, cloneFrom: null | string) => Promise<void>
open: boolean
profiles: ProfileInfo[]
}) {
const { t } = useI18n()
const p = t.profiles
const [name, setName] = useState('')
const [cloneFromDefault, setCloneFromDefault] = useState(true)
const [cloneFrom, setCloneFrom] = useState<null | string>('default')
const [saving, setSaving] = useState(false)
const [error, setError] = useState<null | string>(null)
@@ -472,7 +476,7 @@ function CreateProfileDialog({
}
setName('')
setCloneFromDefault(true)
setCloneFrom('default')
setError(null)
setSaving(false)
}, [open])
@@ -493,7 +497,7 @@ function CreateProfileDialog({
setError(null)
try {
await onCreate(trimmed, cloneFromDefault)
await onCreate(trimmed, cloneFrom)
onClose()
} catch (err) {
setError(err instanceof Error ? err.message : p.failedCreate)
@@ -528,18 +532,25 @@ function CreateProfileDialog({
</p>
</div>
<label className="flex cursor-pointer items-center gap-2 rounded-md border border-border/40 bg-background/50 px-3 py-2 text-sm">
<input
checked={cloneFromDefault}
className="size-4 accent-primary"
onChange={event => setCloneFromDefault(event.target.checked)}
type="checkbox"
/>
<span>
<span className="font-medium">{p.cloneFromDefault}</span>
<span className="ml-2 text-xs text-muted-foreground">{p.cloneFromDefaultDesc}</span>
</span>
</label>
<div className="grid gap-1.5">
<label className="text-xs font-medium" htmlFor="new-profile-clone-from">
{p.cloneFrom}
</label>
<Select onValueChange={value => setCloneFrom(value === '__none__' ? null : value)} value={cloneFrom ?? '__none__'}>
<SelectTrigger className="h-9 rounded-md" id="new-profile-clone-from">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="__none__">{p.cloneFromNone}</SelectItem>
{profiles.map(profile => (
<SelectItem key={profile.name} value={profile.name}>
{profile.name}
</SelectItem>
))}
</SelectContent>
</Select>
<p className="text-xs text-muted-foreground">{p.cloneFromDesc}</p>
</div>
{error && (
<div className="flex items-start gap-2 rounded-md border border-destructive/30 bg-destructive/10 px-3 py-2 text-xs text-destructive">
+3
View File
@@ -903,6 +903,9 @@ export const en: Translations = {
deleting: 'Deleting...',
createDesc: 'Profiles are independent Hermes environments: separate config, skills, and SOUL.md.',
nameLabel: 'Name',
cloneFrom: 'Clone from',
cloneFromNone: 'None (blank)',
cloneFromDesc: 'Copies config, skills, and SOUL.md from the selected source profile.',
cloneFromDefault: 'Clone from default',
cloneFromDefaultDesc: 'Copy config, skills, and SOUL.md from your default profile.',
invalidName: hint => `Invalid name. ${hint}`,
+3
View File
@@ -1041,6 +1041,9 @@ export const ja = defineLocale({
deleting: '削除中...',
createDesc: 'プロファイルは独立した Hermes 環境です:設定、スキル、SOUL.md が別々になります。',
nameLabel: '名前',
cloneFrom: '複製元',
cloneFromNone: 'なし(空)',
cloneFromDesc: '選択したプロファイルから設定、スキル、SOUL.md をコピーします。',
cloneFromDefault: 'デフォルトプロファイルから設定を複製',
cloneFromDefaultDesc: 'デフォルトプロファイルから設定、スキル、SOUL.md をコピーします。',
invalidName: hint => `無効なプロファイル名。${hint}`,
+3
View File
@@ -695,6 +695,9 @@ export interface Translations {
deleting: string
createDesc: string
nameLabel: string
cloneFrom: string
cloneFromNone: string
cloneFromDesc: string
cloneFromDefault: string
cloneFromDefaultDesc: string
invalidName: (hint: string) => string
+3
View File
@@ -999,6 +999,9 @@ export const zhHant = defineLocale({
deleting: '刪除中…',
createDesc: '設定檔是獨立的 Hermes 環境:各自擁有獨立的設定、技能和 SOUL.md。',
nameLabel: '名稱',
cloneFrom: '複製來源',
cloneFromNone: '無(空白)',
cloneFromDesc: '從選取的來源設定檔複製設定、技能和 SOUL.md。',
cloneFromDefault: '從預設設定檔複製設定',
cloneFromDefaultDesc: '從您的預設設定檔複製設定、技能和 SOUL.md。',
invalidName: hint => `設定檔名稱無效。${hint}`,
+3
View File
@@ -1092,6 +1092,9 @@ export const zh: Translations = {
deleting: '删除中…',
createDesc: '配置档案是相互独立的 Hermes 环境:各自拥有独立的配置、技能和 SOUL.md。',
nameLabel: '名称',
cloneFrom: '克隆来源',
cloneFromNone: '无(空白)',
cloneFromDesc: '从选中的来源配置档案复制配置、技能和 SOUL.md。',
cloneFromDefault: '从默认档案克隆',
cloneFromDefaultDesc: '从你的默认配置档案复制配置、技能和 SOUL.md。',
invalidName: hint => `名称无效。${hint}`,
+1 -1
View File
@@ -470,7 +470,7 @@ export interface CronJobUpdates {
export interface ProfileCreatePayload {
clone_all?: boolean
clone_from?: string
clone_from?: null | string
clone_from_default?: boolean
name: string
no_skills?: boolean
+2
View File
@@ -85,6 +85,8 @@ Extend `ProfileCreate` and the create endpoint — no new endpoints, no rewrite:
```python
class ProfileCreate(BaseModel):
name: str
clone_from: Optional[str] = None
# Backward compatibility for older dashboard/desktop clients.
clone_from_default: bool = False
clone_all: bool = False
no_skills: bool = False
+126 -27
View File
@@ -37,10 +37,12 @@ class GatewayAuthorizationMixin:
Mirrors ``BasePlatformAdapter.enforces_own_access_policy``. Adapters
such as WeCom, Weixin, Yuanbao, QQBot, and WhatsApp evaluate their
documented ``dm_policy`` / ``group_policy`` / ``allow_from`` config before a
message is dispatched to the gateway, so a message that reaches
``_is_user_authorized`` has already been authorized by the adapter.
Defaults to ``False`` when the adapter is unknown or doesn't expose
the flag.
message is dispatched to the gateway. The flag alone is NOT "already
authorized": these adapters default to ``open``, which forwards every
sender, so ``_is_user_authorized`` only trusts the adapter when its
effective policy for the chat type is an actual ``allowlist`` restriction
(see that method). Defaults to ``False`` when the adapter is unknown or
doesn't expose the flag.
"""
if not platform:
return False
@@ -65,10 +67,11 @@ class GatewayAuthorizationMixin:
env var is not always bridged back into ``config.extra``) and falls
back to ``config.extra`` for bare runners built without a live adapter.
Used by ``_is_user_authorized`` to carve ``dm_policy: pairing`` out of
the adapter-trust shortcut: in pairing mode the adapter forwards the DM
so the gateway can run its pairing handshake, so "reached the gateway"
must not be read as "authorized".
Used by ``_is_user_authorized`` to decide whether an own-policy adapter
actually restricted DM senders to a configured allowlist (trustworthy)
or merely forwarded everyone under ``dm_policy: open`` / for a pairing
handshake (not authorization). "Reached the gateway" only carries an
authorization signal in the ``allowlist`` case.
"""
if not platform:
return ""
@@ -87,6 +90,89 @@ class GatewayAuthorizationMixin:
policy = extra.get("dm_policy")
return str(policy or "").strip().lower()
def _adapter_group_policy(self, platform: Optional[Platform]) -> str:
"""Best-effort read of an own-policy adapter's effective group policy.
Mirror of ``_adapter_dm_policy`` for group / forum / channel traffic:
returns the lowercased ``group_policy`` (``"open"`` / ``"allowlist"`` /
``"disabled"``) for *platform*, or ``""`` when unknown. Prefers the live
adapter's resolved ``_group_policy`` and falls back to ``config.extra``
for bare runners built without a live adapter.
Used by ``_is_user_authorized`` to decide whether an own-policy adapter
restricted group senders to a configured allowlist (trustworthy) or
forwarded the whole channel under ``group_policy: open`` (not
authorization).
"""
if not platform:
return ""
adapters = getattr(self, "adapters", None) or {}
adapter = adapters.get(platform)
policy = getattr(adapter, "_group_policy", None) if adapter is not None else None
if policy is None:
config = getattr(self, "config", None)
platform_cfg = (
config.platforms.get(platform)
if config is not None and hasattr(config, "platforms")
else None
)
extra = getattr(platform_cfg, "extra", None) if platform_cfg else None
if isinstance(extra, dict):
policy = extra.get("group_policy")
return str(policy or "").strip().lower()
def _adapter_group_has_sender_allowlist(
self,
platform: Optional[Platform],
chat_id: Optional[str],
) -> bool:
"""Whether a per-group sender allowlist gated this group message.
WeCom supports ``groups.<group_id>.allow_from`` on top of the top-level
``group_policy``. A group may be open at the chat level while still
restricting which senders inside that group can invoke Hermes. If such a
message reached the gateway, the adapter already checked that sender
allowlist, so it is a trustworthy intake decision rather than the
fail-open ``group_policy: open`` case.
"""
if not platform or not chat_id:
return False
adapters = getattr(self, "adapters", None) or {}
adapter = adapters.get(platform)
groups = getattr(adapter, "_groups", None) if adapter is not None else None
if groups is None:
config = getattr(self, "config", None)
platform_cfg = (
config.platforms.get(platform)
if config is not None and hasattr(config, "platforms")
else None
)
extra = getattr(platform_cfg, "extra", None) if platform_cfg else None
if isinstance(extra, dict):
groups = extra.get("groups")
if not isinstance(groups, dict):
return False
chat_id_str = str(chat_id)
group_cfg = groups.get(chat_id_str)
if not isinstance(group_cfg, dict):
lowered = chat_id_str.lower()
for key, value in groups.items():
if isinstance(key, str) and key.lower() == lowered and isinstance(value, dict):
group_cfg = value
break
if not isinstance(group_cfg, dict):
group_cfg = groups.get("*")
if not isinstance(group_cfg, dict):
return False
sender_allow = group_cfg.get("allow_from") or group_cfg.get("allowFrom")
if isinstance(sender_allow, str):
return bool(sender_allow.strip())
if isinstance(sender_allow, (list, tuple, set)):
return any(str(item).strip() for item in sender_allow)
return False
def _is_user_authorized(self, source: SessionSource) -> bool:
"""
Check if a user is authorized to use the bot.
@@ -237,27 +323,40 @@ class GatewayAuthorizationMixin:
global_allowlist = os.getenv("GATEWAY_ALLOWED_USERS", "").strip()
if not platform_allowlist and not group_user_allowlist and not group_chat_allowlist and not global_allowlist:
# No env allowlists configured. Adapters that own their own
# No env allowlist configured. Adapters that own their own
# config-driven access policy (dm_policy / group_policy /
# allow_from / group_allow_from) already gated this message at
# intake — it would not have reached the gateway otherwise — so
# honor that decision instead of falling through to the
# env-only default-deny below, which would silently break
# `dm_policy: open` and config-only allowlists. (#34515)
# allow_from / group_allow_from) gate access at intake, so for those
# platforms we can honor the adapter's decision instead of the
# env-only default-deny below -- but ONLY when that decision was an
# actual allowlist restriction.
#
# The adapters default dm_policy / group_policy to "open", which
# forwards EVERY sender. Reading "reached the gateway" as
# authorization in that case would admit the whole external network
# with no operator-configured allowlist -- the fail-open SECURITY.md
# §2.6 forbids ("an allowlist is required for every enabled
# network-exposed adapter ... code paths that fail open when no
# allowlist is configured are code bugs"). "disabled" never
# forwards, and "pairing" forwards unpaired DMs only so the gateway
# can run its pairing handshake (the pairing-store check above
# already denied this sender). So trust the adapter only when its
# effective policy for THIS chat type is "allowlist"; for "open" /
# "pairing" / anything else, fall through to default-deny, where
# GATEWAY_ALLOW_ALL_USERS, the per-platform {PLATFORM}_ALLOW_ALL_USERS
# flag (checked above), and the pairing flow remain the explicit
# opt-ins to broader access. (#34515 follow-up: trusting "open" was a
# fail-open.)
if self._adapter_enforces_own_access_policy(source.platform):
# Exception: `dm_policy: pairing` does NOT authorize at intake.
# The adapter forwards the DM precisely so the gateway can run
# its pairing handshake (issue a code, consult the pairing
# store). The pairing-store approval check above already ran and
# returned False for this sender, so blanket-trusting the
# adapter here would silently turn pairing mode into open
# access. Fall through to default-deny so the unpaired sender is
# offered a pairing code instead. (Pairing is DM-only; group
# traffic keeps the adapter-trust path.)
if not (
source.chat_type == "dm"
and self._adapter_dm_policy(source.platform) == "pairing"
):
if source.chat_type in {"group", "forum", "channel"}:
effective_policy = self._adapter_group_policy(source.platform)
if self._adapter_group_has_sender_allowlist(
source.platform,
source.chat_id,
):
return True
else:
effective_policy = self._adapter_dm_policy(source.platform)
if effective_policy == "allowlist":
return True
# No allowlists configured -- check global allow-all flag
return os.getenv("GATEWAY_ALLOW_ALL_USERS", "").lower() in {"true", "1", "yes"}
+56 -8
View File
@@ -1128,8 +1128,11 @@ SUPPORTED_DOCUMENT_TYPES = {
".ini": "text/plain",
".cfg": "text/plain",
".zip": "application/zip",
".doc": "application/msword",
".docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
".xls": "application/vnd.ms-excel",
".xlsx": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
".ppt": "application/vnd.ms-powerpoint",
".pptx": "application/vnd.openxmlformats-officedocument.presentationml.presentation",
".ts": "text/plain",
".py": "text/plain",
@@ -1913,16 +1916,21 @@ class BasePlatformAdapter(ABC):
enforce it at intake: a message is dropped inside the adapter and never
reaches the gateway unless it already passed that policy.
The gateway's env-based allowlist check runs *after* the adapter, so for
these platforms a message arriving at ``_is_user_authorized`` has, by
definition, already been authorized by the adapter. Without this flag the
gateway would then deny it again (no env allowlist default deny),
silently breaking ``dm_policy: open`` and config-only allowlists.
The gateway's env-based allowlist check runs *after* the adapter. When
no env allowlist is configured, the gateway consults this flag so it can
honor a config-only ``dm_policy: allowlist`` / ``allow_from`` (which the
adapter already enforced) instead of double-denying it. Crucially, the
flag alone is NOT "already authorized": these adapters default
``dm_policy`` / ``group_policy`` to ``"open"``, which forwards every
sender, so the gateway trusts the adapter only when its effective policy
for the chat type is an actual ``"allowlist"`` restriction never for
``"open"`` (that would be the network-exposed fail-open SECURITY.md §2.6
forbids). Open access still requires an explicit
``{PLATFORM}_ALLOW_ALL_USERS`` / ``GATEWAY_ALLOW_ALL_USERS`` opt-in.
Adapters that own their access policy override this to return ``True``.
The gateway treats that as "already authorized at intake" and skips the
env-allowlist default-deny. Adapters that delegate access control to the
gateway leave it ``False`` (the default).
Adapters that delegate access control to the gateway leave it ``False``
(the default).
"""
return False
@@ -1945,6 +1953,46 @@ class BasePlatformAdapter(ABC):
"""
return False
def prefers_fresh_final_streaming(
self,
content: str,
metadata: Optional[Dict[str, Any]] = None,
) -> bool:
"""Whether the stream consumer should finalize a streamed reply by
sending a *fresh* final message (and deleting the preview) instead of
final-editing the preview.
Some adapters can send richer final messages than their current edit
implementation supports. Telegram is the motivating case: Hermes sends
final replies through ``sendRichMessage`` but still finalizes streamed
previews through its existing MarkdownV2 edit path until Bot API 10.1's
``rich_message`` edit parameter is wired directly. Such adapters
override this to ask the consumer to re-deliver the completed answer as
a new rich message and best-effort delete the stale preview, so the
final rendering matches the rich send path.
Default implementation returns False legacy platforms keep the
edit-in-place finalization path.
"""
return False
def streaming_overflow_limit(self) -> Optional[int]:
"""Max single-message length (in this adapter's ``message_len_fn``
units) the stream consumer may accumulate before it splits, when the
adapter can deliver a larger message than its legacy per-message limit.
Telegram Bot API 10.1 Rich Messages accept up to 32,768 chars in a
single ``sendRichMessage`` / ``sendRichMessageDraft``, far above the
4,096 MarkdownV2 limit. Adapters with such a richer send/draft path
override this so the consumer doesn't fragment a reply that fits one
rich message; the live edit preview is still bound by the platform's
edit limit, but the finalized reply (and DM draft preview) is delivered
whole.
Return ``None`` (default) to use ``MAX_MESSAGE_LENGTH``.
"""
return None
async def send_draft(
self,
chat_id: str,
+1 -1
View File
@@ -209,7 +209,7 @@ class _MatrixHtmlSanitizer(HTMLParser):
_ALLOWED_TAGS = {
"a", "b", "blockquote", "br", "code", "del", "em", "h1", "h2", "h3",
"h4", "h5", "h6", "hr", "i", "li", "ol", "p", "pre", "s", "strike",
"strong", "ul",
"strong", "table", "tbody", "td", "th", "thead", "tr", "ul",
}
_VOID_TAGS = {"br", "hr"}
+73 -15
View File
@@ -349,8 +349,11 @@ class TelegramAdapter(BasePlatformAdapter):
MAX_MESSAGE_LENGTH = 4096
supports_code_blocks = True # Telegram MarkdownV2 renders fenced code blocks
# Bot API 10.1 Rich Messages cap the raw markdown/html text at 32,768
# UTF-8 bytes. Content above this is sent via the legacy chunking path.
RICH_MESSAGE_MAX_BYTES = 32768
# UTF-8 characters. Content above this is sent via the legacy chunking path.
RICH_MESSAGE_MAX_CHARS = 32768
# Backwards-compatible alias for tests/external callers that referenced the
# initial implementation name. The API limit is character-based, not bytes.
RICH_MESSAGE_MAX_BYTES = RICH_MESSAGE_MAX_CHARS
# Threshold for detecting Telegram client-side message splits.
# When a chunk is near this limit, a continuation is almost certain.
_SPLIT_THRESHOLD = 4000
@@ -920,19 +923,20 @@ class TelegramAdapter(BasePlatformAdapter):
# the RAW agent markdown so richer constructs (tables, task lists,
# collapsible details, math, ...) render natively. The legacy MarkdownV2
# send() path stays as the fallback for unsupported/oversized content and
# older PTB/clients. Streaming edits/drafts are intentionally untouched —
# Telegram exposes no rich-edit method.
# older PTB/clients. Streaming edits stay on Hermes' existing MarkdownV2
# edit path for now; finalization can re-send as rich and delete the stale
# preview until rich_message edit support is wired directly.
# ------------------------------------------------------------------
def _content_fits_rich_limits(self, content: str) -> bool:
"""Cheap pre-check for the one hard rich limit we can count locally.
Only the 32,768 UTF-8 byte text cap is enforced here. Other Bot API
Only the 32,768 UTF-8 character text cap is enforced here. Other Bot API
rich limits (500 blocks, 16 nesting levels, 20 table columns, ...) are
not pre-counted; if exceeded Telegram returns a BadRequest, which
:meth:`_is_rich_fallback_error` classifies as permanent so the send
degrades to the legacy chunking path.
"""
return len(content.encode("utf-8")) <= self.RICH_MESSAGE_MAX_BYTES
return len(content) <= self.RICH_MESSAGE_MAX_CHARS
def _bot_supports_rich(self) -> bool:
"""True when the bound bot can issue raw ``sendRichMessage`` calls.
@@ -946,15 +950,57 @@ class TelegramAdapter(BasePlatformAdapter):
"""
return inspect.iscoroutinefunction(getattr(self._bot, "do_api_request", None))
def _should_attempt_rich(self, content: str) -> bool:
def _should_attempt_rich(
self, content: str, metadata: Optional[Dict[str, Any]] = None
) -> bool:
return bool(
not getattr(self, "_rich_send_disabled", False)
and not (metadata or {}).get("expect_edits")
and content
and content.strip()
and self._content_fits_rich_limits(content)
and self._bot_supports_rich()
)
def prefers_fresh_final_streaming(
self, content: str, metadata: Optional[Dict[str, Any]] = None
) -> bool:
"""Finalize rich-eligible streamed replies with a fresh sendRichMessage
instead of Hermes' current MarkdownV2 edit path.
The final edit path has not yet been upgraded to Bot API 10.1's
``rich_message`` edit parameter, so finalizing through edit would lose
rich constructs such as tables/task lists. When the completed content
is rich-eligible, re-send it via ``sendRichMessage`` and delete the
preview (see ``gateway.stream_consumer._try_fresh_final``).
``metadata`` is intentionally ignored: the preview was sent with
``expect_edits=True`` (to stay on the editable path mid-stream), but the
FINAL answer is a brand-new message that should render rich. Gating
otherwise matches :meth:`_should_attempt_rich`: rich not latched off,
content present and within the rich character limit, and the bot exposes
an async ``do_api_request``.
"""
return self._should_attempt_rich(content)
def streaming_overflow_limit(self) -> Optional[int]:
"""Allow the stream consumer to accumulate up to the rich-message cap
before splitting, so a reply that fits one ``sendRichMessage`` /
``sendRichMessageDraft`` isn't fragmented at the 4,096 MarkdownV2 limit.
Gated on the same rich capability as the send path (minus the
content-length check raising that cap is the whole point): rich not
latched off and the bot exposes an async ``do_api_request``. Returns
``None`` ( legacy 4,096 limit) when rich isn't available, so non-rich
streams split exactly as before.
"""
if (
not getattr(self, "_rich_send_disabled", False)
and self._bot_supports_rich()
):
return self.RICH_MESSAGE_MAX_CHARS
return None
def _rich_message_payload(
self, content: str, *, skip_entity_detection: bool = False
) -> Dict[str, Any]:
@@ -976,16 +1022,19 @@ class TelegramAdapter(BasePlatformAdapter):
rejections (BadRequest from a parser/limit issue) are NOT capability
errors: the next message may be fine.
"""
name = exc.__class__.__name__.lower()
if name in {"endpointnotfound", "invalidtoken"}:
return True
if isinstance(exc, (AttributeError, TypeError, NotImplementedError)):
return True
if getattr(exc, "error_code", None) == 404:
return True
s = str(exc).lower()
if ("method" in s and "not found" in s) or "no such method" in s:
if ("method" in s or "endpoint" in s) and (
"not found" in s or "does not exist" in s
):
return True
if "unsupported" in s or "not implemented" in s:
return True
return False
return "no such method" in s
def _is_rich_fallback_error(self, exc: Exception) -> bool:
"""True ⇒ permanent/capability error ⇒ safe to fall back to legacy.
@@ -997,7 +1046,10 @@ class TelegramAdapter(BasePlatformAdapter):
"""
if self._is_bad_request_error(exc):
return True
return self._is_rich_capability_error(exc)
if self._is_rich_capability_error(exc):
return True
s = str(exc).lower()
return "unsupported" in s or "not implemented" in s
def _compute_single_send_routing(
self,
@@ -1070,6 +1122,8 @@ class TelegramAdapter(BasePlatformAdapter):
# which must not be sent as a stray field on the raw endpoint.
payload.update({k: v for k, v in thread_kwargs.items() if v is not None})
payload.update(self._notification_kwargs(metadata))
if getattr(self, "_disable_link_previews", False):
payload["link_preview_options"] = {"is_disabled": True}
if reply_to_id is not None:
# Spec: sendRichMessage takes reply_parameters (ReplyParameters
# object), NOT the legacy reply_to_message_id scalar. Unknown
@@ -1078,8 +1132,12 @@ class TelegramAdapter(BasePlatformAdapter):
payload["reply_parameters"] = {"message_id": reply_to_id}
try:
# Take the raw Bot API result (dict under real PTB). Passing
# return_type=Message would make PTB deserialize a Bot API 10.1
# response shape it does not fully model yet; a post-delivery parse
# error must not be mistaken for a sendable failure.
msg = await self._bot.do_api_request(
"sendRichMessage", api_kwargs=payload, return_type=Message
"sendRichMessage", api_kwargs=payload
)
except Exception as exc:
if self._is_rich_fallback_error(exc):
@@ -1116,7 +1174,7 @@ class TelegramAdapter(BasePlatformAdapter):
if isinstance(msg, dict):
message_id = msg.get("message_id")
if message_id is None:
message_id = msg.get("result", {}).get("message_id")
message_id = (msg.get("result") or {}).get("message_id")
else:
message_id = getattr(msg, "message_id", None)
return SendResult(
@@ -2146,7 +2204,7 @@ class TelegramAdapter(BasePlatformAdapter):
# through to the legacy MarkdownV2 path on permanent/capability
# errors or DM-topic routing skips; returns directly on success or
# on a transient failure (which must NOT be legacy-resent).
if self._should_attempt_rich(content):
if self._should_attempt_rich(content, metadata=metadata):
rich_result = await self._try_send_rich(chat_id, content, reply_to, metadata)
if rich_result is not None:
if rich_result.success:
+35 -1
View File
@@ -4556,6 +4556,14 @@ class GatewayRunner(GatewayAuthorizationMixin, GatewayKanbanWatchersMixin, Gatew
)
continue
# Claim the session slot *before* spawning the task so that an
# inbound message arriving between task creation and the task's
# first await (where _process_message_background sets the real
# sentinel) sees the slot as occupied and queues behind it
# instead of spinning up a duplicate AIAgent (#45456).
self._running_agents[entry.session_key] = _AGENT_PENDING_SENTINEL
self._running_agents_ts[entry.session_key] = time.time()
# Empty-text internal event — the _is_resume_pending branch in
# _handle_message_with_agent prepends the proper reason-aware
# system note before the turn runs.
@@ -4565,7 +4573,33 @@ class GatewayRunner(GatewayAuthorizationMixin, GatewayKanbanWatchersMixin, Gatew
source=source,
internal=True,
)
task = asyncio.create_task(adapter.handle_message(event))
async def _guarded_handle_message(
_adapter: Any, _event: MessageEvent, _key: str = entry.session_key,
) -> None:
"""Ensure the pre-claimed sentinel is always released.
In the normal flow the resume turn reaches
``_handle_message``, which replaces our pre-claim with
its own ``_AGENT_PENDING_SENTINEL`` (and releases it in
its ``finally`` block) once the run begins. If
``handle_message`` raises *before* the runner takes over
the slot (e.g. during topic recovery or session-key
resolution), nobody clears our pre-claim so we do it
here unconditionally. The ``is _AGENT_PENDING_SENTINEL``
guard below only releases the slot we ourselves placed,
never one a live run currently owns.
"""
try:
await _adapter.handle_message(_event)
finally:
# Only release if the sentinel we set is still there
# (i.e. _process_message_background hasn't replaced
# and cleaned it already).
if self._running_agents.get(_key) is _AGENT_PENDING_SENTINEL:
self._release_running_agent_state(_key)
task = asyncio.create_task(_guarded_handle_message(adapter, event))
self._background_tasks.add(task)
task.add_done_callback(self._background_tasks.discard)
scheduled += 1
+143 -22
View File
@@ -143,6 +143,13 @@ class GatewayStreamConsumer:
# timestamps would be stale by completion time. Ported from
# openclaw/openclaw#72038.
self._message_created_ts: Optional[float] = None
# Every real preview message id the consumer has put on screen during
# this response (first send + any continuation messages from oversized
# edits/sends). The fresh-final path deletes all of them when it
# re-delivers the completed answer as a single (rich) message, so a
# reply that was split across the platform's edit limit while streaming
# doesn't leave stale fragments above the final message.
self._preview_message_ids: "set[str]" = set()
self._already_sent = False
self._edit_supported = True # Disabled when progressive edits are no longer usable
self._last_edit_time = 0.0
@@ -420,7 +427,10 @@ class GatewayStreamConsumer:
if isinstance(self.adapter, _BasePlatformAdapter)
else len
)
_raw_limit = getattr(self.adapter, "MAX_MESSAGE_LENGTH", 4096)
# Rich-capable adapters (Telegram rich messages) raise this above the
# legacy per-message limit so a reply that fits one rich send/draft
# isn't fragmented at 4096 while streaming. See _raw_message_limit.
_raw_limit = self._raw_message_limit()
_safe_limit = max(500, _raw_limit - _len_fn(self.cfg.cursor) - 100)
# Resolve native draft streaming once per run. When enabled the
@@ -589,9 +599,20 @@ class GatewayStreamConsumer:
if self._accumulated:
if self._fallback_final_send:
await self._send_fallback_final(self._accumulated)
elif current_update_visible and (
not self._adapter_requires_finalize
or self._last_edit_overflowed
elif self._final_response_sent:
# A finalize=True tick above already delivered the
# final answer via the adapter's fresh-final path
# (_try_fresh_final sent a fresh rich message and
# deleted the preview). Running a second finalize
# edit here would duplicate the message / re-delete,
# so just record delivery and stop.
self._final_content_delivered = True
elif (
current_update_visible
and (
not self._adapter_requires_finalize
or self._last_edit_overflowed
)
):
# Mid-stream edit above already delivered the
# final accumulated content. Skip the redundant
@@ -729,6 +750,9 @@ class GatewayStreamConsumer:
return reply_to_id
try:
meta = dict(self.metadata) if self.metadata else {}
# This chunk becomes the next edit target — adapters that support
# rich final sends (Telegram) must keep it on the editable path.
meta["expect_edits"] = True
result = await self.adapter.send(
chat_id=self.chat_id,
content=text,
@@ -737,6 +761,7 @@ class GatewayStreamConsumer:
)
if result.success and result.message_id:
self._message_id = str(result.message_id)
self._track_preview_ids_from_result(result)
self._already_sent = True
self._last_sent_text = text
# Fresh content bubble — close off any stale tool bubble
@@ -1114,6 +1139,76 @@ class GatewayStreamConsumer:
age = time.monotonic() - self._message_created_ts
return age >= threshold
def _raw_message_limit(self) -> int:
"""Per-message length budget (in the adapter's ``message_len_fn`` units)
before the consumer splits an overflowing reply.
Adapters with a richer send/draft path (e.g. Telegram rich messages)
can raise this above ``MAX_MESSAGE_LENGTH`` via
``streaming_overflow_limit`` so a reply that fits one rich message isn't
fragmented at the legacy edit limit. Falls back to
``MAX_MESSAGE_LENGTH`` (4096 default) for everyone else.
"""
base = getattr(self.adapter, "MAX_MESSAGE_LENGTH", 4096)
# isinstance gate: MagicMock adapters return mock objects (truthy, not
# ints) for arbitrary attribute access — keep them on the base limit.
if isinstance(self.adapter, _BasePlatformAdapter):
try:
cap = self.adapter.streaming_overflow_limit()
except Exception as e:
logger.debug("streaming_overflow_limit check failed: %s", e)
cap = None
if isinstance(cap, int) and cap > base:
return cap
return base
def _track_preview_id(self, message_id: Optional[str]) -> None:
"""Record a real preview message id for fresh-final cleanup."""
if message_id and message_id != "__no_edit__":
self._preview_message_ids.add(str(message_id))
def _track_preview_ids_from_result(self, result: Any) -> None:
"""Record every message id a send/edit result exposes: the primary id
plus any continuation ids from an oversized split
(``continuation_message_ids`` or ``raw_response['message_ids']``)."""
self._track_preview_id(getattr(result, "message_id", None))
for mid in (getattr(result, "continuation_message_ids", None) or ()):
self._track_preview_id(mid)
raw = getattr(result, "raw_response", None) or {}
if isinstance(raw, dict):
for mid in (raw.get("message_ids") or ()):
self._track_preview_id(mid)
def _adapter_prefers_fresh_final(self, text: str) -> bool:
"""Return True when the adapter would rather finalize a streamed reply
by sending a fresh message and deleting the preview than by editing the
preview in place e.g. Telegram, whose ``sendRichMessage`` send path
currently renders richer markdown than Hermes' MarkdownV2 edit path.
Returns False when there is no real preview to replace (no message id,
or the ``__no_edit__`` sentinel), when the adapter doesn't expose the
hook, or on any error (the consumer then keeps the edit-in-place path).
"""
if not self._message_id or self._message_id == "__no_edit__":
return False
fn = getattr(self.adapter, "prefers_fresh_final_streaming", None)
if fn is None:
return False
try:
try:
result = fn(text, metadata=self.metadata)
except TypeError:
# Adapter / test double whose hook doesn't accept the metadata
# keyword — fall back to the positional-only form.
result = fn(text)
except Exception as e:
logger.debug("prefers_fresh_final_streaming check failed: %s", e)
return False
# ``is True`` (not ``bool(...)``) so a MagicMock adapter's auto-child
# method — truthy by default in tests — does not wrongly enable the
# fresh-final path. Mirrors the REQUIRES_EDIT_FINALIZE gate in __init__.
return result is True
async def _try_fresh_final(self, text: str, *, is_turn_final: bool = True) -> bool:
"""Send ``text`` as a brand-new message (best-effort delete the old
preview) so the platform's visible timestamp reflects completion
@@ -1127,7 +1222,13 @@ class GatewayStreamConsumer:
Ported from openclaw/openclaw#72038.
"""
old_message_id = self._message_id
# Every preview message the user has seen for this response: the
# current one plus any continuation fragments tracked while streaming
# (an oversized reply split across the platform's edit limit). All of
# them are replaced by the single fresh message below.
stale_ids = set(self._preview_message_ids)
if self._message_id and self._message_id != "__no_edit__":
stale_ids.add(self._message_id)
try:
result = await self.adapter.send(
chat_id=self.chat_id,
@@ -1139,25 +1240,29 @@ class GatewayStreamConsumer:
return False
if not getattr(result, "success", False):
return False
# Successful fresh send — try to delete the stale preview so the
# user doesn't see the old edit-stuck message underneath. Cleanup
# is best-effort; platforms that don't implement ``delete_message``
# just leave the preview behind (still an acceptable outcome —
# the visible final timestamp is the important part).
if old_message_id and old_message_id != "__no_edit__":
delete_fn = getattr(self.adapter, "delete_message", None)
if delete_fn is not None:
try:
await delete_fn(self.chat_id, old_message_id)
except Exception as e:
logger.debug(
"Fresh-final preview cleanup failed (%s): %s",
old_message_id, e,
)
# Adopt the new message id as the current message so subsequent
# callers (e.g. overflow split loops, finalize retries) see a
# consistent state.
new_message_id = getattr(result, "message_id", None)
# Successful fresh send — try to delete the stale preview(s) so the
# user doesn't see the old edit-stuck message(s) underneath. Cleanup
# is best-effort; platforms that don't implement ``delete_message``
# just leave the preview behind (still an acceptable outcome — the
# visible final timestamp is the important part). Never delete the
# message we just sent.
delete_fn = getattr(self.adapter, "delete_message", None)
if delete_fn is not None:
for stale_id in stale_ids:
if not stale_id or stale_id == "__no_edit__" or stale_id == new_message_id:
continue
try:
await delete_fn(self.chat_id, stale_id)
except Exception as e:
logger.debug(
"Fresh-final preview cleanup failed (%s): %s",
stale_id, e,
)
self._preview_message_ids = set()
if new_message_id:
self._message_id = new_message_id
self._message_created_ts = time.monotonic()
@@ -1267,9 +1372,19 @@ class GatewayStreamConsumer:
# old preview follows. Ported from
# openclaw/openclaw#72038. Gated by config so the
# legacy edit-in-place path stays the default.
#
# Adapters can also opt in regardless of the time threshold
# via prefers_fresh_final_streaming (e.g. Telegram, whose
# send path renders richer markdown than its edit path):
# finalizing through edit would visibly downgrade a rich
# preview, so re-deliver as a fresh message + delete the
# preview instead.
if (
finalize
and self._should_send_fresh_final()
and (
self._should_send_fresh_final()
or self._adapter_prefers_fresh_final(text)
)
and await self._try_fresh_final(
text, is_turn_final=is_turn_final,
)
@@ -1283,6 +1398,9 @@ class GatewayStreamConsumer:
)
if result.success:
self._already_sent = True
# Record any continuation fragments an oversized edit
# split off, so fresh-final can clean them all up.
self._track_preview_ids_from_result(result)
# Adapter may have split-and-delivered an oversized
# edit across the original message + N continuations.
# When that happens, ``message_id`` is the LAST visible
@@ -1405,7 +1523,7 @@ class GatewayStreamConsumer:
chat_id=self.chat_id,
content=text,
reply_to=self._initial_reply_to_id,
metadata=self.metadata,
metadata={**(self.metadata or {}), "expect_edits": True},
)
if result.success:
if result.message_id:
@@ -1414,6 +1532,9 @@ class GatewayStreamConsumer:
# the user so fresh-final logic can detect stale
# preview timestamps on long-running responses.
self._message_created_ts = time.monotonic()
# Record this (and any continuation fragments from an
# oversized first send) for fresh-final cleanup.
self._track_preview_ids_from_result(result)
else:
self._edit_supported = False
self._already_sent = True
+201 -9
View File
@@ -8030,6 +8030,182 @@ def _run_pre_update_backup(args) -> None:
print()
def _write_update_planned_stop_marker(profile_path: Path, pid: int) -> bool:
"""Write a planned-stop marker into a specific profile home."""
try:
from datetime import timezone
from gateway.status import _get_process_start_time
from utils import atomic_json_write
record = {
"target_pid": pid,
"target_start_time": _get_process_start_time(pid),
"stopper_pid": os.getpid(),
"written_at": datetime.now(timezone.utc).isoformat(),
}
atomic_json_write(
Path(profile_path) / ".gateway-planned-stop.json",
record,
indent=None,
separators=(",", ":"),
)
return True
except (OSError, PermissionError):
return False
def _wait_for_windows_update_gateway_exit(
pids: list[int], *, timeout: float
) -> set[int]:
"""Wait for the given gateway PIDs to exit, returning survivors."""
if not pids:
return set()
from gateway.status import _pid_exists
remaining = set(pids)
deadline = _time.monotonic() + max(timeout, 0.0)
while remaining and _time.monotonic() < deadline:
for pid in list(remaining):
try:
if not _pid_exists(pid):
remaining.discard(pid)
except Exception:
remaining.discard(pid)
if remaining:
_time.sleep(0.25)
survivors: set[int] = set()
for pid in remaining:
try:
if _pid_exists(pid):
survivors.add(pid)
except Exception:
pass
return survivors
def _pause_windows_gateways_for_update() -> dict | None:
"""Stop running Windows gateways before mutating the checkout or venv.
Windows scheduled/startup gateways run through pythonw.exe, so the generic
hermes.exe concurrent-instance guard does not see them. They still import
from the checkout and can keep files locked while ``git`` or ``uv`` updates
the install. Stop only PIDs that the gateway discovery code identifies.
"""
if not _is_windows():
return None
try:
from gateway.status import terminate_pid
from hermes_cli.gateway import (
_get_restart_drain_timeout,
find_gateway_pids,
find_profile_gateway_processes,
)
except Exception as exc:
logger.debug("Could not prepare Windows gateway pause for update: %s", exc)
return None
try:
running_pids = list(dict.fromkeys(find_gateway_pids(all_profiles=True)))
except Exception as exc:
logger.debug("Could not discover Windows gateway PIDs before update: %s", exc)
return None
if not running_pids:
return None
profile_processes = {}
try:
profile_processes = {
proc.pid: proc for proc in find_profile_gateway_processes()
}
except Exception as exc:
logger.debug("Could not map Windows gateway PIDs to profiles: %s", exc)
profiles: dict[str, int] = {}
mapped_pids = []
for pid in running_pids:
proc = profile_processes.get(pid)
if proc is None:
continue
profiles[str(proc.profile)] = int(pid)
mapped_pids.append(int(pid))
_write_update_planned_stop_marker(Path(proc.path), int(pid))
print("→ Stopping Windows gateway process(es) before updating Hermes...")
try:
drain_timeout = max(float(_get_restart_drain_timeout()), 1.0)
except Exception:
drain_timeout = 10.0
survivors = _wait_for_windows_update_gateway_exit(
mapped_pids,
timeout=drain_timeout,
)
unmapped_pids = [pid for pid in running_pids if pid not in profile_processes]
force_killed = []
for pid in sorted(set(survivors).union(unmapped_pids)):
try:
terminate_pid(int(pid), force=True)
force_killed.append(int(pid))
except (ProcessLookupError, PermissionError, OSError):
pass
if profiles:
print(f" ✓ Paused gateway profile(s): {', '.join(sorted(profiles))}")
if force_killed:
print(f" → Force-stopped {len(force_killed)} gateway process(es)")
if unmapped_pids:
print(
f" → Stopped {len(unmapped_pids)} gateway process(es) without profile mapping"
)
print(" Restart manually after update: hermes gateway run")
return {
"resume_needed": True,
"profiles": profiles,
"unmapped_pids": unmapped_pids,
}
def _resume_windows_gateways_after_update(token: dict | None) -> None:
"""Restart Windows profile gateways previously paused for update."""
if not token or not token.get("resume_needed"):
return
token["resume_needed"] = False
if not _is_windows():
return
profiles = token.get("profiles") or {}
if not profiles:
return
try:
from hermes_cli.gateway import launch_detached_profile_gateway_restart
except Exception as exc:
logger.debug("Could not load Windows gateway restart helper: %s", exc)
return
relaunched = []
for profile, old_pid in sorted(profiles.items()):
try:
if launch_detached_profile_gateway_restart(str(profile), int(old_pid)):
relaunched.append(str(profile))
except Exception as exc:
logger.debug(
"Could not restart Windows gateway profile %s after update: %s",
profile,
exc,
)
if relaunched:
print()
print(f" ✓ Restarting Windows gateway profile(s): {', '.join(relaunched)}")
def _discard_lockfile_churn(git_cmd, repo_root):
"""Restore tracked ``package-lock.json`` files that npm dirtied locally.
@@ -8232,6 +8408,15 @@ def _cmd_update_impl(args, gateway_mode: bool):
# always roll back to the exact state they had before this update.
_run_pre_update_backup(args)
_windows_gateway_resume = _pause_windows_gateways_for_update()
if _windows_gateway_resume:
import atexit as _atexit
_atexit.register(
_resume_windows_gateways_after_update,
_windows_gateway_resume,
)
# Try git-based update first, fall back to ZIP download on Windows
# when git file I/O is broken (antivirus, NTFS filter drivers, etc.)
use_zip_update = False
@@ -8294,7 +8479,10 @@ def _cmd_update_impl(args, gateway_mode: bool):
if use_zip_update:
# ZIP-based update for Windows when git is broken
_update_via_zip(args)
try:
_update_via_zip(args)
finally:
_resume_windows_gateways_after_update(_windows_gateway_resume)
return
# Fetch and pull
@@ -8431,6 +8619,7 @@ def _cmd_update_impl(args, gateway_mode: bool):
check=False,
)
print("✓ Already up to date!")
_resume_windows_gateways_after_update(_windows_gateway_resume)
return
print(f"→ Found {commit_count} new commit(s)")
@@ -9627,6 +9816,8 @@ def _cmd_update_impl(args, gateway_mode: bool):
except Exception as e:
logger.debug("Gateway restart during update failed: %s", e)
_resume_windows_gateways_after_update(_windows_gateway_resume)
# Warn if legacy Hermes gateway unit files are still installed.
# When both hermes.service (from a pre-rename install) and the
# current hermes-gateway.service are enabled, they SIGTERM-fight
@@ -9860,19 +10051,20 @@ def cmd_profile(args):
try:
clone_from = getattr(args, "clone_from", None)
clone_config = clone or clone_from is not None
profile_dir = create_profile(
name=name,
clone_from=clone_from,
clone_all=clone_all,
clone_config=clone,
clone_config=clone_config,
no_alias=no_alias,
no_skills=no_skills,
description=getattr(args, "description", None),
)
print(f"\nProfile '{name}' created at {profile_dir}")
if clone or clone_all:
if clone_config or clone_all:
source_label = (
getattr(args, "clone_from", None) or get_active_profile_name()
)
@@ -9886,8 +10078,8 @@ def cmd_profile(args):
f"Cloned config, .env, SOUL.md, and skills from {source_label}."
)
# Auto-clone Honcho config for the new profile (only with --clone/--clone-all)
if clone or clone_all:
# Auto-clone Honcho config for the new profile (only with clone operations)
if clone_config or clone_all:
try:
from plugins.memory.honcho.cli import clone_honcho_for_profile
@@ -9896,10 +10088,10 @@ def cmd_profile(args):
except Exception:
pass # Honcho plugin not installed or not configured
# Seed bundled skills (skip if --clone-all already copied them, or
# if --no-skills was passed — in which case seed_profile_skills()
# honors the marker file and returns skipped_opt_out=True).
if not clone_all:
# Seed bundled skills for fresh profiles only. Clone operations
# already copied the source profile's skills, including any
# user-installed or intentionally removed skills.
if not (clone_config or clone_all):
result = seed_profile_skills(profile_dir)
if result and result.get("skipped_opt_out"):
print(
+2 -1
View File
@@ -84,7 +84,6 @@ _STRIP_VENDOR_ONLY_PROVIDERS: frozenset[str] = frozenset({
# Providers whose native naming is authoritative -- pass through unchanged.
_AUTHORITATIVE_NATIVE_PROVIDERS: frozenset[str] = frozenset({
"gemini",
"huggingface",
})
@@ -103,6 +102,8 @@ _MATCHING_PREFIX_STRIP_PROVIDERS: frozenset[str] = frozenset({
"arcee",
"ollama-cloud",
"custom",
"gemini",
"xai",
})
# Providers whose APIs require lowercase model IDs. Xiaomi's
+27
View File
@@ -135,6 +135,20 @@ def _sanitize_plugin_name(
return target
_GITHUB_BROWSER_SEGMENTS = {
"actions",
"blob",
"commit",
"commits",
"issues",
"pull",
"pulls",
"releases",
"tree",
"wiki",
}
def _resolve_git_url(identifier: str) -> tuple[str, Optional[str]]:
"""Turn an identifier into a cloneable Git URL and optional subdirectory.
@@ -146,6 +160,8 @@ def _resolve_git_url(identifier: str) -> tuple[str, Optional[str]]:
- Full URL: https://github.com/owner/repo.git
- Full URL: git@github.com:owner/repo.git
- Full URL: ssh://git@github.com/owner/repo.git
- Browser URL: https://github.com/owner/repo/tree/main/path
(https://github.com/owner/repo.git, "path")
- Shorthand: owner/repo https://github.com/owner/repo.git
- Shorthand w/ subdir: owner/repo/path/to/plugin
(https://github.com/owner/repo.git, "path/to/plugin")
@@ -161,6 +177,17 @@ def _resolve_git_url(identifier: str) -> tuple[str, Optional[str]]:
"""
# Already a URL.
if identifier.startswith(("https://", "http://", "git@", "ssh://", "file://")):
if identifier.startswith("https://github.com/"):
path = identifier[len("https://github.com/") :]
path = path.split("?", 1)[0].split("#", 1)[0].strip("/")
parts = path.split("/")
if len(parts) >= 3 and all(parts[:2]) and parts[2] in _GITHUB_BROWSER_SEGMENTS:
repo = parts[1].removesuffix(".git")
subdir = None
if parts[2] == "tree" and len(parts) >= 5:
subdir = "/".join(p for p in parts[4:] if p).strip("/") or None
return f"https://github.com/{parts[0]}/{repo}.git", subdir
# Explicit ``#subdir`` fragment — unambiguous for any scheme.
if "#" in identifier:
git_url, _, frag = identifier.partition("#")
+2 -2
View File
@@ -784,9 +784,9 @@ def create_profile(
Path
The newly created profile directory.
"""
if no_skills and (clone_config or clone_all):
if no_skills and (clone_from is not None or clone_config or clone_all):
raise ValueError(
"--no-skills is mutually exclusive with --clone / --clone-all "
"--no-skills is mutually exclusive with --clone / --clone-from / --clone-all "
"(cloning explicitly copies skills from the source profile)."
)
canon = normalize_profile_name(name)
+3 -3
View File
@@ -35,17 +35,17 @@ def build_profile_parser(subparsers, *, cmd_profile: Callable) -> None:
profile_create.add_argument(
"--clone",
action="store_true",
help="Copy config.yaml, .env, SOUL.md from active profile",
help="Copy config.yaml, .env, SOUL.md, and skills from active profile",
)
profile_create.add_argument(
"--clone-all",
action="store_true",
help="Full copy of active profile (all state)",
help="Full copy of active profile (all state, excluding per-profile history)",
)
profile_create.add_argument(
"--clone-from",
metavar="SOURCE",
help="Source profile to clone from (default: active)",
help="Source profile to clone from; implies --clone unless --clone-all is set",
)
profile_create.add_argument(
"--no-alias", action="store_true", help="Skip wrapper script creation"
+43 -14
View File
@@ -1230,7 +1230,12 @@ def _managed_files_policy(request: Request, *, create_root: bool = True) -> Mana
root = _ensure_managed_root(raw_forced_root) if create_root else _canonical_path(Path(raw_forced_root))
return ManagedFilesPolicy(default_path=root, locked_root=root, can_change_path=False)
if not _local_dashboard_request(request) or _default_hermes_root_is_opt_data():
# Remote/OAuth access does not imply a hosted container. Users can expose a
# local dashboard through the auth gate (for example a macOS launchd install)
# and still expect the Files page to browse their local home directory. Lock
# to /opt/data only when the installation's Hermes root is actually /opt/data
# (the container/hosted layout) or when HERMES_DASHBOARD_FILES_ROOT is set.
if _default_hermes_root_is_opt_data():
root = _ensure_managed_root(_HOSTED_MANAGED_FILES_ROOT) if create_root else _HOSTED_MANAGED_FILES_ROOT
return ManagedFilesPolicy(default_path=root, locked_root=root, can_change_path=False)
@@ -1640,17 +1645,16 @@ async def get_status():
# Module not importable yet (early startup) — leave as [].
pass
return {
# Always-public liveness + auth-gate shape. Safe for external uptime
# probes (NAS's wildcard-subdomain liveness probe), the SPA's pre-login
# bootstrap, and anyone who can curl the host — i.e. exactly the audience
# ``PUBLIC_API_PATHS`` documents this endpoint as serving.
status = {
"version": __version__,
"release_date": __release_date__,
"hermes_home": str(get_hermes_home()),
"config_path": str(get_config_path()),
"env_path": str(get_env_path()),
"config_version": current_ver,
"latest_config_version": latest_ver,
"gateway_running": gateway_running,
"gateway_pid": gateway_pid,
"gateway_health_url": _GATEWAY_HEALTH_URL,
"gateway_state": gateway_state,
"gateway_platforms": gateway_platforms,
"gateway_exit_reason": gateway_exit_reason,
@@ -1660,6 +1664,27 @@ async def get_status():
"auth_providers": auth_providers,
}
# Absolute host paths, the gateway PID, and the internal gateway health
# URL are deployment recon a liveness probe never needs. ``/api/status``
# is in ``PUBLIC_API_PATHS`` so it bypasses dashboard auth; on a
# network-exposed (gated) bind that means *any* unauthenticated caller
# reaches it, and leaking host metadata there contradicts the allowlist's
# own contract ("version, gateway state, active session count, and the
# dashboard auth-gate shape. No bodies, no session content, no secrets").
# Surface this detail only on a loopback / ``--insecure`` bind, where the
# dashboard is local-only and the caller is already inside the trust
# envelope — the same loopback/gated split ``should_require_auth`` draws.
if not auth_required:
status.update({
"hermes_home": str(get_hermes_home()),
"config_path": str(get_config_path()),
"env_path": str(get_env_path()),
"gateway_pid": gateway_pid,
"gateway_health_url": _GATEWAY_HEALTH_URL,
})
return status
_WINDOWS_11_MIN_BUILD = 22000
@@ -8493,15 +8518,13 @@ async def scan_skill_hub(identifier: str = ""):
class ProfileCreate(BaseModel):
name: str
clone_from: Optional[str] = None
# Backward compatibility for older dashboard/desktop clients. New clients
# send clone_from="default" (or another profile name) explicitly.
clone_from_default: bool = False
clone_all: bool = False
no_skills: bool = False
description: Optional[str] = None
# Explicit source profile to clone from (e.g. duplicating an existing
# profile). When set, it takes precedence over ``clone_from_default``,
# which always sources from "default". ``clone_all`` still selects a full
# state copytree vs. a config/skills/SOUL copy.
clone_from: Optional[str] = None
provider: Optional[str] = None
model: Optional[str] = None
# Profile-builder additions — all optional, all applied best-effort AFTER
@@ -8773,10 +8796,16 @@ async def create_profile_endpoint(body: ProfileCreate):
clone = True
clone_from = explicit_source
clone_config = not body.clone_all
elif body.clone_all:
# Preserve the dashboard's historical clone-all behavior: a full-copy
# request with no explicit dropdown source copies from default.
clone = True
clone_from = "default"
clone_config = False
else:
clone = body.clone_from_default or body.clone_all
clone = body.clone_from_default
clone_from = "default" if clone else None
clone_config = body.clone_from_default and not body.clone_all
clone_config = clone
try:
path = profiles_mod.create_profile(
name=body.name,
+21 -2
View File
@@ -3246,7 +3246,11 @@ class AIAgent:
return sanitize_api_messages(messages)
@staticmethod
def _is_thinking_only_assistant(msg: Dict[str, Any]) -> bool:
def _is_thinking_only_assistant(
msg: Dict[str, Any],
*,
drop_codex_reasoning_items: bool = True,
) -> bool:
"""Return True if ``msg`` is an assistant turn whose only payload is reasoning.
"Thinking-only" means the model emitted reasoning (``reasoning`` or
@@ -3297,15 +3301,30 @@ class AIAgent:
rd = msg.get("reasoning_details")
if isinstance(rd, list) and rd:
return True
# Codex Responses stores encrypted reasoning state under a separate
# assistant-message key. Treat only real reasoning items as
# thinking-only; empty/junk lists should fall through to the generic
# empty-turn handling instead of being dropped here.
codex_items = msg.get("codex_reasoning_items")
if drop_codex_reasoning_items and isinstance(codex_items, list):
return any(
isinstance(item, dict) and item.get("type") == "reasoning"
for item in codex_items
)
return False
@staticmethod
def _drop_thinking_only_and_merge_users(
messages: List[Dict[str, Any]],
*,
drop_codex_reasoning_items: bool = True,
) -> List[Dict[str, Any]]:
"""Forwarder — see ``agent.agent_runtime_helpers.drop_thinking_only_and_merge_users``."""
from agent.agent_runtime_helpers import drop_thinking_only_and_merge_users
return drop_thinking_only_and_merge_users(messages)
return drop_thinking_only_and_merge_users(
messages,
drop_codex_reasoning_items=drop_codex_reasoning_items,
)
@staticmethod
def _cap_delegate_task_calls(tool_calls: list) -> list:
+5
View File
@@ -81,6 +81,8 @@ AUTHOR_MAP = {
"290859878+synapsesx@users.noreply.github.com": "synapsesx",
"157689911+itsflownium@users.noreply.github.com": "itsflownium",
"dirtyren@users.noreply.github.com": "dirtyren",
"achaljhawar03@gmail.com": "achaljhawar",
"claytonchew@ClaytonMacMiniM4.local": "claytonchew",
"hbentel@gmail.com": "hbentel",
"JustinBao@outlook.com": "justinbao19",
"kdunn926@gmail.com": "kdunn926",
@@ -494,6 +496,7 @@ AUTHOR_MAP = {
"20nik.nosov21@gmail.com": "nik1t7n",
"90299797+nik1t7n@users.noreply.github.com": "nik1t7n",
"suncokret@protonmail.com": "suncokret12",
"WompaJango@protonmail.com": "WompaJango",
"mio.imoto.ai@gmail.com": "mioimotoai-lgtm",
"aamirjawaid@microsoft.com": "heyitsaamir",
"johnnncenaaa77@gmail.com": "johnncenae",
@@ -989,6 +992,7 @@ AUTHOR_MAP = {
"tuancanhnguyen706@gmail.com": "xxxigm",
"larcombe.n@gmail.com": "NickLarcombe",
"54813621+xxxigm@users.noreply.github.com": "xxxigm",
"xxxigm@users.noreply.github.com": "xxxigm",
"asurla@nvidia.com": "anniesurla",
"kchantharuan@nvidia.com": "nv-kasikritc",
"bbednarski@nvidia.com": "bbednarski9",
@@ -1527,6 +1531,7 @@ AUTHOR_MAP = {
"chanhokyim@gmail.com": "joel611", # PR #33958 salvage (DISCORD_ALLOWED_ROLES role_authorized gateway flag)
"desg38@gmail.com": "dschnurbusch", # PR #42373 salvage (archive compressed conversation lineages)
"bsmith@bramarstrategicservices.com": "bcsmith528", # PR #20589 salvage (register_slack_action_handler plugin API)
"sunsky.lau@gmail.com": "liuhao1024", # PR #45494 salvage (claim session slot before auto-resume task; #45456)
}
+41
View File
@@ -146,6 +146,47 @@ class TestBuildCallKwargsMaxTokens:
assert "max_completion_tokens" not in kwargs
class TestNousTagsScoping:
def test_tags_injected_when_provider_is_nous(self, monkeypatch):
import agent.auxiliary_client as aux
monkeypatch.setattr(aux, "auxiliary_is_nous", False)
kwargs = aux._build_call_kwargs(
provider="nous",
model="hermes-4",
messages=[{"role": "user", "content": "hi"}],
)
assert kwargs["extra_body"]["tags"] == aux._nous_portal_tags()
def test_tags_not_injected_for_gemini_when_main_is_nous(self, monkeypatch):
import agent.auxiliary_client as aux
monkeypatch.setattr(aux, "auxiliary_is_nous", True)
kwargs = aux._build_call_kwargs(
provider="gemini",
model="gemini-2.5-flash",
messages=[{"role": "user", "content": "hi"}],
)
assert "extra_body" not in kwargs
def test_tags_not_injected_for_openrouter_when_main_is_nous(self, monkeypatch):
import agent.auxiliary_client as aux
monkeypatch.setattr(aux, "auxiliary_is_nous", True)
kwargs = aux._build_call_kwargs(
provider="openrouter",
model="openai/gpt-5.4",
messages=[{"role": "user", "content": "hi"}],
)
assert "extra_body" not in kwargs
class TestNormalizeAuxProvider:
def test_maps_github_copilot_aliases(self):
assert _normalize_aux_provider("github") == "copilot"
+68
View File
@@ -605,6 +605,74 @@ class TestBuildConverseKwargs:
assert kwargs["inferenceConfig"]["temperature"] == 0.7
assert kwargs["inferenceConfig"]["topP"] == 0.9
def test_omits_sampling_params_for_bedrock_opus_4_7(self):
from agent.bedrock_adapter import build_converse_kwargs
for model_id in (
"anthropic.claude-opus-4-7-20260101-v1:0",
"us.anthropic.claude-opus-4-7",
):
kwargs = build_converse_kwargs(
model=model_id,
messages=[{"role": "user", "content": "Hi"}],
temperature=0.7,
top_p=0.9,
)
assert "temperature" not in kwargs["inferenceConfig"]
assert "topP" not in kwargs["inferenceConfig"]
def test_omits_sampling_params_for_bedrock_opus_4_8_variants(self):
from agent.bedrock_adapter import build_converse_kwargs
for model_id in (
"anthropic.claude-opus-4-8-20270101-v1:0",
"us.anthropic.claude-opus-4-8",
"anthropic.claude-opus-4.8",
):
kwargs = build_converse_kwargs(
model=model_id,
messages=[{"role": "user", "content": "Hi"}],
temperature=0.5,
top_p=0.95,
)
assert "temperature" not in kwargs["inferenceConfig"]
assert "topP" not in kwargs["inferenceConfig"]
def test_keeps_sampling_params_for_bedrock_non_restricted_models(self):
from agent.bedrock_adapter import build_converse_kwargs
for model_id in (
"anthropic.claude-sonnet-4-6-20250514-v1:0",
"anthropic.claude-haiku-4-5",
"test-model",
):
kwargs = build_converse_kwargs(
model=model_id,
messages=[{"role": "user", "content": "Hi"}],
temperature=0.7,
top_p=0.9,
)
assert kwargs["inferenceConfig"].get("temperature") == 0.7
assert kwargs["inferenceConfig"].get("topP") == 0.9
def test_bedrock_opus_strips_sampling_params_but_keeps_stop_sequences(self):
from agent.bedrock_adapter import build_converse_kwargs
kwargs = build_converse_kwargs(
model="us.anthropic.claude-opus-4-8",
messages=[{"role": "user", "content": "Hi"}],
temperature=0.7,
top_p=0.9,
stop_sequences=["END"],
)
assert "temperature" not in kwargs["inferenceConfig"]
assert "topP" not in kwargs["inferenceConfig"]
assert kwargs["inferenceConfig"]["stopSequences"] == ["END"]
def test_includes_guardrail_config(self):
from agent.bedrock_adapter import build_converse_kwargs
guardrail = {
+39
View File
@@ -198,6 +198,45 @@ def test_native_client_uses_x_goog_api_key_and_native_models_endpoint(monkeypatc
assert response.choices[0].message.content == "hello"
@pytest.mark.parametrize("model, expected", [
("google/gemini-2.0-flash", "gemini-2.0-flash"),
("gemini/gemini-3-pro-preview", "gemini-3-pro-preview"),
("Google/Gemini-2.5-Pro", "Gemini-2.5-Pro"),
("models/gemini-x", "models/gemini-x"),
("tunedModels/my-tune", "tunedModels/my-tune"),
])
def test_bare_gemini_model_id_strips_only_self_prefix(model, expected):
from agent.gemini_native_adapter import bare_gemini_model_id
assert bare_gemini_model_id(model) == expected
def test_native_client_strips_self_prefix_from_model_url(monkeypatch):
from agent.gemini_native_adapter import GeminiNativeClient
recorded = {}
class DummyHTTP:
def post(self, url, json=None, headers=None, timeout=None):
recorded["url"] = url
return DummyResponse(payload={
"candidates": [{"content": {"parts": [{"text": "ok"}]}, "finishReason": "STOP"}],
"usageMetadata": {"promptTokenCount": 1, "candidatesTokenCount": 1, "totalTokenCount": 2},
})
def close(self):
return None
monkeypatch.setattr("agent.gemini_native_adapter.httpx.Client", lambda *a, **k: DummyHTTP())
client = GeminiNativeClient(api_key="AIza-test", base_url="https://generativelanguage.googleapis.com/v1beta")
client.chat.completions.create(
model="google/gemini-2.0-flash",
messages=[{"role": "user", "content": "Hello"}],
)
assert recorded["url"] == "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent"
def test_native_http_error_keeps_status_and_retry_after():
from agent.gemini_native_adapter import gemini_http_error
+5
View File
@@ -887,6 +887,11 @@ class TestPromptBuilderConstants:
assert "table" in lowered
assert "task list" in lowered
assert "math" in lowered
# Hint should proactively steer toward structured formatting, not just
# permit it: bullet + numbered lists for scannable, structured output.
assert "bullet" in lowered
assert "numbered" in lowered
# Local media delivery guidance must remain intact.
assert "include MEDIA:" in hint
def test_platform_hints_mattermost(self):
@@ -155,6 +155,46 @@ class TestCodexBuildKwargs:
)
assert "max_output_tokens" not in kw
def test_codex_backend_does_not_set_extra_headers(self, transport):
messages = [{"role": "user", "content": "Hi"}]
kw = transport.build_kwargs(
model="gpt-5.4",
messages=messages,
tools=[],
session_id="conv-codex-1",
is_codex_backend=True,
)
assert "extra_headers" not in kw
def test_codex_backend_strips_caller_extra_headers(self, transport):
messages = [{"role": "user", "content": "Hi"}]
kw = transport.build_kwargs(
model="gpt-5.4",
messages=messages,
tools=[],
session_id="conv-codex-1",
is_codex_backend=True,
request_overrides={"extra_headers": {"x-test": "1"}},
)
assert "extra_headers" not in kw
def test_non_codex_responses_preserves_caller_extra_headers(self, transport):
messages = [{"role": "user", "content": "Hi"}]
kw = transport.build_kwargs(
model="gpt-5.4",
messages=messages,
tools=[],
is_codex_backend=False,
request_overrides={"extra_headers": {"x-test": "1"}},
)
assert kw["extra_headers"] == {"x-test": "1"}
def test_xai_headers(self, transport):
messages = [{"role": "user", "content": "Hi"}]
kw = transport.build_kwargs(
+118 -19
View File
@@ -8,16 +8,21 @@ a message is dropped inside the adapter and never reaches the gateway unless it
already passed that policy.
The gateway's env-based allowlist check (``_is_user_authorized``) runs *after*
the adapter. Before the fix it fell through to an env-only default-deny when no
``PLATFORM_ALLOWED_USERS`` env var was set, silently rejecting ``dm_policy:
open`` and config-only allowlists even though the adapter had already
authorized the sender.
the adapter. Adapters that own their access policy declare
``enforces_own_access_policy`` (a ``BasePlatformAdapter`` property, default
``False``) so the gateway can honor a config-only ``dm_policy: allowlist`` /
``allow_from`` (which the adapter already enforced) instead of double-denying it
when no ``PLATFORM_ALLOWED_USERS`` env var is set.
The fix is a single drift-proof contract: adapters that own their access policy
declare ``enforces_own_access_policy`` (a ``BasePlatformAdapter`` property,
default ``False``). The gateway trusts that flag and skips the env-only
default-deny for those platforms, rather than re-implementing each adapter's
policy logic a second time.
Crucially, the flag is NOT a blanket "already authorized" pass. These adapters
default ``dm_policy`` / ``group_policy`` to ``"open"``, which forwards *every*
sender, so the gateway trusts the adapter only when its effective policy for the
chat type is an actual ``"allowlist"`` restriction. Trusting ``"open"`` here
admitted the whole external network with no operator-configured allowlist the
fail-open SECURITY.md §2.6 forbids for network-exposed adapters ("an allowlist
is required for every enabled network-exposed adapter ... code paths that fail
open when no allowlist is configured are code bugs"). Open access requires an
explicit ``{PLATFORM}_ALLOW_ALL_USERS`` / ``GATEWAY_ALLOW_ALL_USERS`` opt-in.
"""
from types import SimpleNamespace
@@ -128,15 +133,16 @@ def test_own_policy_adapters_declare_the_flag(module_path, class_name):
@pytest.mark.parametrize("platform", _OWN_POLICY_PLATFORMS)
def test_own_policy_platform_authorized_without_env_allowlist(monkeypatch, platform):
"""A message reaching the gateway from an own-policy adapter is trusted.
def test_own_policy_allowlist_authorized_without_env_allowlist(monkeypatch, platform):
"""A config-only ``dm_policy: allowlist`` is trusted without an env allowlist.
With no env allowlist set, the gateway must NOT default-deny the adapter
already authorized the sender at intake (e.g. ``dm_policy: open``).
The adapter only forwards an allowlisted sender under ``allowlist`` policy,
so a message reaching the gateway *was* authorized for this specific sender.
The gateway must honor that instead of double-denying (the #34515 case).
"""
_clear_auth_env(monkeypatch)
config = GatewayConfig(
platforms={platform: PlatformConfig(enabled=True, extra={"dm_policy": "open"})}
platforms={platform: PlatformConfig(enabled=True, extra={"dm_policy": "allowlist"})}
)
runner, _adapter = _make_runner(platform, config, enforces=True)
@@ -144,15 +150,104 @@ def test_own_policy_platform_authorized_without_env_allowlist(monkeypatch, platf
@pytest.mark.parametrize("platform", _OWN_POLICY_PLATFORMS)
def test_own_policy_platform_authorized_for_group_chat(monkeypatch, platform):
"""Group traffic from an own-policy adapter is trusted the same way."""
def test_own_policy_open_dm_not_authorized_without_allowlist(monkeypatch, platform):
"""``dm_policy: open`` forwards everyone → NOT authorization (SECURITY.md §2.6).
With no env allowlist and no per-platform allow-all flag, an own-policy
adapter running ``open`` (the default) must NOT fail open: the gateway falls
through to default-deny so the whole external network can't reach the agent.
"""
_clear_auth_env(monkeypatch)
config = GatewayConfig(
platforms={platform: PlatformConfig(enabled=True, extra={"dm_policy": "open"})}
)
runner, _adapter = _make_runner(platform, config, enforces=True)
assert runner._is_user_authorized(_source(platform)) is False
@pytest.mark.parametrize("platform", _OWN_POLICY_PLATFORMS)
def test_own_policy_default_open_dm_is_fail_closed(monkeypatch, platform):
"""The adapters' *default* ``open`` policy (no config at all) fails closed.
Operators who enable an own-policy adapter with only credentials get
``dm_policy = "open"`` resolved on the live adapter. Simulate that resolved
state (empty config.extra, adapter ``_dm_policy = "open"``) and confirm the
gateway denies the do-nothing default must not be open to the world.
"""
_clear_auth_env(monkeypatch)
config = GatewayConfig(platforms={platform: PlatformConfig(enabled=True, extra={})})
runner, adapter = _make_runner(platform, config, enforces=True)
adapter._dm_policy = "open" # as the live adapter resolves the default
assert runner._is_user_authorized(_source(platform)) is False
@pytest.mark.parametrize("platform", _OWN_POLICY_PLATFORMS)
def test_own_policy_allowlist_authorized_for_group_chat(monkeypatch, platform):
"""A config-only ``group_policy: allowlist`` is trusted for group traffic."""
_clear_auth_env(monkeypatch)
config = GatewayConfig(
platforms={platform: PlatformConfig(enabled=True, extra={"group_policy": "allowlist"})}
)
runner, _adapter = _make_runner(platform, config, enforces=True)
assert runner._is_user_authorized(_source(platform, chat_type="group")) is True
@pytest.mark.parametrize("platform", _OWN_POLICY_PLATFORMS)
def test_own_policy_open_group_not_authorized_without_allowlist(monkeypatch, platform):
"""``group_policy: open`` is the same fail-open class as DM open → deny."""
_clear_auth_env(monkeypatch)
config = GatewayConfig(
platforms={platform: PlatformConfig(enabled=True, extra={"group_policy": "open"})}
)
runner, _adapter = _make_runner(platform, config, enforces=True)
assert runner._is_user_authorized(_source(platform, chat_type="group")) is True
assert runner._is_user_authorized(_source(platform, chat_type="group")) is False
def test_wecom_open_group_with_per_group_sender_allowlist_is_authorized(monkeypatch):
"""WeCom ``groups.<id>.allow_from`` is an adapter-enforced restriction.
The top-level group policy is still ``open`` for the chat ID, but the
adapter has already checked the sender allowlist before dispatching to the
gateway. That is not the fail-open case and must not be double-denied.
"""
_clear_auth_env(monkeypatch)
config = GatewayConfig(
platforms={
Platform.WECOM: PlatformConfig(
enabled=True,
extra={
"group_policy": "open",
"groups": {"some-chat": {"allow_from": ["some-user"]}},
},
)
}
)
runner, _adapter = _make_runner(Platform.WECOM, config, enforces=True)
assert runner._is_user_authorized(_source(Platform.WECOM, chat_type="group")) is True
def test_wecom_open_group_with_wildcard_sender_allowlist_is_authorized(monkeypatch):
"""Wildcard group config also gates senders before gateway auth runs."""
_clear_auth_env(monkeypatch)
config = GatewayConfig(
platforms={
Platform.WECOM: PlatformConfig(
enabled=True,
extra={
"group_policy": "open",
"groups": {"*": {"allow_from": ["user_admin"]}},
},
)
}
)
runner, _adapter = _make_runner(Platform.WECOM, config, enforces=True)
assert runner._is_user_authorized(_source(Platform.WECOM, chat_type="group")) is True
def test_non_owning_platform_still_default_denies(monkeypatch):
@@ -259,12 +354,16 @@ def test_pairing_carveout_reads_adapter_when_env_set(monkeypatch):
def test_pairing_dm_policy_group_chat_still_trusted(monkeypatch):
"""Pairing is DM-only — group traffic keeps the adapter-trust path."""
"""Pairing is DM-only — the DM pairing carve-out doesn't gate group traffic.
Group access is governed by ``group_policy``, so an allowlisted group is
still trusted even while DMs are in ``pairing`` mode.
"""
_clear_auth_env(monkeypatch)
config = GatewayConfig(
platforms={
Platform.WECOM: PlatformConfig(
enabled=True, extra={"dm_policy": "pairing", "group_policy": "open"}
enabled=True, extra={"dm_policy": "pairing", "group_policy": "allowlist"}
)
}
)
+12 -1
View File
@@ -151,7 +151,18 @@ class TestSupportedDocumentTypes:
@pytest.mark.parametrize(
"ext",
[".pdf", ".md", ".txt", ".zip", ".docx", ".xlsx", ".pptx"],
[
".pdf",
".md",
".txt",
".zip",
".doc",
".docx",
".xls",
".xlsx",
".ppt",
".pptx",
],
)
def test_expected_extensions_present(self, ext):
assert ext in SUPPORTED_DOCUMENT_TYPES
+18
View File
@@ -1090,6 +1090,24 @@ class TestMatrixMarkdownToHtml:
assert "<code" in result
assert "print" in result
def test_matrix_markdown_preserves_table_structure(self):
table = "\n".join(
[
"| Item | Quantity |",
"| --- | --- |",
"| Apples | 4 |",
"| Bread | 1 |",
]
)
result = self.adapter._markdown_to_html(table)
assert "<table>" in result
assert "<thead>" in result
assert "<tbody>" in result
assert "<th>Item</th>" in result
assert "<td>Apples</td>" in result
# ---------------------------------------------------------------------------
# Helper: display name extraction
@@ -35,6 +35,7 @@ import pytest
from gateway.config import GatewayConfig, HomeChannel, Platform
from gateway.platforms.base import MessageEvent, MessageType, SendResult
from gateway.run import (
_AGENT_PENDING_SENTINEL,
_auto_continue_freshness_window,
_coerce_gateway_timestamp,
_is_fresh_gateway_interruption,
@@ -1420,3 +1421,194 @@ class TestStuckLoopEscalation:
{"indent": None},
)
]
@pytest.mark.asyncio
async def test_auto_resume_sets_sentinel_before_task_execution():
"""Auto-resume must claim the session slot before the task starts.
Regression for #45456: between ``asyncio.create_task()`` and the task's
first await (where ``_process_message_background`` sets the real
sentinel), an inbound message could arrive and spin up a duplicate
AIAgent. The fix pre-claims the slot so the inbound path sees it as
occupied.
"""
runner, adapter = make_restart_runner()
source = make_restart_source(chat_id="race-chat")
pending_entry = SessionEntry(
session_key="agent:main:telegram:dm:race-chat",
session_id="sid",
created_at=datetime.now(),
updated_at=datetime.now(),
origin=source,
platform=Platform.TELEGRAM,
chat_type="dm",
resume_pending=True,
resume_reason="restart_interrupted",
last_resume_marked_at=datetime.now(),
)
runner.session_store._entries = {pending_entry.session_key: pending_entry}
# Slow mock: hold the task open so we can inspect _running_agents
# while it's in-flight.
gate = asyncio.Event()
async def _slow_handle(event):
await gate.wait()
adapter.handle_message = _slow_handle
scheduled = runner._schedule_resume_pending_sessions()
assert scheduled == 1
# The sentinel must be set immediately — before the task starts executing.
assert pending_entry.session_key in runner._running_agents
assert runner._running_agents[pending_entry.session_key] is _AGENT_PENDING_SENTINEL
assert pending_entry.session_key in runner._running_agents_ts
# Release the task and let it complete.
gate.set()
await asyncio.sleep(0.05)
# After the task completes, the sentinel should be cleaned up.
assert pending_entry.session_key not in runner._running_agents
@pytest.mark.asyncio
async def test_auto_resume_sentinel_cleaned_on_task_failure():
"""If handle_message raises before _process_message_background, the
sentinel must still be released so the session is not locked forever.
"""
runner, adapter = make_restart_runner()
source = make_restart_source(chat_id="fail-chat")
pending_entry = SessionEntry(
session_key="agent:main:telegram:dm:fail-chat",
session_id="sid",
created_at=datetime.now(),
updated_at=datetime.now(),
origin=source,
platform=Platform.TELEGRAM,
chat_type="dm",
resume_pending=True,
resume_reason="restart_interrupted",
last_resume_marked_at=datetime.now(),
)
runner.session_store._entries = {pending_entry.session_key: pending_entry}
async def _failing_handle(event):
raise RuntimeError("adapter exploded")
adapter.handle_message = _failing_handle
scheduled = runner._schedule_resume_pending_sessions()
assert scheduled == 1
# Sentinel is set immediately.
assert pending_entry.session_key in runner._running_agents
# Let the task run and fail.
await asyncio.sleep(0.05)
# The sentinel must be cleaned up despite the failure.
assert pending_entry.session_key not in runner._running_agents
@pytest.mark.asyncio
async def test_auto_resume_runs_agent_exactly_once_through_full_path():
"""Full-path regression: the pre-claim must NOT make auto-resume a no-op.
The two tests above mock ``adapter.handle_message`` outright, so they
only prove the sentinel is set/cleaned around a stub they never
exercise the real dispatch chain. This drives the production path
end to end:
_schedule_resume_pending_sessions
-> _guarded_handle_message
-> adapter.handle_message (real)
-> _process_message_background (real)
-> _handle_message (real)
The risk the pre-claim introduces is a *self-bounce*: the resume
turn's own ``_handle_message`` sees the sentinel it pre-claimed at
the early running-agent guard, queues the event into
``_pending_messages`` and returns ``None`` without running the
agent. The adapter's late-arrival drain (in
``_process_message_background``'s ``finally``) re-dispatches the
queued event, and because the guard wrapper's ``finally`` releases
the pre-claim before the spawned drain task starts, the agent runs
exactly once. This test locks that invariant in: the resume agent
must run once never zero (regression) and never twice (the bug
the fix targets).
"""
runner, adapter = make_restart_runner()
source = make_restart_source(chat_id="full-path-chat")
session_key = runner._session_key_for_source(source)
pending_entry = SessionEntry(
session_key=session_key,
session_id="sid",
created_at=datetime.now(),
updated_at=datetime.now(),
origin=source,
platform=Platform.TELEGRAM,
chat_type="dm",
resume_pending=True,
resume_reason="restart_interrupted",
last_resume_marked_at=datetime.now(),
)
runner.session_store._entries = {session_key: pending_entry}
# Wire the REAL runner pipeline that _handle_message depends on.
from gateway.run import GatewayRunner
runner._handle_message = GatewayRunner._handle_message.__get__(
runner, GatewayRunner
)
runner._release_running_agent_state = (
GatewayRunner._release_running_agent_state.__get__(runner, GatewayRunner)
)
runner._check_slash_access = lambda *a, **k: None
runner._begin_session_run_generation = lambda session_key: 1
runner._is_session_run_current = lambda session_key, generation: True
runner._invalidate_session_run_generation = lambda *a, **k: 0
runner._claim_active_session_slot = lambda session_key, source: (object(), None)
runner._active_session_leases = {}
runner._busy_ack_ts = {}
runner._post_turn_goal_continuation = AsyncMock()
runner.session_store.get_or_create_session.return_value = None
# Count how many times an actual agent run is started for this session.
agent_runs: list[str] = []
async def _fake_run(event, source, _quick_key, run_generation):
agent_runs.append(_quick_key)
return "RESUMED OK"
runner._handle_message_with_agent = _fake_run
# Route the adapter's real background pipeline at the real handler,
# and stub the leaf send/typing calls so delivery is a no-op.
adapter.set_message_handler(runner._handle_message)
adapter.send = AsyncMock()
adapter._keep_typing = AsyncMock()
adapter._stop_typing_refresh = AsyncMock()
adapter._send_with_retry = AsyncMock(
return_value=SendResult(success=True, message_id="1")
)
adapter._run_processing_hook = AsyncMock()
scheduled = runner._schedule_resume_pending_sessions()
assert scheduled == 1
# Pre-claim must be visible immediately.
assert runner._running_agents.get(session_key) is _AGENT_PENDING_SENTINEL
# Let the guarded task, the background task, and the late-arrival
# drain task all settle.
for _ in range(20):
await asyncio.sleep(0.02)
# Exactly one agent run for the resumed session — not zero (the
# pre-claim did not swallow the resume) and not two (no duplicate).
assert agent_runs == [session_key]
# No leaked sentinel and no orphaned queued event.
assert session_key not in runner._running_agents
assert session_key not in getattr(adapter, "_pending_messages", {})
+186
View File
@@ -321,3 +321,189 @@ class TestAlreadySentInDraftMode:
# After the regular sendMessage finalize, _already_sent is True.
assert consumer._already_sent is True
def _make_fresh_final_adapter():
"""Build a non-draft adapter that prefers a fresh final send.
Mirrors Telegram's rich-message contract: REQUIRES_EDIT_FINALIZE so the
final tick is routed through even when the text is unchanged, and
prefers_fresh_final_streaming() True so the consumer delivers the final
answer via a *fresh* send + preview delete instead of an edit.
``send`` returns two distinct ids so the test can tell the preview
(first send) from the fresh final (second send) and assert the preview
is the one deleted.
"""
from gateway.platforms.base import BasePlatformAdapter, SendResult
FreshFinalAdapter = type(
"FreshFinalAdapter",
(BasePlatformAdapter,),
{"MAX_MESSAGE_LENGTH": 4096, "REQUIRES_EDIT_FINALIZE": True},
)
FreshFinalAdapter.__abstractmethods__ = frozenset()
adapter = FreshFinalAdapter.__new__(FreshFinalAdapter)
adapter._typing_paused = set()
adapter._fatal_error_message = None
# Edit-based path only — no native drafts.
adapter.supports_draft_streaming = lambda chat_type=None, metadata=None: False
# Accepts the metadata kwarg the consumer passes; ignores it (like Telegram).
adapter.prefers_fresh_final_streaming = lambda content, metadata=None: True
adapter.send = AsyncMock(side_effect=[
SendResult(success=True, message_id="preview1"),
SendResult(success=True, message_id="final1"),
])
adapter.edit_message = AsyncMock(return_value=SendResult(success=True))
adapter.delete_message = AsyncMock(return_value=True)
return adapter
class TestAdapterPrefersFreshFinal:
"""An adapter whose send path is richer than its edit path (e.g. Telegram
rich messages) finalizes a streamed reply by sending a fresh final message
and deleting the preview, instead of final-editing the preview."""
@pytest.mark.asyncio
async def test_edit_stream_finalizes_with_fresh_send_and_deletes_preview(self):
adapter = _make_fresh_final_adapter()
cfg = StreamConsumerConfig(
transport="auto", chat_type="dm",
edit_interval=0.01, buffer_threshold=5, cursor="",
fresh_final_after_seconds=0.0, # only the adapter hook drives fresh-final
)
consumer = GatewayStreamConsumer(adapter, "12345", cfg)
consumer.on_delta("Full answer here")
task = asyncio.create_task(consumer.run())
# Let the first send land so a real preview message_id exists before
# finalization — the fresh-final path only engages with a live preview.
await asyncio.sleep(0.05)
consumer.finish()
await task
# Two sends: the streaming preview, then the fresh final.
assert adapter.send.await_count == 2
first_content = adapter.send.call_args_list[0].kwargs.get("content")
second_content = adapter.send.call_args_list[1].kwargs.get("content")
# First update delivered the preview via adapter.send.
assert first_content == "Full answer here"
# Finalization re-sent the same completed content as a fresh message.
assert second_content == "Full answer here"
# The edit path must NOT be used to finalize a rich preview.
adapter.edit_message.assert_not_called()
# The stale preview is best-effort deleted (by its id, not the final's).
adapter.delete_message.assert_awaited_once_with("12345", "preview1")
assert consumer.final_response_sent is True
def _make_rich_capable_adapter(*, overflow_limit=32768, send_results=None):
"""Non-draft adapter that mimics Telegram rich messages: REQUIRES_EDIT_FINALIZE,
prefers a fresh (rich) final send, and reports a 32,768 streaming overflow
limit so the consumer doesn't pre-split a reply that fits one rich message.
"""
from gateway.platforms.base import BasePlatformAdapter, SendResult
RichAdapter = type(
"RichCapableAdapter",
(BasePlatformAdapter,),
{"MAX_MESSAGE_LENGTH": 4096, "REQUIRES_EDIT_FINALIZE": True},
)
RichAdapter.__abstractmethods__ = frozenset()
adapter = RichAdapter.__new__(RichAdapter)
adapter._typing_paused = set()
adapter._fatal_error_message = None
adapter.supports_draft_streaming = lambda chat_type=None, metadata=None: False
adapter.prefers_fresh_final_streaming = lambda content, metadata=None: True
adapter.streaming_overflow_limit = lambda: overflow_limit
adapter.send = AsyncMock(side_effect=send_results) if send_results else AsyncMock(
return_value=SendResult(success=True, message_id="m1"),
)
adapter.edit_message = AsyncMock(return_value=SendResult(success=True))
adapter.delete_message = AsyncMock(return_value=True)
return adapter
class TestRichAwareOverflow:
"""Rich-capable adapters raise the consumer's overflow limit so a reply that
fits one rich message isn't fragmented at the legacy 4,096 edit limit."""
def test_raw_message_limit_uses_adapter_rich_cap(self):
adapter = _make_rich_capable_adapter(overflow_limit=32768)
consumer = GatewayStreamConsumer(adapter, "12345", StreamConsumerConfig())
assert consumer._raw_message_limit() == 32768
def test_raw_message_limit_falls_back_to_max_length(self):
# Adapter whose hook returns None (default) keeps the legacy limit.
adapter = _make_rich_capable_adapter()
adapter.streaming_overflow_limit = lambda: None
consumer = GatewayStreamConsumer(adapter, "12345", StreamConsumerConfig())
assert consumer._raw_message_limit() == 4096
def test_raw_message_limit_mock_adapter_is_safe(self):
# MagicMock adapters (many existing tests) must not crash or wrongly
# inflate the limit from a truthy auto-attribute.
adapter = MagicMock()
adapter.MAX_MESSAGE_LENGTH = 4096
consumer = GatewayStreamConsumer(adapter, "12345", StreamConsumerConfig())
assert consumer._raw_message_limit() == 4096
@pytest.mark.asyncio
async def test_long_rich_reply_not_split_and_final_is_whole(self):
from gateway.platforms.base import SendResult
long_text = "x" * 5000 # > 4096 legacy limit, < 32768 rich limit
adapter = _make_rich_capable_adapter(send_results=[
SendResult(success=True, message_id="preview1"),
SendResult(success=True, message_id="final1"),
])
cfg = StreamConsumerConfig(
transport="auto", chat_type="dm",
edit_interval=0.01, buffer_threshold=5, cursor="",
fresh_final_after_seconds=0.0,
)
consumer = GatewayStreamConsumer(adapter, "12345", cfg)
consumer.on_delta(long_text)
task = asyncio.create_task(consumer.run())
await asyncio.sleep(0.05)
consumer.finish()
await task
# Exactly two whole sends: the preview and the fresh final — NOT split
# into ~4096 chunks. Both carry the full 5,000-char reply.
assert adapter.send.await_count == 2
assert adapter.send.call_args_list[0].kwargs.get("content") == long_text
assert adapter.send.call_args_list[1].kwargs.get("content") == long_text
adapter.edit_message.assert_not_called()
adapter.delete_message.assert_awaited_once_with("12345", "preview1")
assert consumer.final_response_sent is True
@pytest.mark.asyncio
async def test_fresh_final_deletes_all_preview_fragments(self):
from gateway.platforms.base import SendResult
adapter = _make_rich_capable_adapter(send_results=[
SendResult(success=True, message_id="final1"),
])
consumer = GatewayStreamConsumer(adapter, "12345", StreamConsumerConfig())
# Simulate a reply that was split across the edit limit while streaming:
# three preview fragments, the last of which is the current message.
consumer._message_id = "frag3"
consumer._preview_message_ids = {"frag1", "frag2", "frag3"}
ok = await consumer._try_fresh_final("the whole completed answer")
assert ok is True
# All three stale fragments deleted; the fresh final never deleted.
deleted = {c.args[1] for c in adapter.delete_message.await_args_list}
assert deleted == {"frag1", "frag2", "frag3"}
assert "final1" not in deleted
assert consumer._message_id == "final1"
assert consumer._preview_message_ids == set()
assert consumer.final_response_sent is True
@@ -103,7 +103,8 @@ class TestInitialReplyToId:
await consumer._send_or_edit("Test")
call_kwargs = adapter.send.call_args[1]
assert call_kwargs["metadata"] == metadata
assert call_kwargs["metadata"] == {**metadata, "expect_edits": True}
assert metadata == {"thread_id": "omt_topic789"}
class TestOverflowFirstMessage:
+141 -2
View File
@@ -25,6 +25,20 @@ from telegram.error import BadRequest, NetworkError, TimedOut
# and a task list. Pipes / brackets must survive untouched into the payload.
RICH_CONTENT = "## Results\n\n| Case | Status |\n|---|---|\n| rich | ✅ |\n\n- [x] table renders"
# PTB 22.6's real unknown-endpoint errors: do_api_request can raise
# EndPointNotFound for Bot API 404s, and the request layer can wrap that same
# missing endpoint as InvalidToken. Use class names here so the tests don't
# depend on optional PTB internals.
EndPointNotFound = type("EndPointNotFound", (Exception,), {})
InvalidToken = type("InvalidToken", (Exception,), {})
PTB_ENDPOINT_NOT_FOUND = EndPointNotFound(
"Endpoint 'sendRichMessage' not found in Bot API"
)
PTB_INVALID_TOKEN_404 = InvalidToken(
"Either the bot token was rejected by Telegram or the endpoint "
"'sendRichMessage' does not exist."
)
def _make_adapter(extra=None):
"""Build a TelegramAdapter with a mock bot wired for the rich path."""
@@ -48,6 +62,31 @@ def _rich_api_kwargs(adapter):
return call.kwargs["api_kwargs"]
@pytest.mark.asyncio
@pytest.mark.parametrize(
("raw", "expected_id"),
[
(SimpleNamespace(message_id=123), "123"),
({"message_id": 123}, "123"),
({"result": {"message_id": 123}}, "123"),
({"result": None}, None),
],
)
async def test_rich_result_shapes_extract_message_id(raw, expected_id):
"""The raw Bot API path may return either a PTB object or a raw dict."""
adapter = _make_adapter()
adapter._bot.do_api_request = AsyncMock(return_value=raw)
result = await adapter.send("12345", RICH_CONTENT)
assert result.success is True
assert result.message_id == expected_id
bot = adapter._bot
assert bot is not None
bot.do_api_request.assert_awaited_once()
bot.send_message.assert_not_called()
@pytest.mark.asyncio
async def test_rich_happy_path_sends_raw_markdown():
adapter = _make_adapter()
@@ -79,12 +118,31 @@ async def test_legacy_rich_messages_config_is_ignored():
adapter._bot.send_message.assert_not_called()
@pytest.mark.asyncio
async def test_expect_edits_metadata_keeps_preview_on_legacy_path():
adapter = _make_adapter()
result = await adapter.send(
"12345",
RICH_CONTENT,
metadata={"expect_edits": True},
)
assert result.success is True
# Streaming preview sends will be edited later, so they must not be born as
# rich messages until Hermes wires rich_message edits directly.
bot = adapter._bot
assert bot is not None
bot.do_api_request.assert_not_called()
bot.send_message.assert_awaited()
@pytest.mark.asyncio
async def test_oversized_content_skips_rich_and_chunks():
adapter = _make_adapter()
# > 32,768 UTF-8 bytes -> rich pre-check fails, legacy chunking takes over.
# > 32,768 characters -> rich pre-check fails, legacy chunking takes over.
oversized = "a" * 40000
assert len(oversized.encode("utf-8")) > TelegramAdapter.RICH_MESSAGE_MAX_BYTES
assert len(oversized) > TelegramAdapter.RICH_MESSAGE_MAX_CHARS
result = await adapter.send("12345", oversized)
@@ -94,6 +152,23 @@ async def test_oversized_content_skips_rich_and_chunks():
assert adapter._bot.send_message.await_count > 1
@pytest.mark.asyncio
async def test_rich_limit_is_characters_not_bytes():
"""Telegram's rich limit is UTF-8 characters, not encoded bytes."""
adapter = _make_adapter()
cjk = "" * 20000 # 20k chars, 60k UTF-8 bytes
assert len(cjk.encode("utf-8")) > TelegramAdapter.RICH_MESSAGE_MAX_BYTES
assert len(cjk) <= TelegramAdapter.RICH_MESSAGE_MAX_CHARS
result = await adapter.send("12345", cjk)
assert result.success is True
bot = adapter._bot
assert bot is not None
bot.do_api_request.assert_awaited_once()
bot.send_message.assert_not_called()
@pytest.mark.asyncio
@pytest.mark.parametrize(
"exc",
@@ -145,6 +220,33 @@ async def test_capability_error_latches_rich_send_off():
adapter._bot.send_message.assert_awaited()
@pytest.mark.asyncio
@pytest.mark.parametrize("exc", [PTB_ENDPOINT_NOT_FOUND, PTB_INVALID_TOKEN_404])
async def test_real_ptb_endpoint_missing_falls_back_and_latches_off(exc):
adapter = _make_adapter()
adapter._bot.do_api_request = AsyncMock(side_effect=exc)
result = await adapter.send("12345", RICH_CONTENT)
assert result.success is True
bot = adapter._bot
assert bot is not None
bot.do_api_request.assert_awaited_once()
bot.send_message.assert_awaited()
assert adapter._rich_send_disabled is True
@pytest.mark.asyncio
async def test_rich_payload_preserves_link_preview_disable():
adapter = _make_adapter(extra={"disable_link_previews": True})
result = await adapter.send("12345", "See https://example.com")
assert result.success is True
api_kwargs = _rich_api_kwargs(adapter)
assert api_kwargs["link_preview_options"] == {"is_disabled": True}
@pytest.mark.asyncio
async def test_per_message_bad_request_does_not_latch_off():
"""A parser/limit BadRequest is per-message — rich must stay enabled
@@ -326,3 +428,40 @@ async def test_rich_draft_oversized_uses_legacy():
assert result.success is True
adapter._bot.do_api_request.assert_not_called()
adapter._bot.send_message_draft.assert_awaited_once()
# ----------------------------------------------------------------------
# prefers_fresh_final_streaming: the stream consumer asks the adapter whether
# to finalize a streamed reply by sending a fresh (rich) message + deleting the
# preview, instead of final-editing the preview through the non-rich edit path.
# Telegram opts in exactly when the content is rich-eligible.
# ----------------------------------------------------------------------
def test_prefers_fresh_final_streaming_when_rich_enabled():
adapter = _make_adapter()
assert adapter.prefers_fresh_final_streaming(RICH_CONTENT) is True
def test_prefers_fresh_final_streaming_ignores_legacy_toggle():
adapter = _make_adapter(extra={"rich_messages": False})
assert adapter.prefers_fresh_final_streaming(RICH_CONTENT) is True
# ----------------------------------------------------------------------
# streaming_overflow_limit: with rich on, the stream consumer may accumulate up
# to the 32,768-char rich cap before splitting, so a reply that fits one
# sendRichMessage / sendRichMessageDraft isn't fragmented at the 4,096 limit.
# ----------------------------------------------------------------------
def test_streaming_overflow_limit_is_rich_cap_when_enabled():
adapter = _make_adapter()
assert adapter.streaming_overflow_limit() == TelegramAdapter.RICH_MESSAGE_MAX_CHARS
def test_streaming_overflow_limit_ignores_legacy_toggle():
adapter = _make_adapter(extra={"rich_messages": False})
assert adapter.streaming_overflow_limit() == TelegramAdapter.RICH_MESSAGE_MAX_CHARS
def test_streaming_overflow_limit_none_when_rich_latched_off():
adapter = _make_adapter()
adapter._rich_send_disabled = True
assert adapter.streaming_overflow_limit() is None
@@ -96,3 +96,40 @@ def test_status_preserves_existing_fields(loopback_client):
}
missing = expected_keys - set(body.keys())
assert not missing, f"/api/status dropped fields: {missing}"
# Host-local detail (absolute paths, PID, internal gateway URL) is deployment
# recon a liveness probe never needs. ``/api/status`` bypasses dashboard auth
# (it is in ``PUBLIC_API_PATHS``), so on a network-exposed bind it must not
# leak that detail to anonymous callers.
_HOST_DETAIL_FIELDS = frozenset({
"hermes_home", "config_path", "env_path", "gateway_pid",
"gateway_health_url",
})
def test_status_withholds_host_detail_in_gated_mode(gated_client):
"""On a gated (non-loopback) bind, the public ``/api/status`` probe must
expose only the liveness + auth-gate shape never absolute host paths,
the gateway PID, or the internal gateway health URL. The endpoint
bypasses dashboard auth, so anyone who can reach the host hits it cold."""
r = gated_client.get("/api/status")
assert r.status_code == 200
body = r.json()
# Liveness / auth-gate shape stays public.
for key in ("version", "gateway_state", "auth_required", "auth_providers"):
assert key in body, f"liveness field {key!r} must stay public"
# Deployment recon must be withheld from the anonymous public probe.
leaked = _HOST_DETAIL_FIELDS & set(body.keys())
assert not leaked, f"/api/status leaked host detail under the gate: {leaked}"
def test_status_includes_host_detail_in_loopback_mode(loopback_client):
"""Counterpart to the gated case: a loopback bind is local-only, so the
full payload (including host paths and PID) is still served preserving
the StatusPage / ``hermes status`` experience for local operators."""
r = loopback_client.get("/api/status")
assert r.status_code == 200
body = r.json()
missing = _HOST_DETAIL_FIELDS - set(body.keys())
assert not missing, f"loopback /api/status should keep host detail: {missing}"
+2 -1
View File
@@ -143,7 +143,8 @@ class TestGeminiModelNormalization:
assert normalize_model_for_provider("gemini-2.5-flash", "gemini") == "gemini-2.5-flash"
def test_strip_vendor_prefix(self):
assert normalize_model_for_provider("google/gemini-2.5-flash", "gemini") == "google/gemini-2.5-flash"
assert normalize_model_for_provider("google/gemini-2.5-flash", "gemini") == "gemini-2.5-flash"
assert normalize_model_for_provider("gemini/gemini-2.5-flash", "gemini") == "gemini-2.5-flash"
def test_gemma_vendor_detection(self):
assert detect_vendor("gemma-4-31b-it") == "google"
+4 -1
View File
@@ -167,10 +167,13 @@ class TestAggregatorProviders:
class TestIssue6211NativeProviderPrefixNormalization:
@pytest.mark.parametrize("model,target_provider,expected", [
("zai/glm-5.1", "zai", "glm-5.1"),
("google/gemini-2.5-pro", "gemini", "google/gemini-2.5-pro"),
("google/gemini-2.5-pro", "gemini", "gemini-2.5-pro"),
("gemini/gemini-2.5-pro", "gemini", "gemini-2.5-pro"),
("moonshot/kimi-k2.5", "kimi-coding", "kimi-k2.5"),
("anthropic/claude-sonnet-4.6", "openrouter", "anthropic/claude-sonnet-4.6"),
("x-ai/grok-4-fast-reasoning", "xai", "grok-4-fast-reasoning"),
("Qwen/Qwen3.5-397B-A17B", "huggingface", "Qwen/Qwen3.5-397B-A17B"),
("openai/gpt-5.4", "xai", "openai/gpt-5.4"),
("modal/zai-org/GLM-5-FP8", "custom", "modal/zai-org/GLM-5-FP8"),
])
def test_native_provider_prefixes_are_only_stripped_on_matching_provider(
+48
View File
@@ -173,6 +173,54 @@ class TestResolveGitUrl:
assert url == "git@github.com:owner/repo.git"
assert subdir == "sub"
@pytest.mark.parametrize(
"identifier",
[
"https://github.com/owner/repo/tree/main",
"https://github.com/owner/repo/blob/main/README.md",
"https://github.com/owner/repo/pull/123",
"https://github.com/owner/repo/commit/abc123def",
"https://github.com/owner/repo/releases/tag/v1.0",
"https://github.com/owner/repo/issues/42",
],
)
def test_github_browser_url_normalized_to_repo(self, identifier):
url, subdir = _resolve_git_url(identifier)
assert url == "https://github.com/owner/repo.git"
assert subdir is None
@pytest.mark.parametrize(
("identifier", "expected_subdir"),
[
("https://github.com/owner/repo/tree/main/plugins/foo", "plugins/foo"),
("https://github.com/owner/repo/tree/feature-branch/plugin", "plugin"),
("https://github.com/owner/repo/tree/main/plugins/foo?plain=1", "plugins/foo"),
("https://github.com/owner/repo.git/tree/main/plugins/foo", "plugins/foo"),
],
)
def test_github_tree_browser_url_preserves_subdir(self, identifier, expected_subdir):
url, subdir = _resolve_git_url(identifier)
assert url == "https://github.com/owner/repo.git"
assert subdir == expected_subdir
@pytest.mark.parametrize(
"identifier",
[
"https://github.com/owner/repo",
"https://github.com/owner/repo.git",
"https://github.com/owner",
"https://github.com/owner/repo/branches",
"https://github.com/owner//tree/main",
"https://gitlab.com/owner/repo/tree/main",
"git@github.com:owner/repo.git",
"file:///tmp/repo/tree/main",
],
)
def test_non_browser_urls_passthrough(self, identifier):
url, subdir = _resolve_git_url(identifier)
assert url == identifier
assert subdir is None
# ── _resolve_subdir_within ──────────────────────────────────────────────────
@@ -7,6 +7,7 @@ Windows-specific code paths can be exercised on any host.
from __future__ import annotations
import json
import os
import sys
import types
@@ -446,6 +447,97 @@ def test_quarantine_actionable_warning_when_everything_fails(
assert "Hermes Desktop" in captured or "gateway" in captured.lower()
# ---------------------------------------------------------------------------
# Windows gateway pause/resume before update mutation
# ---------------------------------------------------------------------------
@patch.object(cli_main, "_is_windows", return_value=True)
def test_pause_windows_gateways_for_update_stops_profile_and_unmapped_pids(
_winp,
monkeypatch,
tmp_path,
capsys,
):
import gateway.status as status_mod
import hermes_cli.gateway as gateway_mod
profile_home = tmp_path / "profiles" / "work"
profile_home.mkdir(parents=True)
profile_proc = SimpleNamespace(profile="work", path=profile_home, pid=101)
monkeypatch.setattr(gateway_mod, "find_gateway_pids", lambda **_k: [101, 202])
monkeypatch.setattr(
gateway_mod,
"find_profile_gateway_processes",
lambda **_k: [profile_proc],
)
monkeypatch.setattr(gateway_mod, "_get_restart_drain_timeout", lambda: 0.1)
waited_for = []
def fake_wait(pids, *, timeout):
waited_for.extend(pids)
return set()
monkeypatch.setattr(cli_main, "_wait_for_windows_update_gateway_exit", fake_wait)
terminated = []
monkeypatch.setattr(
status_mod,
"terminate_pid",
lambda pid, force=False: terminated.append((pid, force)),
)
token = cli_main._pause_windows_gateways_for_update()
assert token == {
"resume_needed": True,
"profiles": {"work": 101},
"unmapped_pids": [202],
}
assert waited_for == [101]
assert terminated == [(202, True)]
marker = json.loads((profile_home / ".gateway-planned-stop.json").read_text())
assert marker["target_pid"] == 101
assert marker["stopper_pid"] == os.getpid()
captured = capsys.readouterr().out
assert "Paused gateway profile(s): work" in captured
assert "without profile mapping" in captured
@patch.object(cli_main, "_is_windows", return_value=True)
def test_resume_windows_gateways_after_update_relaunches_paused_profiles(
_winp,
monkeypatch,
capsys,
):
import hermes_cli.gateway as gateway_mod
relaunched = []
monkeypatch.setattr(
gateway_mod,
"launch_detached_profile_gateway_restart",
lambda profile, old_pid: relaunched.append((profile, old_pid)) or True,
)
token = {
"resume_needed": True,
"profiles": {"default": 101, "work": 202},
"unmapped_pids": [],
}
cli_main._resume_windows_gateways_after_update(token)
assert token["resume_needed"] is False
assert relaunched == [("default", 101), ("work", 202)]
assert (
"Restarting Windows gateway profile(s): default, work"
in capsys.readouterr().out
)
# ---------------------------------------------------------------------------
# cmd_update integration — concurrent-instance gate
# ---------------------------------------------------------------------------
+26 -4
View File
@@ -2633,7 +2633,7 @@ class TestNewEndpoints:
resp = self.client.post(
"/api/profiles",
json={"name": "writer", "clone_from_default": False},
json={"name": "writer", "clone_from": None},
)
assert resp.status_code == 200
@@ -2641,7 +2641,7 @@ class TestNewEndpoints:
assert wrapper_path.exists()
assert wrapper_path.read_text() == '#!/bin/sh\nexec hermes -p writer "$@"\n'
def test_profiles_create_with_clone_from_default_copies_default_skills(self, monkeypatch):
def test_profiles_create_with_clone_from_copies_source_skills(self, monkeypatch):
from hermes_constants import get_hermes_home
import hermes_cli.profiles as profiles_mod
@@ -2652,7 +2652,7 @@ class TestNewEndpoints:
resp = self.client.post(
"/api/profiles",
json={"name": "cloned", "clone_from_default": True},
json={"name": "cloned", "clone_from": "default"},
)
assert resp.status_code == 200
@@ -2685,6 +2685,28 @@ class TestNewEndpoints:
)
assert cloned_skill.exists()
def test_profiles_create_clone_all_from_named_source(self, monkeypatch):
from hermes_constants import get_hermes_home
import hermes_cli.profiles as profiles_mod
monkeypatch.setattr(profiles_mod, "create_wrapper_script", lambda name: None)
assert self.client.post("/api/profiles", json={"name": "full-src"}).status_code == 200
source_dir = get_hermes_home() / "profiles" / "full-src"
(source_dir / "config.yaml").write_text("model:\n provider: source-only\n", encoding="utf-8")
(source_dir / "workspace" / "artifact.txt").parent.mkdir(parents=True, exist_ok=True)
(source_dir / "workspace" / "artifact.txt").write_text("copied", encoding="utf-8")
resp = self.client.post(
"/api/profiles",
json={"name": "full-copy", "clone_from": "full-src", "clone_all": True},
)
assert resp.status_code == 200
target_dir = get_hermes_home() / "profiles" / "full-copy"
assert (target_dir / "config.yaml").read_text(encoding="utf-8") == "model:\n provider: source-only\n"
assert (target_dir / "workspace" / "artifact.txt").read_text(encoding="utf-8") == "copied"
def test_profiles_create_without_clone_seeds_bundled_skills(self, monkeypatch):
from hermes_constants import get_hermes_home
import hermes_cli.profiles as profiles_mod
@@ -2701,7 +2723,7 @@ class TestNewEndpoints:
resp = self.client.post(
"/api/profiles",
json={"name": "fresh", "clone_from_default": False},
json={"name": "fresh", "clone_from": None},
)
assert resp.status_code == 200
+27
View File
@@ -193,6 +193,33 @@ def test_local_mode_defaults_to_home_and_can_jump_to_absolute_path(local_files_c
assert other_listing.json()["entries"][0]["path"] == str(other / "other.txt")
def test_gated_local_mode_still_defaults_to_home(monkeypatch, tmp_path):
home = tmp_path / "home"
home.mkdir()
monkeypatch.delenv("HERMES_DASHBOARD_FILES_ROOT", raising=False)
monkeypatch.delenv("HERMES_MANAGED", raising=False)
monkeypatch.setenv("HOME", str(home))
monkeypatch.setenv("HERMES_HOME", str(home / ".hermes"))
prev_auth_required = getattr(web_server.app.state, "auth_required", None)
prev_bound_host = getattr(web_server.app.state, "bound_host", None)
web_server.app.state.auth_required = True
web_server.app.state.bound_host = "0.0.0.0"
try:
request = SimpleNamespace(
app=web_server.app,
client=SimpleNamespace(host="10.0.0.2"),
url=SimpleNamespace(hostname="example.com"),
)
policy = web_server._managed_files_policy(request, create_root=False)
finally:
_restore_app_state(prev_auth_required, prev_bound_host)
assert policy.default_path == home.resolve()
assert policy.locked_root is None
assert policy.can_change_path is True
def test_local_mode_upload_read_mkdir_delete_roundtrip(local_files_client):
client, home = local_files_client
folder = home / "workspace"
@@ -154,6 +154,22 @@ def _codex_ack_message_response(text: str):
)
def _codex_final_answer_with_top_level_incomplete_response(text: str):
return SimpleNamespace(
output=[
SimpleNamespace(
type="message",
phase="final_answer",
status="completed",
content=[SimpleNamespace(type="output_text", text=text)],
)
],
usage=SimpleNamespace(input_tokens=4, output_tokens=2, total_tokens=6),
status="incomplete",
model="gpt-5.4",
)
class _FakeCreateStream:
"""Iterable-only fake for ``responses.create(stream=True)`` outputs.
@@ -1351,6 +1367,92 @@ def test_normalize_codex_response_marks_commentary_only_message_as_incomplete(mo
assert "inspect the repository" in (assistant_message.content or "")
def test_normalize_codex_response_final_answer_overrides_top_level_incomplete(monkeypatch):
from agent.codex_responses_adapter import _normalize_codex_response
assistant_message, finish_reason = _normalize_codex_response(
_codex_final_answer_with_top_level_incomplete_response(
"Briefly:\n\n- I'm Ramsay, your assistant."
)
)
assert finish_reason == "stop"
assert "Ramsay" in (assistant_message.content or "")
def test_normalize_codex_response_top_level_incomplete_without_final_answer_stays_incomplete(monkeypatch):
from agent.codex_responses_adapter import _normalize_codex_response
response = SimpleNamespace(
output=[
SimpleNamespace(
type="message",
status="completed",
content=[SimpleNamespace(type="output_text", text="Partial...")],
)
],
usage=SimpleNamespace(input_tokens=4, output_tokens=2, total_tokens=6),
status="incomplete",
model="gpt-5.4",
)
_, finish_reason = _normalize_codex_response(response)
assert finish_reason == "incomplete"
@pytest.mark.parametrize("top_level_status", ["queued", "in_progress"])
def test_normalize_codex_response_final_answer_does_not_override_streaming_status(
monkeypatch, top_level_status
):
from agent.codex_responses_adapter import _normalize_codex_response
response = SimpleNamespace(
output=[
SimpleNamespace(
type="message",
phase="final_answer",
status="completed",
content=[SimpleNamespace(type="output_text", text="Interim answer.")],
)
],
usage=SimpleNamespace(input_tokens=4, output_tokens=2, total_tokens=6),
status=top_level_status,
model="gpt-5.4",
)
_, finish_reason = _normalize_codex_response(response)
assert finish_reason == "incomplete"
def test_normalize_codex_response_final_answer_does_not_override_per_item_in_progress(monkeypatch):
from agent.codex_responses_adapter import _normalize_codex_response
response = SimpleNamespace(
output=[
SimpleNamespace(
type="message",
phase="final_answer",
status="completed",
content=[SimpleNamespace(type="output_text", text="Partial final.")],
),
SimpleNamespace(
type="message",
status="in_progress",
content=[SimpleNamespace(type="output_text", text="")],
),
],
usage=SimpleNamespace(input_tokens=4, output_tokens=2, total_tokens=6),
status="completed",
model="gpt-5.4",
)
_, finish_reason = _normalize_codex_response(response)
assert finish_reason == "incomplete"
def test_normalize_codex_response_preserves_message_status_for_replay(monkeypatch):
"""Incomplete Codex output messages must not be replayed as completed."""
agent = _build_agent(monkeypatch)
@@ -1418,6 +1520,42 @@ def test_normalize_codex_response_detects_leaked_tool_call_text(monkeypatch):
assert assistant_message.tool_calls == []
def test_scan_for_leaked_tool_call_checks_prefix_window_only(monkeypatch):
from agent.codex_responses_adapter import (
_TOOL_CALL_LEAK_SCAN_LIMIT,
_scan_for_leaked_tool_call,
)
marker = "to=functions.terminal {\"command\": \"pwd\"}"
assert _scan_for_leaked_tool_call(marker) is True
assert _scan_for_leaked_tool_call("x" * (_TOOL_CALL_LEAK_SCAN_LIMIT - 10) + " " + marker) is True
assert _scan_for_leaked_tool_call("x" * (_TOOL_CALL_LEAK_SCAN_LIMIT + 10) + marker) is False
def test_normalize_codex_response_ignores_late_tool_call_marker_past_scan_window(monkeypatch):
from agent.codex_responses_adapter import _TOOL_CALL_LEAK_SCAN_LIMIT, _normalize_codex_response
late_marker = "x" * (_TOOL_CALL_LEAK_SCAN_LIMIT + 100) + " to=functions.terminal {\"command\": \"pwd\"}"
response = SimpleNamespace(
output=[
SimpleNamespace(
type="message",
status="completed",
content=[SimpleNamespace(type="output_text", text=late_marker)],
)
],
usage=SimpleNamespace(input_tokens=4, output_tokens=2, total_tokens=6),
status="completed",
model="gpt-5.4",
)
assistant_message, finish_reason = _normalize_codex_response(response)
assert finish_reason == "stop"
assert assistant_message.content == late_marker
def test_normalize_codex_response_ignores_tool_call_text_when_real_tool_call_present(monkeypatch):
"""If the model emitted BOTH a structured function_call AND some text that
happens to contain `to=functions.*` (unlikely but possible), trust the
@@ -104,6 +104,38 @@ class TestIsThinkingOnlyAssistant:
}
assert AIAgent._is_thinking_only_assistant(msg)
def test_codex_reasoning_items_list_form_detected(self):
msg = {
"role": "assistant",
"content": "",
"codex_reasoning_items": [
{"type": "reasoning", "id": "rs_123", "encrypted_content": "enc_blob"}
],
}
assert AIAgent._is_thinking_only_assistant(msg)
def test_codex_reasoning_items_with_visible_text_is_not_thinking_only(self):
msg = {
"role": "assistant",
"content": "Visible answer",
"codex_reasoning_items": [
{"type": "reasoning", "id": "rs_123", "encrypted_content": "enc_blob"}
],
}
assert not AIAgent._is_thinking_only_assistant(msg)
def test_empty_codex_reasoning_items_list_is_not_thinking_only(self):
msg = {"role": "assistant", "content": "", "codex_reasoning_items": []}
assert not AIAgent._is_thinking_only_assistant(msg)
def test_non_reasoning_codex_items_are_not_thinking_only(self):
msg = {
"role": "assistant",
"content": "",
"codex_reasoning_items": [None, "x", {"type": "other"}],
}
assert not AIAgent._is_thinking_only_assistant(msg)
def test_user_message_never_thinking_only(self):
assert not AIAgent._is_thinking_only_assistant({"role": "user", "content": ""})
+111
View File
@@ -12,6 +12,7 @@ the codebase were migrated to the helper; these tests pin that invariant.
"""
from __future__ import annotations
import errno
import json
import os
import sys
@@ -158,3 +159,113 @@ def test_atomic_replace_broken_symlink_creates_target(tmp_path: Path) -> None:
assert link.is_symlink(), "symlink must be preserved"
assert missing.exists(), "real target should now exist"
assert missing.read_text(encoding="utf-8") == "created-through-link\n"
# ─── EXDEV / EBUSY copy fallback ───────────────────────────────────────────
@pytest.mark.parametrize("fail_errno", [errno.EXDEV, errno.EBUSY])
def test_atomic_replace_copy_fallback(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, fail_errno: int
) -> None:
target = tmp_path / "config.yaml"
target.write_text("old\n", encoding="utf-8")
tmp = _write_tmp(tmp_path, "new\n")
def fail_replace(src: str, dst: str) -> None:
raise OSError(fail_errno, os.strerror(fail_errno), src, None, dst)
monkeypatch.setattr("utils.os.replace", fail_replace)
assert Path(atomic_replace(tmp, target)) == target
assert target.read_text(encoding="utf-8") == "new\n"
assert not tmp.exists()
def test_atomic_replace_copy_fallback_preserves_symlink(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
real = tmp_path / "real.yaml"
link = tmp_path / "link.yaml"
real.write_text("old\n", encoding="utf-8")
link.symlink_to(real)
tmp = _write_tmp(tmp_path, "new\n")
def fail_replace(src: str, dst: str) -> None:
raise OSError(errno.EXDEV, os.strerror(errno.EXDEV), src, None, dst)
monkeypatch.setattr("utils.os.replace", fail_replace)
assert Path(atomic_replace(tmp, link)) == real
assert link.is_symlink()
assert real.read_text(encoding="utf-8") == "new\n"
assert not tmp.exists()
def test_atomic_replace_copy_fallback_preserves_metadata(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
if os.name != "posix":
pytest.skip("POSIX-only")
target = tmp_path / "config.yaml"
target.write_text("old\n", encoding="utf-8")
os.chmod(target, 0o600)
tmp = _write_tmp(tmp_path, "new\n")
os.chmod(tmp, 0o644)
def fail_replace(src: str, dst: str) -> None:
raise OSError(errno.EBUSY, os.strerror(errno.EBUSY), src, None, dst)
monkeypatch.setattr("utils.os.replace", fail_replace)
atomic_replace(tmp, target)
assert target.read_text(encoding="utf-8") == "new\n"
assert target.stat().st_mode & 0o777 == 0o644
def test_atomic_replace_other_oserror_propagates(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
target = tmp_path / "config.yaml"
target.write_text("old\n", encoding="utf-8")
tmp = _write_tmp(tmp_path, "new\n")
def fail_replace(src: str, dst: str) -> None:
raise OSError(errno.EACCES, os.strerror(errno.EACCES), src, None, dst)
monkeypatch.setattr("utils.os.replace", fail_replace)
with pytest.raises(OSError) as excinfo:
atomic_replace(tmp, target)
assert excinfo.value.errno == errno.EACCES
assert target.read_text(encoding="utf-8") == "old\n"
assert tmp.exists()
def test_atomic_replace_real_cross_device(tmp_path: Path) -> None:
shm = Path("/dev/shm")
if os.name != "posix" or not os.access(shm, os.W_OK):
pytest.skip("requires writable /dev/shm")
import shutil as _shutil
import uuid as _uuid
other_fs_dir = shm / f"hermes-exdev-test-{_uuid.uuid4().hex[:8]}"
other_fs_dir.mkdir()
try:
real = other_fs_dir / "config.yaml"
real.write_text("old\n", encoding="utf-8")
if os.stat(real).st_dev == os.stat(tmp_path).st_dev:
pytest.skip("/dev/shm is not a separate filesystem here")
link = tmp_path / "config.yaml"
link.symlink_to(real)
tmp = _write_tmp(tmp_path, "new\n")
assert Path(atomic_replace(tmp, link)) == real
assert link.is_symlink()
assert real.read_text(encoding="utf-8") == "new\n"
assert not tmp.exists()
finally:
_shutil.rmtree(other_fs_dir, ignore_errors=True)
+105
View File
@@ -369,6 +369,12 @@ class TestTeePattern:
assert dangerous is True
assert key is not None
def test_tee_absolute_home_bashrc(self):
bashrc = Path.home() / ".bashrc"
dangerous, key, desc = detect_dangerous_command(f"echo x | tee {bashrc}")
assert dangerous is True
assert key is not None
def test_tee_custom_hermes_home_env(self):
dangerous, key, desc = detect_dangerous_command("echo x | tee $HERMES_HOME/.env")
assert dangerous is True
@@ -560,11 +566,37 @@ class TestSensitiveRedirectPattern:
assert dangerous is True
assert key is not None
def test_append_to_absolute_home_ssh_authorized_keys(self):
authorized_keys = Path.home() / ".ssh" / "authorized_keys"
dangerous, key, desc = detect_dangerous_command(f"cat key >> {authorized_keys}")
assert dangerous is True
assert key is not None
def test_append_to_tilde_ssh_authorized_keys(self):
dangerous, key, desc = detect_dangerous_command("cat key >> ~/.ssh/authorized_keys")
assert dangerous is True
assert key is not None
def test_redirect_to_absolute_home_bashrc(self):
bashrc = Path.home() / ".bashrc"
dangerous, key, desc = detect_dangerous_command(f"echo 'alias ll=\"ls -la\"' > {bashrc}")
assert dangerous is True
assert key is not None
def test_redirect_to_home_set_after_import(self, monkeypatch, tmp_path):
late_home = tmp_path / "late-home"
late_home.mkdir()
monkeypatch.setenv("HOME", str(late_home))
dangerous, key, desc = detect_dangerous_command(f"echo x > {late_home}/.bashrc")
assert dangerous is True
assert key is not None
def test_redirect_to_other_absolute_home_bashrc_is_not_current_user_sensitive(self):
dangerous, key, desc = detect_dangerous_command("echo x > /tmp/not-current-home/.bashrc")
assert dangerous is False
assert key is None
def test_redirect_to_safe_tmp_file(self):
dangerous, key, desc = detect_dangerous_command("echo hello > /tmp/output.txt")
assert dangerous is False
@@ -633,6 +665,79 @@ class TestProjectSensitiveCopyPattern:
assert desc is None
class TestSensitiveCopyMovePattern:
"""cp/mv/install OVERWRITING ~/.ssh/*, credential files (~/.netrc etc.),
shell rc files, or ~/.hermes/config.yaml/.env must require approval the
tee/redirection forms were already gated (#14639 family / commit 4e9d886d),
but cp/mv/install on these targets was an unpaired half-door (key implant /
shell-rc command injection slipped through auto-approve)."""
def test_cp_to_ssh_authorized_keys(self):
dangerous, key, desc = detect_dangerous_command("cp /tmp/evil ~/.ssh/authorized_keys")
assert dangerous is True
assert key is not None
def test_mv_to_ssh_private_key(self):
dangerous, key, desc = detect_dangerous_command("mv /tmp/k ~/.ssh/id_rsa")
assert dangerous is True
def test_install_to_netrc(self):
dangerous, key, desc = detect_dangerous_command("install -m600 /tmp/c ~/.netrc")
assert dangerous is True
def test_cp_to_bashrc(self):
dangerous, key, desc = detect_dangerous_command("cp /tmp/e ~/.bashrc")
assert dangerous is True
def test_cp_to_hermes_config(self):
dangerous, key, desc = detect_dangerous_command("cp /tmp/evil.yaml ~/.hermes/config.yaml")
assert dangerous is True
def test_cp_from_ssh_is_safe(self):
dangerous, key, desc = detect_dangerous_command("cp ~/.ssh/config /tmp/x")
assert dangerous is False
def test_cp_unrelated_files_safe(self):
dangerous, key, desc = detect_dangerous_command("cp a.txt b.txt")
assert dangerous is False
class TestSensitiveInPlaceEditPattern:
"""Detect in-place edits to user startup and credential files."""
def test_sed_in_place_bashrc(self):
dangerous, key, desc = detect_dangerous_command("sed -i 's/a/b/' ~/.bashrc")
assert dangerous is True
assert key is not None
def test_sed_long_in_place_ssh_authorized_keys(self):
dangerous, key, desc = detect_dangerous_command(
"sed --in-place 's/key/newkey/' ~/.ssh/authorized_keys"
)
assert dangerous is True
assert key is not None
def test_perl_in_place_netrc(self):
dangerous, key, desc = detect_dangerous_command(
"perl -i -pe 's/pass/pass2/' ~/.netrc"
)
assert dangerous is True
assert key is not None
def test_ruby_in_place_absolute_home_zshrc(self):
zshrc = Path.home() / ".zshrc"
dangerous, key, desc = detect_dangerous_command(
f"ruby -i -pe 'gsub(/a/, \"b\")' {zshrc}"
)
assert dangerous is True
assert key is not None
def test_sed_in_place_regular_file_safe(self):
dangerous, key, desc = detect_dangerous_command("sed -i 's/a/b/' notes.txt")
assert dangerous is False
assert key is None
class TestProjectSensitiveTeePattern:
def test_tee_to_local_dotenv_requires_approval(self):
dangerous, key, desc = detect_dangerous_command("printenv | tee .env.local")
+295
View File
@@ -0,0 +1,295 @@
#!/usr/bin/env python3
"""
Tests for structured-document extraction in the read_file tool.
Covers .ipynb / .docx / .xlsx extraction (ported from Kilo-Org/kilocode
#10733, #10737, #10740) and the read_file_tool integration: pagination,
line-numbering, graceful fallback on malformed input, and hidden-sheet
omission.
Run with: python -m pytest tests/tools/test_read_extract.py -v
"""
import json
import os
import tempfile
import unittest
import zipfile
from tools.read_extract import (
ExtractionError,
extract_document_text,
is_extractable_document,
)
from tools.file_tools import read_file_tool
# ---------------------------------------------------------------------------
# Fixture builders — construct minimal valid OOXML / notebook files.
# ---------------------------------------------------------------------------
def _write_notebook(path, cells, nbformat=4):
nb = {"cells": cells, "metadata": {}, "nbformat": nbformat, "nbformat_minor": 5}
with open(path, "w", encoding="utf-8") as fh:
json.dump(nb, fh)
def _write_docx(path, document_xml):
with zipfile.ZipFile(path, "w") as z:
z.writestr("[Content_Types].xml", "<Types/>")
z.writestr("word/document.xml", document_xml)
def _write_xlsx(path, *, workbook, rels, shared, sheets):
"""sheets: dict of part-name -> xml string."""
with zipfile.ZipFile(path, "w") as z:
z.writestr("xl/workbook.xml", workbook)
z.writestr("xl/_rels/workbook.xml.rels", rels)
if shared is not None:
z.writestr("xl/sharedStrings.xml", shared)
for part, xml in sheets.items():
z.writestr(part, xml)
_NS_W = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
_NS_S = "http://schemas.openxmlformats.org/spreadsheetml/2006/main"
# ---------------------------------------------------------------------------
# is_extractable_document
# ---------------------------------------------------------------------------
class TestIsExtractable(unittest.TestCase):
def test_recognized_extensions(self):
self.assertTrue(is_extractable_document("a.ipynb"))
self.assertTrue(is_extractable_document("/x/B.DOCX"))
self.assertTrue(is_extractable_document("report.xlsx"))
def test_unrecognized_extensions(self):
self.assertFalse(is_extractable_document("a.py"))
self.assertFalse(is_extractable_document("a.pdf"))
self.assertFalse(is_extractable_document("a.txt"))
# ---------------------------------------------------------------------------
# Notebooks (.ipynb) — #10733
# ---------------------------------------------------------------------------
class TestNotebookExtraction(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.mkdtemp(prefix="rex_nb_")
def tearDown(self):
import shutil
shutil.rmtree(self.tmp, ignore_errors=True)
def test_markdown_and_code_in_order(self):
p = os.path.join(self.tmp, "nb.ipynb")
_write_notebook(p, [
{"cell_type": "markdown", "source": ["# Title\n", "para"]},
{"cell_type": "code", "source": "x = 1\nprint(x)",
"outputs": [{"output_type": "stream", "text": ["1\n"]}],
"execution_count": 1},
])
text = extract_document_text(p)
self.assertIn("# Title", text)
self.assertIn("print(x)", text)
# Output payloads must NOT leak into the extracted text.
self.assertNotIn("output_type", text)
self.assertNotIn("execution_count", text)
# Order preserved: markdown before code.
self.assertLess(text.index("Title"), text.index("print(x)"))
def test_string_source_form(self):
p = os.path.join(self.tmp, "nb2.ipynb")
_write_notebook(p, [{"cell_type": "code", "source": "single string source"}])
self.assertIn("single string source", extract_document_text(p))
def test_legacy_worksheets_form(self):
p = os.path.join(self.tmp, "nb3.ipynb")
nb = {"worksheets": [{"cells": [
{"cell_type": "code", "input": "ignored", "source": "legacy cell"}]}],
"nbformat": 3}
with open(p, "w") as fh:
json.dump(nb, fh)
self.assertIn("legacy cell", extract_document_text(p))
def test_malformed_notebook_raises(self):
p = os.path.join(self.tmp, "bad.ipynb")
with open(p, "w") as fh:
fh.write("{ not valid json")
with self.assertRaises(ExtractionError):
extract_document_text(p)
def test_empty_cells_raises(self):
p = os.path.join(self.tmp, "empty.ipynb")
_write_notebook(p, [])
with self.assertRaises(ExtractionError):
extract_document_text(p)
# ---------------------------------------------------------------------------
# Word documents (.docx) — #10737
# ---------------------------------------------------------------------------
class TestDocxExtraction(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.mkdtemp(prefix="rex_docx_")
def tearDown(self):
import shutil
shutil.rmtree(self.tmp, ignore_errors=True)
def _doc(self, body):
return (f'<?xml version="1.0"?><w:document xmlns:w="{_NS_W}">'
f'<w:body>{body}</w:body></w:document>')
def test_paragraphs_and_runs(self):
p = os.path.join(self.tmp, "d.docx")
_write_docx(p, self._doc(
'<w:p><w:r><w:t>Hello </w:t></w:r><w:r><w:t>World</w:t></w:r></w:p>'
'<w:p><w:r><w:t>Second</w:t></w:r></w:p>'))
text = extract_document_text(p)
self.assertIn("Hello World", text)
self.assertIn("Second", text)
def test_tabs_and_breaks(self):
p = os.path.join(self.tmp, "d2.docx")
_write_docx(p, self._doc(
'<w:p><w:r><w:t>A</w:t><w:tab/><w:t>B</w:t><w:br/><w:t>C</w:t></w:r></w:p>'))
text = extract_document_text(p)
self.assertIn("A\tB", text)
self.assertIn("C", text)
def test_not_a_zip_raises(self):
p = os.path.join(self.tmp, "bad.docx")
with open(p, "wb") as fh:
fh.write(b"plain bytes, not a zip")
with self.assertRaises(ExtractionError):
extract_document_text(p)
def test_missing_document_xml_raises(self):
p = os.path.join(self.tmp, "nodoc.docx")
with zipfile.ZipFile(p, "w") as z:
z.writestr("other.xml", "<x/>")
with self.assertRaises(ExtractionError):
extract_document_text(p)
# ---------------------------------------------------------------------------
# Excel workbooks (.xlsx) — #10740
# ---------------------------------------------------------------------------
class TestXlsxExtraction(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.mkdtemp(prefix="rex_xlsx_")
def tearDown(self):
import shutil
shutil.rmtree(self.tmp, ignore_errors=True)
def _build(self, path, *, include_hidden=True):
r = "http://schemas.openxmlformats.org/officeDocument/2006/relationships"
hidden_sheet = (f'<sheet name="Hidden" sheetId="2" state="hidden" '
f'xmlns:r="{r}" r:id="rId2"/>') if include_hidden else ""
workbook = (
f'<workbook xmlns="{_NS_S}" xmlns:r="{r}"><sheets>'
f'<sheet name="Data" sheetId="1" r:id="rId1"/>{hidden_sheet}'
f'</sheets></workbook>')
rels = (
'<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">'
'<Relationship Id="rId1" Target="worksheets/sheet1.xml" Type="x"/>'
'<Relationship Id="rId2" Target="worksheets/sheet2.xml" Type="x"/>'
'</Relationships>')
shared = (f'<sst xmlns="{_NS_S}"><si><t>Name</t></si><si><t>Score</t></si>'
f'<si><t>Alice</t></si></sst>')
sheet1 = (
f'<worksheet xmlns="{_NS_S}"><sheetData>'
'<row r="1"><c r="A1" t="s"><v>0</v></c><c r="B1" t="s"><v>1</v></c></row>'
'<row r="2"><c r="A2" t="s"><v>2</v></c><c r="B2"><v>95</v></c></row>'
'</sheetData></worksheet>')
sheet2 = (f'<worksheet xmlns="{_NS_S}"><sheetData>'
'<row r="1"><c r="A1" t="str"><v>SECRETDATA</v></c></row>'
'</sheetData></worksheet>')
_write_xlsx(path, workbook=workbook, rels=rels, shared=shared,
sheets={"xl/worksheets/sheet1.xml": sheet1,
"xl/worksheets/sheet2.xml": sheet2})
def test_visible_sheet_content(self):
p = os.path.join(self.tmp, "wb.xlsx")
self._build(p)
text = extract_document_text(p)
self.assertIn("Data", text) # sheet label
self.assertIn("Name\tScore", text) # shared-string header row
self.assertIn("Alice\t95", text) # string + numeric cells
def test_hidden_sheet_omitted(self):
p = os.path.join(self.tmp, "wb2.xlsx")
self._build(p)
text = extract_document_text(p)
self.assertNotIn("SECRETDATA", text)
self.assertNotIn("Hidden", text)
def test_not_a_zip_raises(self):
p = os.path.join(self.tmp, "bad.xlsx")
with open(p, "wb") as fh:
fh.write(b"nope")
with self.assertRaises(ExtractionError):
extract_document_text(p)
# ---------------------------------------------------------------------------
# read_file_tool integration
# ---------------------------------------------------------------------------
class TestReadFileToolIntegration(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.mkdtemp(prefix="rex_int_")
def tearDown(self):
import shutil
shutil.rmtree(self.tmp, ignore_errors=True)
def test_notebook_read_is_line_numbered(self):
p = os.path.join(self.tmp, "nb.ipynb")
_write_notebook(p, [
{"cell_type": "markdown", "source": "# H"},
{"cell_type": "code", "source": "print(1)"},
])
res = json.loads(read_file_tool(p))
self.assertTrue(res.get("extracted_document"))
self.assertIn("1|", res["content"]) # line-number gutter
self.assertIn("print(1)", res["content"])
def test_pagination(self):
p = os.path.join(self.tmp, "nb.ipynb")
_write_notebook(p, [
{"cell_type": "code", "source": "a\nb\nc\nd\ne\nf"},
])
res = json.loads(read_file_tool(p, offset=1, limit=2))
self.assertTrue(res.get("truncated"))
self.assertIn("offset=3", res.get("hint", ""))
# Only first 2 lines present.
self.assertIn("1|# ── Code cell 1 ──", res["content"])
def test_corrupt_docx_falls_through_to_binary_guard(self):
p = os.path.join(self.tmp, "bad.docx")
with open(p, "wb") as fh:
fh.write(b"not a zip")
res = json.loads(read_file_tool(p))
# Should NOT crash; falls through to the binary-extension guard.
self.assertIn("error", res)
self.assertIn("binary", res["error"].lower())
def test_docx_read_extracts(self):
p = os.path.join(self.tmp, "d.docx")
_write_docx(p, (f'<?xml version="1.0"?><w:document xmlns:w="{_NS_W}">'
'<w:body><w:p><w:r><w:t>Report body</w:t></w:r></w:p>'
'</w:body></w:document>'))
res = json.loads(read_file_tool(p))
self.assertTrue(res.get("extracted_document"))
self.assertIn("Report body", res["content"])
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,133 @@
from unittest.mock import MagicMock
import pytest
from tools.file_operations import ExecuteResult, ShellFileOperations, _search_stdout_and_limit
TIMEOUT = "[Command timed out after 60s]"
@pytest.fixture()
def ops():
env = MagicMock(cwd="/tmp/test")
env.execute.return_value = {"output": "", "returncode": 0}
return ShellFileOperations(env)
def timeout_output(*lines: str) -> str:
return "\n".join([*lines, TIMEOUT])
def path_exists_or(output: str, returncode: int = 124):
def execute(command, **kwargs):
if "test -e" in command:
return {"output": "exists", "returncode": 0}
return {"output": output, "returncode": returncode}
return execute
def assert_timed_out(result):
assert result.error is None
assert result.truncated is True
assert result.limit_reason == "search_timeout"
assert result.to_dict()["limit_reason"] == "search_timeout"
def test_timeout_helper_strips_only_trailing_marker():
assert _search_stdout_and_limit(ExecuteResult(timeout_output("a.py"), 124)) == ("a.py", "search_timeout")
assert _search_stdout_and_limit(ExecuteResult("a.py\nnot a marker", 0)) == ("a.py\nnot a marker", None)
@pytest.mark.parametrize(
("target", "output_mode", "raw", "expected"),
[
("files", "content", timeout_output("src/a.py", "src/b.py"), ["src/a.py", "src/b.py"]),
("content", "files_only", timeout_output("src/a.py", "src/b.py"), ["src/a.py", "src/b.py"]),
("content", "content", timeout_output("src/a.py:10:foo", "src/b.py:20:foo"), ["src/a.py", "src/b.py"]),
],
)
def test_rg_timeout_returns_partial_results_without_marker(ops, monkeypatch, target, output_mode, raw, expected):
ops.env.execute.side_effect = path_exists_or(raw)
monkeypatch.setattr(ops, "_has_command", lambda cmd: cmd == "rg")
result = ops.search("foo", path="/big", target=target, output_mode=output_mode)
assert_timed_out(result)
if target == "content" and output_mode == "content":
assert [match.path for match in result.matches] == expected
assert all("timed out" not in match.content for match in result.matches)
else:
assert result.files == expected
assert all("timed out" not in path for path in result.files)
def test_rg_count_timeout_returns_partial_counts(ops, monkeypatch):
ops.env.execute.side_effect = path_exists_or(timeout_output("src/a.py:3", "src/b.py:5"))
monkeypatch.setattr(ops, "_has_command", lambda cmd: cmd == "rg")
result = ops.search("foo", path="/big", target="content", output_mode="count")
assert_timed_out(result)
assert result.counts == {"src/a.py": 3, "src/b.py": 5}
def test_rg_file_timeout_does_not_retry_unsorted(ops, monkeypatch):
calls = 0
def execute(command, **kwargs):
nonlocal calls
if "test -e" in command:
return {"output": "exists", "returncode": 0}
calls += 1
return {"output": timeout_output(), "returncode": 124}
ops.env.execute.side_effect = execute
monkeypatch.setattr(ops, "_has_command", lambda cmd: cmd == "rg")
result = ops.search("*.py", path="/big", target="files")
assert calls == 1
assert_timed_out(result)
assert result.files == []
def test_grep_timeout_returns_partial_match(ops, monkeypatch):
ops.env.execute.side_effect = path_exists_or(timeout_output("src/a.py:10:foo"))
monkeypatch.setattr(ops, "_has_command", lambda cmd: cmd == "grep")
result = ops.search("foo", path="/big", target="content")
assert_timed_out(result)
assert [match.path for match in result.matches] == ["src/a.py"]
def test_find_timeout_returns_partial_files_and_does_not_retry(ops, monkeypatch):
calls = 0
def execute(command, **kwargs):
nonlocal calls
if "test -e" in command:
return {"output": "exists", "returncode": 0}
calls += 1
return {"output": timeout_output("1700000000.0 /big/a.py"), "returncode": 124}
ops.env.execute.side_effect = execute
monkeypatch.setattr(ops, "_has_command", lambda cmd: cmd == "find")
result = ops.search("*.py", path="/big", target="files")
assert calls == 1
assert_timed_out(result)
assert result.files == ["/big/a.py"]
def test_real_rg_error_still_hard_fails(ops, monkeypatch):
ops.env.execute.side_effect = path_exists_or("rg: regex parse error:", returncode=2)
monkeypatch.setattr(ops, "_has_command", lambda cmd: cmd == "rg")
result = ops.search("[", path="/big", target="content")
assert result.error == "Search failed: rg: regex parse error:"
assert result.limit_reason is None
+61
View File
@@ -208,6 +208,11 @@ _SENSITIVE_WRITE_TARGET = (
rf'{_SHELL_RC_FILES}|'
rf'{_CREDENTIAL_FILES})'
)
_USER_SENSITIVE_WRITE_TARGET = (
rf'(?:{_SSH_SENSITIVE_PATH}|'
rf'{_SHELL_RC_FILES}|'
rf'{_CREDENTIAL_FILES})'
)
_PROJECT_SENSITIVE_WRITE_TARGET = rf'(?:{_PROJECT_ENV_PATH}|{_PROJECT_CONFIG_PATH})'
_COMMAND_TAIL = r'(?:\s*(?:&&|\|\||;).*)?$'
@@ -441,6 +446,27 @@ DANGEROUS_PATTERNS = [
# /private/etc/ mirror).
(rf'\b(cp|mv|install)\b.*\s{_SYSTEM_CONFIG_PATH}', "copy/move file into system config path"),
(rf'\b(cp|mv|install)\b.*\s["\']?{_PROJECT_SENSITIVE_WRITE_TARGET}["\']?{_COMMAND_TAIL}', "overwrite project env/config file"),
# cp/mv/install OVERWRITING a sensitive credential/SSH/shell-rc/Hermes file.
# The tee/redirection patterns above already gate _SENSITIVE_WRITE_TARGET
# (~/.ssh/*, ~/.netrc/.pgpass/.npmrc/.pypirc, shell rc files,
# ~/.hermes/config.yaml/.env), but cp/mv/install was only paired for /etc and
# project-relative env/config — so `cp evil ~/.ssh/authorized_keys` (key
# implant), `cp creds ~/.netrc`, and `cp evil ~/.bashrc` (login-time command
# injection) slipped through with auto-approve. Same unpaired-door rationale
# as #14639 / the sed-tee-redirect pairing on these targets.
# Anchor the sensitive target to the command tail so this fires on the
# DESTINATION (last arg) only — `cp evil ~/.ssh/authorized_keys` is gated,
# but reading OUT of a sensitive path (`cp ~/.ssh/config /tmp/x`) stays safe.
# The trailing `[^\s"\']*` consumes the rest of the destination filename
# (e.g. `authorized_keys` after the `~/.ssh/` fragment).
(rf'\b(cp|mv|install)\b.*\s["\']?{_SENSITIVE_WRITE_TARGET}[^\s"\']*["\']?{_COMMAND_TAIL}', "copy/move file into sensitive credential/SSH/shell-rc path"),
# In-place edits mutate the target file directly, bypassing redirection,
# tee, and copy/move/install coverage. Gate the same user-controlled
# startup/credential files so `sed -i ... ~/.bashrc` and `perl -i ...
# ~/.ssh/authorized_keys` cannot silently plant login commands or keys.
(rf'\bsed\s+-[^\s]*i.*(?:{_USER_SENSITIVE_WRITE_TARGET})[^\s"\']*', "in-place edit of sensitive credential/SSH/shell-rc path"),
(rf'\bsed\s+--in-place\b.*(?:{_USER_SENSITIVE_WRITE_TARGET})[^\s"\']*', "in-place edit of sensitive credential/SSH/shell-rc path (long flag)"),
(rf'\b(?:perl|ruby)\b.*(?:^|\s)-[^\s]*i\b.*(?:{_USER_SENSITIVE_WRITE_TARGET})[^\s"\']*', "in-place edit of sensitive credential/SSH/shell-rc path (perl/ruby)"),
(rf'\bsed\s+-[^\s]*i.*\s{_SYSTEM_CONFIG_PATH}', "in-place edit of system config"),
(rf'\bsed\s+--in-place\b.*\s{_SYSTEM_CONFIG_PATH}', "in-place edit of system config (long flag)"),
# In-place edit of a Hermes-managed security file (~/.hermes/config.yaml or
@@ -547,6 +573,11 @@ def _normalize_command_for_detection(command: str) -> str:
command = re.sub(r'\\([^\n])', r'\1', command)
# Strip empty-string literals that split tokens: r''m → rm, r"\"m → rm.
command = re.sub(r"''|\"\"", '', command)
# Fold the current user's resolved absolute home path into ~/ at detection
# time so static user-sensitive patterns catch /home/alice/.bashrc the same
# way they catch ~/.bashrc. Do not snapshot this at import time: tests and
# profile/session launchers can set HOME after this module is imported.
command = _rewrite_resolved_user_home(command)
# Fold the resolved absolute active-profile home path into the canonical
# ~/.hermes/ form so the Hermes config/env patterns catch it. In Docker and
# gateway deployments the agent often references the resolved absolute path
@@ -558,6 +589,36 @@ def _normalize_command_for_detection(command: str) -> str:
return command
def _rewrite_resolved_user_home(command: str) -> str:
"""Rewrite the current user's absolute home prefix to ``~/``.
Resolves HOME at detection time, including its symlink-resolved form, so
terminal commands targeting absolute home paths are checked by the same
static patterns as tilde and $HOME forms. No-op when HOME is unset or
degenerate.
"""
try:
home = os.path.expanduser("~")
candidates = [
home.rstrip("/"),
os.path.realpath(home).rstrip("/"),
]
except Exception:
return command
seen: set[str] = set()
for path in candidates:
if not path or path in seen:
continue
seen.add(path)
# Require an absolute path below root so a bad HOME cannot rewrite the
# whole filesystem namespace.
normalized = path.rstrip("/")
if not normalized.startswith("/") or normalized.count("/") < 2:
continue
command = command.replace(normalized + "/", "~/")
return command
def _rewrite_resolved_hermes_home(command: str) -> str:
"""Rewrite the resolved absolute Hermes home prefix to ``~/.hermes/``.
+60 -16
View File
@@ -241,10 +241,11 @@ class SearchResult:
counts: Dict[str, int] = field(default_factory=dict)
total_count: int = 0
truncated: bool = False
limit_reason: Optional[str] = None
error: Optional[str] = None
def to_dict(self) -> dict:
result = {"total_count": self.total_count}
result: dict[str, object] = {"total_count": self.total_count}
if self.matches:
result["matches"] = [
{"path": m.path, "line": m.line_number, "content": m.content}
@@ -256,6 +257,8 @@ class SearchResult:
result["counts"] = self.counts
if self.truncated:
result["truncated"] = True
if self.limit_reason:
result["limit_reason"] = self.limit_reason
if self.error:
result["error"] = self.error
return result
@@ -285,6 +288,16 @@ class ExecuteResult:
exit_code: int = 0
_SEARCH_TIMEOUT_MARKER_RE = re.compile(r"\n?\[Command timed out after \d+s\]\s*$")
def _search_stdout_and_limit(result: ExecuteResult) -> tuple[str, Optional[str]]:
"""Return stdout cleaned for parsing and a limit reason for search timeouts."""
if result.exit_code == 124:
return _SEARCH_TIMEOUT_MARKER_RE.sub("", result.stdout), "search_timeout"
return result.stdout, None
def _split_tool_diagnostics(output: str) -> tuple[str, str]:
"""Separate rg/grep diagnostic lines from real match output.
@@ -1967,15 +1980,17 @@ class ShellFileOperations(FileOperations):
f"-printf '%T@ %p\\n' 2>/dev/null | sort -rn{pagination_expr}"
result = self._exec(cmd, timeout=60)
stdout, limit_reason = _search_stdout_and_limit(result)
if not result.stdout.strip():
if not stdout.strip() and not limit_reason:
# Try without -printf (BSD find compatibility -- macOS)
cmd_simple = f"find {self._escape_shell_arg(path)}{hidden_filter_expr} -type f -name {self._escape_shell_arg(search_pattern)} " \
f"2>/dev/null | sort -rn{pagination_expr}"
result = self._exec(cmd_simple, timeout=60)
stdout, limit_reason = _search_stdout_and_limit(result)
files = []
for line in result.stdout.strip().split('\n'):
for line in stdout.strip().split('\n'):
if not line:
continue
parts = line.split(' ', 1)
@@ -2003,7 +2018,9 @@ class ShellFileOperations(FileOperations):
return SearchResult(
files=files,
total_count=len(files)
total_count=len(files),
truncated=bool(limit_reason),
limit_reason=limit_reason,
)
def _search_files_rg(self, pattern: str, path: str, limit: int, offset: int) -> SearchResult:
@@ -2029,9 +2046,10 @@ class ShellFileOperations(FileOperations):
f"| head -n {fetch_limit}"
)
result = self._exec(cmd_sorted, timeout=60)
all_files = [f for f in result.stdout.strip().split('\n') if f]
stdout, limit_reason = _search_stdout_and_limit(result)
all_files = [f for f in stdout.strip().split('\n') if f]
if not all_files:
if not all_files and not limit_reason:
# --sortr may have failed on older rg; retry without it.
cmd_plain = (
f"rg --files -g {self._escape_shell_arg(glob_pattern)} "
@@ -2039,14 +2057,16 @@ class ShellFileOperations(FileOperations):
f"| head -n {fetch_limit}"
)
result = self._exec(cmd_plain, timeout=60)
all_files = [f for f in result.stdout.strip().split('\n') if f]
stdout, limit_reason = _search_stdout_and_limit(result)
all_files = [f for f in stdout.strip().split('\n') if f]
page = all_files[offset:offset + limit]
return SearchResult(
files=page,
total_count=len(all_files),
truncated=len(all_files) >= fetch_limit,
truncated=len(all_files) >= fetch_limit or bool(limit_reason),
limit_reason=limit_reason,
)
def _search_content(self, pattern: str, path: str, file_glob: Optional[str],
@@ -2102,12 +2122,13 @@ class ShellFileOperations(FileOperations):
# introduce false errors on a successful-but-truncated search.
cmd = "set -o pipefail; " + " ".join(cmd_parts)
result = self._exec(cmd, timeout=60)
stdout, limit_reason = _search_stdout_and_limit(result)
# _exec merges stderr into stdout (stderr=subprocess.STDOUT), so rg's
# diagnostic lines ("rg: <file>: <error>", "rg: regex parse error:")
# are interleaved with match output. Split them out: diagnostics must
# not be parsed as matches, and on a hard error they ARE the message.
diagnostics, payload = _split_tool_diagnostics(result.stdout)
diagnostics, payload = _split_tool_diagnostics(stdout)
# rg exit codes: 0=matches found, 1=no matches, 2=error. rg returns 2
# even on partial errors (e.g. one unreadable file in a tree that
@@ -2124,7 +2145,12 @@ class ShellFileOperations(FileOperations):
all_files = [f for f in stdout.strip().split('\n') if f]
total = len(all_files)
page = all_files[offset:offset + limit]
return SearchResult(files=page, total_count=total)
return SearchResult(
files=page,
total_count=total,
truncated=bool(limit_reason),
limit_reason=limit_reason,
)
elif output_mode == "count":
counts = {}
@@ -2136,7 +2162,12 @@ class ShellFileOperations(FileOperations):
counts[parts[0]] = int(parts[1])
except ValueError:
pass
return SearchResult(counts=counts, total_count=sum(counts.values()))
return SearchResult(
counts=counts,
total_count=sum(counts.values()),
truncated=bool(limit_reason),
limit_reason=limit_reason,
)
else:
# Parse content matches and context lines.
@@ -2177,7 +2208,8 @@ class ShellFileOperations(FileOperations):
return SearchResult(
matches=page,
total_count=total,
truncated=total > offset + limit
truncated=total > offset + limit or bool(limit_reason),
limit_reason=limit_reason,
)
def _search_with_grep(self, pattern: str, path: str, file_glob: Optional[str],
@@ -2218,12 +2250,13 @@ class ShellFileOperations(FileOperations):
# pipefail does not turn truncated results into false errors.
cmd = "set -o pipefail; " + " ".join(cmd_parts)
result = self._exec(cmd, timeout=60)
stdout, limit_reason = _search_stdout_and_limit(result)
# _exec merges stderr into stdout, so grep's diagnostic lines
# ("grep: <file>: <error>") are interleaved with matches. Split them
# out so they're never parsed as matches and so a hard error has a
# clean message.
diagnostics, payload = _split_tool_diagnostics(result.stdout)
diagnostics, payload = _split_tool_diagnostics(stdout)
# grep exit codes: 0=matches found, 1=no matches, 2=error. grep
# returns 2 on partial errors (e.g. an unreadable file) even when
@@ -2238,7 +2271,12 @@ class ShellFileOperations(FileOperations):
all_files = [f for f in stdout.strip().split('\n') if f]
total = len(all_files)
page = all_files[offset:offset + limit]
return SearchResult(files=page, total_count=total)
return SearchResult(
files=page,
total_count=total,
truncated=bool(limit_reason),
limit_reason=limit_reason,
)
elif output_mode == "count":
counts = {}
@@ -2250,7 +2288,12 @@ class ShellFileOperations(FileOperations):
counts[parts[0]] = int(parts[1])
except ValueError:
pass
return SearchResult(counts=counts, total_count=sum(counts.values()))
return SearchResult(
counts=counts,
total_count=sum(counts.values()),
truncated=bool(limit_reason),
limit_reason=limit_reason,
)
else:
# grep match lines: "file:lineno:content" (colon)
@@ -2288,5 +2331,6 @@ class ShellFileOperations(FileOperations):
return SearchResult(
matches=page,
total_count=total,
truncated=total > offset + limit
truncated=total > offset + limit or bool(limit_reason),
limit_reason=limit_reason,
)
+47 -1
View File
@@ -760,6 +760,52 @@ def read_file_tool(path: str, offset: int = 1, limit: int = 500, task_id: str =
_resolved = _resolve_path_for_task(path, task_id)
# ── Structured-document extraction ────────────────────────────
# Try before the binary-extension guard so .docx/.xlsx can render as text.
# Malformed documents fall through to the normal path/binary guard.
from tools.read_extract import ExtractionError, extract_document_text, is_extractable_document
if is_extractable_document(str(_resolved)):
try:
extracted_text = extract_document_text(str(_resolved))
except ExtractionError:
logger.debug("document extraction failed for %s", path, exc_info=True)
else:
file_ops = _get_file_ops(task_id)
lines = extracted_text.splitlines()
total_lines = len(lines)
end_line = offset + limit - 1
page_text = "\n".join(lines[offset - 1:end_line])
result_dict = {
"content": file_ops._add_line_numbers(page_text, offset) if page_text else "",
"total_lines": total_lines,
"file_size": os.path.getsize(_resolved),
"truncated": total_lines > end_line,
"extracted_document": True,
}
if result_dict["truncated"]:
result_dict["hint"] = (
f"Use offset={end_line + 1} to continue reading "
f"(showing {offset}-{min(end_line, total_lines)} of {total_lines} lines)"
)
content_len = len(result_dict["content"])
max_chars = _get_max_read_chars()
if content_len > max_chars:
return json.dumps({
"error": (
f"Read produced {content_len:,} characters which exceeds "
f"the safety limit ({max_chars:,} chars). "
"Use offset and limit to read a smaller range. "
f"The document has {total_lines} lines of extracted text."
),
"path": path,
"total_lines": total_lines,
"file_size": result_dict["file_size"],
}, ensure_ascii=False)
if result_dict["content"]:
result_dict["content"] = redact_sensitive_text(result_dict["content"], code_file=True)
return json.dumps(result_dict, ensure_ascii=False)
# ── Binary file guard ─────────────────────────────────────────
# Block binary files by extension (no I/O).
if has_binary_extension(str(_resolved)):
@@ -1427,7 +1473,7 @@ def _check_file_reqs():
READ_FILE_SCHEMA = {
"name": "read_file",
"description": "Read a text file with line numbers and pagination. Use this instead of cat/head/tail in terminal. Output format: 'LINE_NUM|CONTENT'. Suggests similar filenames if not found. Use offset and limit for large files. Reads exceeding ~100K characters are rejected; use offset and limit to read specific sections of large files. NOTE: Cannot read images or binary files — use vision_analyze for images.",
"description": "Read a text file with line numbers and pagination. Use this instead of cat/head/tail in terminal. Output format: 'LINE_NUM|CONTENT'. Suggests similar filenames if not found. Use offset and limit for large files. Reads exceeding ~100K characters are rejected; use offset and limit to read specific sections of large files. Jupyter notebooks (.ipynb), Word documents (.docx), and Excel workbooks (.xlsx) are auto-extracted to readable text. NOTE: Cannot read images or other binary files — use vision_analyze for images.",
"parameters": {
"type": "object",
"properties": {
+248
View File
@@ -0,0 +1,248 @@
"""Stdlib document-to-text extraction for ``read_file``.
Supports Jupyter notebooks, DOCX, and XLSX without adding hard dependencies.
Malformed documents raise :class:`ExtractionError`; callers can then fall back to
normal text/binary handling.
"""
from __future__ import annotations
import json
import posixpath
import zipfile
from pathlib import Path
from xml.etree import ElementTree as ET
__all__ = ["EXTRACTABLE_EXTENSIONS", "ExtractionError", "extract_document_text", "is_extractable_document"]
EXTRACTABLE_EXTENSIONS = frozenset({".ipynb", ".docx", ".xlsx"})
MAX_XLSX_BYTES = 50 * 1024 * 1024
_MAX_XLSX_ROWS_PER_SHEET = 5000
_MAX_XLSX_COLS = 256
_NS_W = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
_NS_S = "http://schemas.openxmlformats.org/spreadsheetml/2006/main"
_NS_REL = "http://schemas.openxmlformats.org/officeDocument/2006/relationships"
_NS_PKG_REL = "http://schemas.openxmlformats.org/package/2006/relationships"
class ExtractionError(Exception):
"""Raised when a supported-looking document cannot be rendered as text."""
def _extension(path: str) -> str:
ext = Path(path).suffix.lower()
return ext if ext in EXTRACTABLE_EXTENSIONS else ""
def is_extractable_document(path: str) -> bool:
return bool(_extension(path))
def extract_document_text(path: str) -> str:
ext = _extension(path)
if ext == ".ipynb":
return _extract_notebook(path)
if ext == ".docx":
return _extract_docx(path)
if ext == ".xlsx":
return _extract_xlsx(path)
raise ExtractionError(f"Unsupported document type: {path!r}")
def _source_text(source) -> str:
if isinstance(source, str):
return source
if isinstance(source, list):
return "".join(item for item in source if isinstance(item, str))
return ""
def _extract_notebook(path: str) -> str:
try:
with open(path, encoding="utf-8", errors="replace") as fh:
nb = json.load(fh)
except (OSError, ValueError, json.JSONDecodeError) as exc:
raise ExtractionError(f"Not a valid notebook: {exc}") from exc
if not isinstance(nb, dict):
raise ExtractionError("Notebook root is not an object")
cells = nb.get("cells")
if not isinstance(cells, list):
cells = [
cell
for ws in nb.get("worksheets", [])
if isinstance(ws, dict)
for cell in ws.get("cells", [])
]
if not cells:
raise ExtractionError("Notebook contains no cells")
counts = {"markdown": 0, "code": 0, "raw": 0}
labels = {"markdown": "Markdown", "code": "Code", "raw": "Raw"}
out: list[str] = []
for cell in cells:
if not isinstance(cell, dict):
continue
typ = cell.get("cell_type")
if typ not in labels:
continue
counts[typ] += 1
suffix = f" {counts[typ]}" if typ != "raw" else ""
out.extend((f"# ── {labels[typ]} cell{suffix} ──", _source_text(cell.get("source", "")).rstrip("\n"), ""))
if not out:
raise ExtractionError("Notebook contains no readable cells")
return "\n".join(out).rstrip("\n") + "\n"
def _zip_xml(zf: zipfile.ZipFile, name: str) -> ET.Element:
try:
return ET.fromstring(zf.read(name))
except KeyError as exc:
raise ExtractionError(f"Missing {name}") from exc
except ET.ParseError as exc:
raise ExtractionError(f"Malformed XML in {name}: {exc}") from exc
def _extract_docx(path: str) -> str:
try:
with zipfile.ZipFile(path) as zf:
root = _zip_xml(zf, "word/document.xml")
except zipfile.BadZipFile as exc:
raise ExtractionError(f"Not a valid DOCX: {exc}") from exc
except OSError as exc:
raise ExtractionError(str(exc)) from exc
w = f"{{{_NS_W}}}"
lines: list[str] = []
for para in root.iter(f"{w}p"):
buf: list[str] = []
for node in para.iter():
if node.tag == f"{w}t":
buf.append(node.text or "")
elif node.tag == f"{w}tab":
buf.append("\t")
elif node.tag in {f"{w}br", f"{w}cr"}:
buf.append("\n")
lines.extend("".join(buf).split("\n"))
if not any(line.strip() for line in lines):
raise ExtractionError("DOCX contains no extractable text")
return "\n".join(lines).rstrip("\n") + "\n"
def _extract_xlsx(path: str) -> str:
try:
with zipfile.ZipFile(path) as zf:
names = set(zf.namelist())
shared = _shared_strings(zf, names)
sheets = _workbook_sheets(zf)
rels = _workbook_rels(zf, names)
out: list[str] = []
for name, state, rid in sheets:
if state in {"hidden", "veryHidden"}:
continue
part = _sheet_part(rels.get(rid, ""))
if part not in names:
continue
try:
rows = _sheet_rows(zf.read(part), shared)
except ET.ParseError:
continue
out.append(f"# ── Sheet: {name} ──")
out.extend("\t".join(row) for row in rows)
if not rows:
out.append("(empty)")
out.append("")
except zipfile.BadZipFile as exc:
raise ExtractionError(f"Not a valid XLSX: {exc}") from exc
except OSError as exc:
raise ExtractionError(str(exc)) from exc
if not out:
raise ExtractionError("XLSX has no visible sheets with content")
return "\n".join(out).rstrip("\n") + "\n"
def _shared_strings(zf: zipfile.ZipFile, names: set[str]) -> list[str]:
if "xl/sharedStrings.xml" not in names:
return []
try:
root = ET.fromstring(zf.read("xl/sharedStrings.xml"))
except ET.ParseError:
return []
s = f"{{{_NS_S}}}"
return ["".join(t.text or "" for t in item.iter(f"{s}t")) for item in root.iter(f"{s}si")]
def _workbook_sheets(zf: zipfile.ZipFile) -> list[tuple[str, str, str]]:
root = _zip_xml(zf, "xl/workbook.xml")
s, r = f"{{{_NS_S}}}", f"{{{_NS_REL}}}"
return [
(sheet.get("name", "Sheet"), sheet.get("state", "visible"), sheet.get(f"{r}id", ""))
for sheet in root.iter(f"{s}sheet")
]
def _workbook_rels(zf: zipfile.ZipFile, names: set[str]) -> dict[str, str]:
rels_path = "xl/_rels/workbook.xml.rels"
if rels_path not in names:
return {}
try:
root = ET.fromstring(zf.read(rels_path))
except ET.ParseError:
return {}
rel_tag = f"{{{_NS_PKG_REL}}}Relationship"
return {rel.get("Id", ""): rel.get("Target", "") for rel in root.iter(rel_tag) if rel.get("Id")}
def _sheet_part(target: str) -> str:
target = target.lstrip("/")
return posixpath.normpath(target if target.startswith("xl/") else f"xl/{target}")
def _col_index(ref: str) -> int:
idx = 0
for ch in ref:
if not ch.isalpha():
break
idx = idx * 26 + ord(ch.upper()) - ord("A") + 1
return max(idx - 1, 0)
def _sheet_rows(xml_bytes: bytes, shared: list[str]) -> list[list[str]]:
root = ET.fromstring(xml_bytes)
s = f"{{{_NS_S}}}"
rows: list[list[str]] = []
for row in root.iter(f"{s}row"):
if len(rows) >= _MAX_XLSX_ROWS_PER_SHEET:
break
cells: dict[int, str] = {}
max_col = -1
for cell in row.iter(f"{s}c"):
col = _col_index(cell.get("r", "")) if cell.get("r") else max_col + 1
if col >= _MAX_XLSX_COLS:
continue
cells[col] = _cell_value(cell, shared, s)
max_col = max(max_col, col)
rows.append([cells.get(i, "") for i in range(max_col + 1)] if max_col >= 0 else [])
while rows and not any(value.strip() for value in rows[-1]):
rows.pop()
return rows
def _cell_value(cell: ET.Element, shared: list[str], s: str) -> str:
value = cell.findtext(f"{s}v") or ""
typ = cell.get("t", "")
if typ == "s":
try:
return shared[int(value)]
except (ValueError, IndexError):
return ""
if typ == "inlineStr":
inline = cell.find(f"{s}is")
return "" if inline is None else "".join(t.text or "" for t in inline.iter(f"{s}t"))
if typ == "b":
return "TRUE" if value.strip() in {"1", "true", "TRUE"} else "FALSE"
if typ == "e":
return value or "#ERROR"
return value
+28 -2
View File
@@ -1,8 +1,10 @@
"""Shared utility functions for hermes-agent."""
import errno
import json
import logging
import os
import shutil
import stat
import tempfile
from pathlib import Path
@@ -71,14 +73,38 @@ def atomic_replace(tmp_path: Union[str, Path], target: Union[str, Path]) -> str:
This helper resolves the symlink first so ``os.replace`` writes to
the real file in-place while the symlink survives. For non-symlink
and non-existent paths the behavior is identical to a plain
``os.replace`` call.
``os.replace`` call unless the rename fails with ``EXDEV`` or ``EBUSY``;
those cases fall back to copy/fsync/unlink for cross-device, bind-mount,
and busy-file deployments.
Returns the resolved real path used for the replace, so callers that
need to re-apply permissions can target it instead of the symlink.
"""
target_str = str(target)
real_path = os.path.realpath(target_str) if os.path.islink(target_str) else target_str
os.replace(str(tmp_path), real_path)
tmp_str = str(tmp_path)
try:
os.replace(tmp_str, real_path)
except OSError as exc:
if exc.errno not in (errno.EXDEV, errno.EBUSY):
raise
logger.debug(
"atomic_replace: %s -> %s failed with %s; falling back to copy",
tmp_str,
real_path,
errno.errorcode.get(exc.errno, exc.errno),
)
shutil.copyfile(tmp_str, real_path)
try:
shutil.copystat(tmp_str, real_path)
except OSError:
pass
try:
with open(real_path, "rb") as f:
os.fsync(f.fileno())
except OSError:
pass
os.unlink(tmp_str)
return real_path
+2 -2
View File
@@ -286,8 +286,8 @@ export const af: Translations = {
nameRequired: "Naam word vereis",
nameRule:
"Slegs kleinletters, syfers, _ en -; moet met 'n letter of syfer begin; tot 64 karakters.",
invalidName: "Ongeldige profielnaam",
cloneFromDefault: "Kloon konfigurasie vanaf verstekprofiel",
invalidName: "Ongeldige profielnaam", cloneFrom: "Kloon konfigurasie vanaf profiel",
cloneFromNone: "Geen (leeg)",
allProfiles: "Profiele",
noProfiles: "Geen profiele gevind nie.",
defaultBadge: "verstek",
+2 -2
View File
@@ -286,8 +286,8 @@ export const de: Translations = {
nameRequired: "Name ist erforderlich",
nameRule:
"Nur Kleinbuchstaben, Ziffern, _ und -; muss mit einem Buchstaben oder einer Ziffer beginnen; maximal 64 Zeichen.",
invalidName: "Ungültiger Profilname",
cloneFromDefault: "Konfiguration vom Standardprofil klonen",
invalidName: "Ungültiger Profilname", cloneFrom: "Konfiguration klonen von",
cloneFromNone: "Keine (leer)",
allProfiles: "Profile",
noProfiles: "Keine Profile gefunden.",
defaultBadge: "Standard",
+2 -1
View File
@@ -297,7 +297,8 @@ export const en: Translations = {
nameRule:
"Lowercase letters, digits, _ and - only; must start with a letter or digit; up to 64 characters.",
invalidName: "Invalid profile name",
cloneFromDefault: "Clone config from default profile",
cloneFrom: "Clone config from",
cloneFromNone: "None (blank)",
allProfiles: "Profiles",
noProfiles: "No profiles found.",
defaultBadge: "default",
+2 -1
View File
@@ -287,7 +287,8 @@ export const es: Translations = {
nameRule:
"Solo letras minúsculas, dígitos, _ y -; debe comenzar con una letra o dígito; hasta 64 caracteres.",
invalidName: "Nombre de perfil no válido",
cloneFromDefault: "Clonar configuración del perfil predeterminado",
cloneFrom: "Clonar desde el perfil",
cloneFromNone: "Ninguno (vacío)",
allProfiles: "Perfiles",
noProfiles: "No se encontraron perfiles.",
defaultBadge: "predeterminado",
+2 -1
View File
@@ -287,7 +287,8 @@ export const fr: Translations = {
nameRule:
"Lettres minuscules, chiffres, _ et - uniquement ; doit commencer par une lettre ou un chiffre ; jusqu'à 64 caractères.",
invalidName: "Nom de profil invalide",
cloneFromDefault: "Cloner la configuration du profil par défaut",
cloneFrom: "Cloner depuis le profil",
cloneFromNone: "Aucun (vide)",
allProfiles: "Profils",
noProfiles: "Aucun profil trouvé.",
defaultBadge: "défaut",
+2 -2
View File
@@ -294,8 +294,8 @@ export const ga: Translations = {
nameRequired: "Tá ainm riachtanach",
nameRule:
"Litreacha cás íochtair, digití, _ agus - amháin; caithfidh tús a chur le litir nó digit; suas le 64 carachtar.",
invalidName: "Ainm próifíle neamhbhailí",
cloneFromDefault: "Clónáil cumraíocht ón bpróifíl réamhshocraithe",
invalidName: "Ainm próifíle neamhbhailí", cloneFrom: "Clónáil cumraíocht ón bpróifíl",
cloneFromNone: "Dada (folamh)",
allProfiles: "Próifílí",
noProfiles: "Níor aimsíodh próifílí.",
defaultBadge: "réamhshocraithe",
+2 -2
View File
@@ -286,8 +286,8 @@ export const hu: Translations = {
nameRequired: "A név kötelező",
nameRule:
"Csak kisbetűk, számjegyek, _ és - karakterek; betűvel vagy számjeggyel kell kezdődnie; legfeljebb 64 karakter.",
invalidName: "Érvénytelen profilnév",
cloneFromDefault: "Konfiguráció klónozása az alapértelmezett profilból",
invalidName: "Érvénytelen profilnév", cloneFrom: "Konfiguráció klónozása ebből a profilból",
cloneFromNone: "Nincs (üres)",
allProfiles: "Profilok",
noProfiles: "Nem található profil.",
defaultBadge: "alapértelmezett",
+2 -2
View File
@@ -286,8 +286,8 @@ export const it: Translations = {
nameRequired: "Il nome è obbligatorio",
nameRule:
"Solo lettere minuscole, cifre, _ e -; deve iniziare con una lettera o cifra; fino a 64 caratteri.",
invalidName: "Nome del profilo non valido",
cloneFromDefault: "Clona la configurazione dal profilo predefinito",
invalidName: "Nome del profilo non valido", cloneFrom: "Clona configurazione dal profilo",
cloneFromNone: "Nessuno (vuoto)",
allProfiles: "Profili",
noProfiles: "Nessun profilo trovato.",
defaultBadge: "predefinito",
+2 -2
View File
@@ -285,8 +285,8 @@ export const ja: Translations = {
nameRequired: "名前は必須です",
nameRule:
"小文字、数字、_ および - のみ使用可能。最初は文字または数字で始める必要があります。最大 64 文字。",
invalidName: "無効なプロファイル名",
cloneFromDefault: "デフォルトプロファイルから設定を複製",
invalidName: "無効なプロファイル名", cloneFrom: "プロファイルから複製",
cloneFromNone: "なし(空)",
allProfiles: "プロファイル",
noProfiles: "プロファイルが見つかりません。",
defaultBadge: "デフォルト",
+2 -2
View File
@@ -285,8 +285,8 @@ export const ko: Translations = {
nameRequired: "이름은 필수입니다",
nameRule:
"소문자, 숫자, _ 및 - 만 사용 가능합니다. 문자나 숫자로 시작해야 하며 최대 64자입니다.",
invalidName: "잘못된 프로필 이름입니다",
cloneFromDefault: "기본 프로필에서 설정 복제",
invalidName: "잘못된 프로필 이름입니다", cloneFrom: "프로필에서 복제",
cloneFromNone: "없음 (빈 상태)",
allProfiles: "프로필",
noProfiles: "프로필을 찾을 수 없습니다.",
defaultBadge: "기본",
+2 -1
View File
@@ -287,7 +287,8 @@ export const pt: Translations = {
nameRule:
"Apenas letras minúsculas, dígitos, _ e -; deve começar com letra ou dígito; até 64 caracteres.",
invalidName: "Nome de perfil inválido",
cloneFromDefault: "Clonar configuração do perfil predefinido",
cloneFrom: "Clonar a partir do perfil",
cloneFromNone: "Nenhum (vazio)",
allProfiles: "Perfis",
noProfiles: "Não foram encontrados perfis.",
defaultBadge: "predefinido",
+2 -2
View File
@@ -286,8 +286,8 @@ export const ru: Translations = {
nameRequired: "Имя обязательно",
nameRule:
"Только строчные буквы, цифры, _ и -; должно начинаться с буквы или цифры; до 64 символов.",
invalidName: "Недопустимое имя профиля",
cloneFromDefault: "Клонировать конфигурацию из профиля по умолчанию",
invalidName: "Недопустимое имя профиля", cloneFrom: "Клонировать конфигурацию из профиля",
cloneFromNone: "Нет (пусто)",
allProfiles: "Профили",
noProfiles: "Профили не найдены.",
defaultBadge: "по умолчанию",
+2 -2
View File
@@ -286,8 +286,8 @@ export const tr: Translations = {
nameRequired: "Ad gereklidir",
nameRule:
"Yalnızca küçük harfler, rakamlar, _ ve - kullanılabilir; harf veya rakamla başlamalı; en fazla 64 karakter.",
invalidName: "Geçersiz profil adı",
cloneFromDefault: "Varsayılan profilden yapılandırmayı klonla",
invalidName: "Geçersiz profil adı", cloneFrom: "Profilden yapılandırmayı klonla",
cloneFromNone: "Hiçbiri (boş)",
allProfiles: "Profiller",
noProfiles: "Profil bulunamadı.",
defaultBadge: "varsayılan",
+2 -1
View File
@@ -354,7 +354,8 @@ export interface Translations {
nameRequired: string;
nameRule: string;
invalidName: string;
cloneFromDefault: string;
cloneFrom: string;
cloneFromNone: string;
allProfiles: string;
noProfiles: string;
defaultBadge: string;
+2 -1
View File
@@ -287,7 +287,8 @@ export const uk: Translations = {
nameRule:
"Лише малі літери, цифри, _ та -; має починатися з літери або цифри; до 64 символів.",
invalidName: "Недопустима назва профілю",
cloneFromDefault: "Клонувати конфігурацію з профілю за замовчуванням",
cloneFrom: "Клонувати з профілю",
cloneFromNone: "Жоден (порожній)",
allProfiles: "Профілі",
noProfiles: "Профілів не знайдено.",
defaultBadge: "за замовчуванням",
+2 -2
View File
@@ -285,8 +285,8 @@ export const zhHant: Translations = {
nameRequired: "名稱為必填",
nameRule:
"僅允許小寫字母、數字、底線及連字號;首字必須為字母或數字;最多 64 個字元。",
invalidName: "設定檔名稱無效",
cloneFromDefault: "從預設設定檔複製設定",
invalidName: "設定檔名稱無效", cloneFrom: "從設定檔複製",
cloneFromNone: "無(空白)",
allProfiles: "設定檔",
noProfiles: "找不到設定檔。",
defaultBadge: "預設",
+2 -2
View File
@@ -282,8 +282,8 @@ export const zh: Translations = {
nameRequired: "名称必填",
nameRule:
"仅允许小写字母、数字、下划线和短横线;首字符必须是字母或数字;最多 64 个字符。",
invalidName: "多Agent配置名称非法",
cloneFromDefault: "从默认多Agent配置克隆配置",
invalidName: "多Agent配置名称非法", cloneFrom: "从配置文件克隆",
cloneFromNone: "无(空白)",
allProfiles: "多Agent配置列表",
noProfiles: "暂无多Agent配置。",
defaultBadge: "默认",
+2 -1
View File
@@ -552,7 +552,8 @@ export const api = {
}),
createProfile: (body: {
name: string;
clone_from_default: boolean;
clone_from?: string | null;
clone_from_default?: boolean;
clone_all?: boolean;
no_skills?: boolean;
description?: string;
+1 -1
View File
@@ -220,7 +220,7 @@ export default function ProfileBuilderPage() {
try {
const res = await api.createProfile({
name: n,
clone_from_default: false,
clone_from: null,
description: description.trim() || undefined,
provider: pickedModel?.provider,
model: pickedModel?.model,
+32 -26
View File
@@ -35,11 +35,11 @@ import { Badge } from "@nous-research/ui/ui/components/badge";
import { Button } from "@nous-research/ui/ui/components/button";
import { Input } from "@nous-research/ui/ui/components/input";
import { Label } from "@nous-research/ui/ui/components/label";
import { Checkbox } from "@nous-research/ui/ui/components/checkbox";
import {
Select,
SelectOption,
} from "@nous-research/ui/ui/components/select";
import { Checkbox } from "@nous-research/ui/ui/components/checkbox";
import { useI18n } from "@/i18n";
import { usePageHeader } from "@/contexts/usePageHeader";
import { cn, themedBody } from "@/lib/utils";
@@ -312,7 +312,7 @@ export default function ProfilesPage() {
// Create modal
const [createModalOpen, setCreateModalOpen] = useState(false);
const [newName, setNewName] = useState("");
const [cloneFromDefault, setCloneFromDefault] = useState(true);
const [cloneFrom, setCloneFrom] = useState<string | null>("default");
const [cloneAll, setCloneAll] = useState(false);
const [noSkills, setNoSkills] = useState(false);
const [newDescription, setNewDescription] = useState("");
@@ -429,7 +429,7 @@ export default function ProfilesPage() {
}
setCreating(true);
try {
const cloning = cloneAll || cloneFromDefault;
const cloning = cloneFrom !== null;
const picked = modelChoice
? modelChoices?.find(
(c) => `${c.provider}\u0000${c.model}` === modelChoice,
@@ -437,8 +437,8 @@ export default function ProfilesPage() {
: undefined;
const res = await api.createProfile({
name,
clone_from_default: cloneAll ? false : cloneFromDefault,
clone_all: cloneAll,
clone_from: cloneFrom,
clone_all: cloning && cloneAll,
no_skills: cloning ? false : noSkills,
description: newDescription.trim() || undefined,
provider: picked?.provider,
@@ -455,7 +455,7 @@ export default function ProfilesPage() {
setNewDescription("");
setNoSkills(false);
setCloneAll(false);
setCloneFromDefault(true);
setCloneFrom("default");
setModelChoice("");
setCreateModalOpen(false);
load();
@@ -772,7 +772,7 @@ export default function ProfilesPage() {
};
}, [setEnd, t.common.create, loading, navigate]);
const cloning = cloneAll || cloneFromDefault;
const cloning = cloneFrom !== null;
if (loading) {
return (
@@ -862,6 +862,26 @@ export default function ProfilesPage() {
</p>
</div>
<div className="grid gap-2">
<Label htmlFor="clone-from">{t.profiles.cloneFrom}</Label>
<Select
id="clone-from"
value={cloneFrom ?? ""}
onValueChange={(v) => {
const next = v || null;
setCloneFrom(next);
if (next === null) setCloneAll(false);
}}
>
<SelectOption value="">{t.profiles.cloneFromNone}</SelectOption>
{profiles.map((profile) => (
<SelectOption key={profile.name} value={profile.name}>
{profile.name}
</SelectOption>
))}
</Select>
</div>
<div className="grid gap-2">
<Label htmlFor="profile-description">
{L.descriptionOptional}
@@ -909,33 +929,19 @@ export default function ProfilesPage() {
{L.advancedOptions}
</legend>
<div className="flex items-center gap-2.5">
<Checkbox
checked={cloneFromDefault}
id="clone-from-default"
disabled={cloneAll}
onCheckedChange={(checked) =>
setCloneFromDefault(checked === true)
}
/>
<Label
className="font-mondwest normal-case tracking-normal text-sm cursor-pointer"
htmlFor="clone-from-default"
>
{t.profiles.cloneFromDefault}
</Label>
</div>
<div className="flex items-center gap-2.5">
<Checkbox
checked={cloneAll}
disabled={!cloning}
id="clone-all"
onCheckedChange={(checked) => setCloneAll(checked === true)}
/>
<Label
className="font-mondwest normal-case tracking-normal text-sm cursor-pointer"
className={cn(
"font-mondwest normal-case tracking-normal text-sm cursor-pointer",
!cloning && "opacity-50",
)}
htmlFor="clone-all"
>
{L.cloneAll}
+1 -1
View File
@@ -1396,7 +1396,7 @@ Manage profiles — multiple isolated Hermes instances, each with its own config
|------------|-------------|
| `list` | List all profiles. |
| `use <name>` | Set a sticky default profile. |
| `create <name> [--clone] [--clone-all] [--clone-from <source>] [--no-alias]` | Create a new profile. `--clone` copies config, `.env`, and `SOUL.md` from the active profile. `--clone-all` copies all state. `--clone-from` specifies a source profile. |
| `create <name> [--clone] [--clone-all] [--clone-from <source>] [--no-alias]` | Create a new profile. `--clone` copies config, `.env`, `SOUL.md`, and skills from the active profile. `--clone-all` copies all state. `--clone-from` specifies a source profile and implies config clone unless paired with `--clone-all`. |
| `delete <name> [-y]` | Delete a profile. |
| `show <name>` | Show profile details (home directory, config, etc.). |
| `alias <name> [--remove] [--name NAME]` | Manage wrapper scripts for quick profile access. |
+1 -1
View File
@@ -634,7 +634,7 @@ No. Each messaging platform (Telegram, Discord, etc.) requires exclusive access
### Do profiles share memory or sessions?
No. Each profile has its own memory store, session database, and skills directory. They are completely isolated. If you want to start a new profile with existing memories and sessions, use `hermes profile create newname --clone-all` to copy everything from the current profile.
No. Each profile has its own memory store, session database, and skills directory. They are completely isolated. If you want to start a new profile with existing memories and sessions, use `hermes profile create newname --clone-all` to copy everything from the current profile, or add `--clone-from <profile>` to copy from a specific source profile.
### What happens when I run `hermes update`?
+7 -4
View File
@@ -80,12 +80,12 @@ Creates a new profile.
| Argument / Option | Description |
|-------------------|-------------|
| `<name>` | Name for the new profile. Must be a valid directory name (alphanumeric, hyphens, underscores). |
| `--clone` | Copy `config.yaml`, `.env`, and `SOUL.md` from the current profile. |
| `--clone` | Copy `config.yaml`, `.env`, `SOUL.md`, and skills from the current profile. |
| `--clone-all` | Copy everything (config, memories, skills, cron, plugins) from the current profile. Excludes per-profile history: sessions, `state.db`, backups, state-snapshots, checkpoints. |
| `--clone-from <profile>` | Clone from a specific profile instead of the current one. Used with `--clone` or `--clone-all`. |
| `--clone-from <profile>` | Clone config/skills/SOUL from a specific profile instead of the current one. Implies `--clone` unless paired with `--clone-all`. |
| `--no-alias` | Skip wrapper script creation. |
| `--description "<text>"` | One- or two-sentence description of what this profile is good at. Used by the kanban orchestrator to route tasks based on role instead of profile name alone. Skip and add later via `hermes profile describe`. Persisted in `<profile_dir>/profile.yaml`. |
| `--no-skills` | Create an **empty** profile with zero bundled skills enabled. Writes a `.no-bundled-skills` marker into the profile so future `hermes update` runs won't re-seed the bundled set, and refuses to combine with `--clone` / `--clone-all` (which would copy skills in anyway). Useful for narrow orchestrator profiles or sandbox profiles that should not inherit the full skill catalog. To toggle this on an already-created profile (including the default `~/.hermes`), use `hermes skills opt-out` / `hermes skills opt-in`. |
| `--no-skills` | Create an **empty** profile with zero bundled skills enabled. Writes a `.no-bundled-skills` marker into the profile so future `hermes update` runs won't re-seed the bundled set, and refuses to combine with `--clone`, `--clone-from`, or `--clone-all` (which would copy skills in anyway). Useful for narrow orchestrator profiles or sandbox profiles that should not inherit the full skill catalog. To toggle this on an already-created profile (including the default `~/.hermes`), use `hermes skills opt-out` / `hermes skills opt-in`. |
Creating a profile does **not** make that profile directory the default project/workspace directory for terminal commands. If you want a profile to start in a specific project, set `terminal.cwd` in that profile's `config.yaml`.
@@ -102,7 +102,10 @@ hermes profile create work --clone
hermes profile create backup --clone-all
# Clone config from a specific profile
hermes profile create work2 --clone --clone-from work
hermes profile create work2 --clone-from work
# Clone everything from a specific profile
hermes profile create work2-backup --clone-from work --clone-all
```
## `hermes profile describe`
+9 -3
View File
@@ -50,7 +50,7 @@ You can also set or auto-generate the description later with `hermes profile des
hermes profile create work --clone
```
Copies your current profile's `config.yaml`, `.env`, and `SOUL.md` into the new profile. Same API keys and model, but fresh sessions and memory. Edit `~/.hermes/profiles/work/.env` for different API keys, or `~/.hermes/profiles/work/SOUL.md` for a different personality.
Copies your current profile's `config.yaml`, `.env`, `SOUL.md`, and skills into the new profile. Same API keys, model, and capabilities, but fresh sessions and memory. Edit `~/.hermes/profiles/work/.env` for different API keys, or `~/.hermes/profiles/work/SOUL.md` for a different personality.
### Clone everything (`--clone-all`)
@@ -63,11 +63,17 @@ Copies **everything** — config, API keys, personality, all memories, skills, c
### Clone from a specific profile
```bash
hermes profile create work --clone --clone-from coder
hermes profile create work --clone-from coder
```
`--clone-from <source>` selects the source profile directly and implies a config/skills/SOUL clone. Combine it with `--clone-all` when you want a full copy of that source profile:
```bash
hermes profile create work-backup --clone-from coder --clone-all
```
:::tip Honcho memory + profiles
When Honcho is enabled, `--clone` automatically creates a dedicated AI peer for the new profile while sharing the same user workspace. Each profile builds its own observations and identity. See [Honcho -- Multi-agent / Profiles](./features/memory-providers.md#honcho) for details.
When Honcho is enabled, clone operations automatically create a dedicated AI peer for the new profile while sharing the same user workspace. Each profile builds its own observations and identity. See [Honcho -- Multi-agent / Profiles](./features/memory-providers.md#honcho) for details.
:::
## Using profiles
@@ -1175,7 +1175,7 @@ hermes profile <subcommand>
|------------|-------------|
| `list` | 列出所有 profile。 |
| `use <name>` | 设置粘性默认 profile。 |
| `create <name> [--clone] [--clone-all] [--clone-from <source>] [--no-alias]` | 创建新 profile。`--clone` 从活跃 profile 复制 config、`.env``SOUL.md``--clone-all` 复制所有状态。`--clone-from` 指定源 profile。 |
| `create <name> [--clone] [--clone-all] [--clone-from <source>] [--no-alias]` | 创建新 profile。`--clone` 从活跃 profile 复制 config、`.env``SOUL.md` 和 skills`--clone-all` 复制所有状态。`--clone-from` 指定源 profile,除非与 `--clone-all` 配合使用,否则会隐含 config 克隆。 |
| `delete <name> [-y]` | 删除 profile。 |
| `show <name>` | 显示 profile 详情(主目录、config 等)。 |
| `alias <name> [--remove] [--name NAME]` | 管理快速访问 profile 的包装脚本。 |
@@ -626,7 +626,7 @@ Profiles 是构建在 `HERMES_HOME` 之上的托管层。您*可以*在每次命
### Profiles 共享记忆或会话吗?
不共享。每个 profile 都有自己独立的记忆存储、会话数据库和技能目录,完全隔离。如果您想用现有的记忆和会话创建新 profile,请使用 `hermes profile create newname --clone-all` 从当前 profile 复制所有内容。
不共享。每个 profile 都有自己独立的记忆存储、会话数据库和技能目录,完全隔离。如果您想用现有的记忆和会话创建新 profile,请使用 `hermes profile create newname --clone-all` 从当前 profile 复制所有内容,或添加 `--clone-from <profile>` 从指定源 profile 复制
### 运行 `hermes update` 时会发生什么?
@@ -79,12 +79,12 @@ hermes profile create <name> [options]
| 参数 / 选项 | 描述 |
|-------------------|-------------|
| `<name>` | 新 profile 的名称。必须是合法的目录名(字母数字、连字符、下划线)。 |
| `--clone` | 从当前 profile 复制 `config.yaml``.env``SOUL.md`。 |
| `--clone-all` | 从当前 profile 复制所有内容(config、memories、skills、sessions、state。 |
| `--clone-from <profile>` | 从指定 profile 克隆,而非当前 profile。与 `--clone``--clone-all` 配合使用。 |
| `--clone` | 从当前 profile 复制 `config.yaml``.env``SOUL.md` 和 skills。 |
| `--clone-all` | 从当前 profile 复制所有内容(config、memories、skills、cron、plugins)。会排除每个 profile 自己的历史数据:sessions、`state.db`、backups、state-snapshots、checkpoints。 |
| `--clone-from <profile>` | 从指定 profile 克隆 config/skills/SOUL,而非当前 profile。除非`--clone-all` 配合使用,否则会隐含 `--clone`。 |
| `--no-alias` | 跳过 wrapper 脚本创建。 |
| `--description "<text>"` | 一到两句话描述该 profile 的用途。供 kanban 编排器根据角色而非仅凭 profile 名称来路由任务。可跳过,稍后通过 `hermes profile describe` 添加。持久化保存在 `<profile_dir>/profile.yaml` 中。 |
| `--no-skills` | 创建一个**空** profile,不启用任何内置 skill。会在 profile 目录中写入 `.no-skills` 标记,使后续 `hermes update` 不再重新植入内置 skill 集,且拒绝与 `--clone` / `--clone-all` 组合使用(因为后者会复制 skill)。适用于不应继承完整 skill 目录的窄化编排器 profile 或沙箱 profile。 |
| `--no-skills` | 创建一个**空** profile,不启用任何内置 skill。会在 profile 目录中写入 `.no-bundled-skills` 标记,使后续 `hermes update` 不再重新植入内置 skill 集,且拒绝与 `--clone``--clone-from` `--clone-all` 组合使用(因为这些选项会复制 skill)。适用于不应继承完整 skill 目录的窄化编排器 profile 或沙箱 profile。 |
创建 profile **不会**将该 profile 目录设为终端命令的默认项目/工作目录。如需让某个 profile 从特定项目目录启动,请在该 profile 的 `config.yaml` 中设置 `terminal.cwd`
@@ -101,7 +101,10 @@ hermes profile create work --clone
hermes profile create backup --clone-all
# 从指定 profile 克隆 config
hermes profile create work2 --clone --clone-from work
hermes profile create work2 --clone-from work
# 从指定 profile 克隆所有内容
hermes profile create work2-backup --clone-from work --clone-all
```
## `hermes profile describe`
@@ -46,7 +46,7 @@ hermes profile create researcher --description "Reads source code and external d
hermes profile create work --clone
```
将当前 profile 的 `config.yaml``.env``SOUL.md` 复制到新 profile。API 密钥模型相同,但会话和记忆是全新的。编辑 `~/.hermes/profiles/work/.env` 可使用不同的 API 密钥,编辑 `~/.hermes/profiles/work/SOUL.md` 可设置不同的人格。
将当前 profile 的 `config.yaml``.env``SOUL.md` 和 skills 复制到新 profile。API 密钥模型和能力相同,但会话和记忆是全新的。编辑 `~/.hermes/profiles/work/.env` 可使用不同的 API 密钥,编辑 `~/.hermes/profiles/work/SOUL.md` 可设置不同的人格。
### 克隆全部内容(`--clone-all`
@@ -54,16 +54,22 @@ hermes profile create work --clone
hermes profile create backup --clone-all
```
复制**所有内容**——配置、API 密钥、人格、所有记忆、完整会话历史、技能、cron 任务、插件。完整快照。适用于备份或 fork 已有上下文的 agent
复制**所有内容**——配置、API 密钥、人格、记忆、技能、cron 任务、插件。会排除每个 profile 自己的历史数据(会话历史、`state.db``backups/``state-snapshots/``checkpoints/`),这些数据属于源 profile 且可能达到数十 GB。若要包含历史的完整备份,请使用 `hermes profile export``hermes backup`
### 从指定 profile 克隆
```bash
hermes profile create work --clone --clone-from coder
hermes profile create work --clone-from coder
```
`--clone-from <source>` 会直接选择源 profile,并隐含执行 config/skills/SOUL 克隆。若要完整复制该源 profile,请与 `--clone-all` 组合使用:
```bash
hermes profile create work-backup --clone-from coder --clone-all
```
:::tip Honcho 记忆 + profiles
启用 Honcho 后,`--clone` 会自动为新 profile 创建专属 AI 对等体,同时共享同一用户工作区。每个 profile 构建各自的观察记录和身份标识。详见 [Honcho——多 agent / Profiles](./features/memory-providers.md#honcho)。
启用 Honcho 后,克隆操作会自动为新 profile 创建专属 AI 对等体,同时共享同一用户工作区。每个 profile 构建各自的观察记录和身份标识。详见 [Honcho——多 agent / Profiles](./features/memory-providers.md#honcho)。
:::
## 使用 profile