Merge main into bb/gui.
Resolve merge conflicts while preserving bb/gui dashboard paths and STT provider support.
This commit is contained in:
Brooklyn Nicholson
@@ -813,12 +813,16 @@ Every model slot in Hermes — auxiliary tasks, compression, fallback — uses t
|
||||
|
||||
When `base_url` is set, Hermes ignores the provider and calls that endpoint directly (using `api_key` or `OPENAI_API_KEY` for auth). When only `provider` is set, Hermes uses that provider's built-in auth and base URL.
|
||||
|
||||
Available providers for auxiliary tasks: `auto`, `main`, plus any provider in the [provider registry](/docs/reference/environment-variables) — `openrouter`, `nous`, `openai-codex`, `copilot`, `copilot-acp`, `anthropic`, `gemini`, `google-gemini-cli`, `qwen-oauth`, `zai`, `kimi-coding`, `kimi-coding-cn`, `minimax`, `minimax-cn`, `minimax-oauth`, `deepseek`, `nvidia`, `xai`, `ollama-cloud`, `alibaba`, `bedrock`, `huggingface`, `arcee`, `xiaomi`, `kilocode`, `opencode-zen`, `opencode-go`, `ai-gateway`, `azure-foundry` — or any named custom provider from your `custom_providers` list (e.g. `provider: "beans"`).
|
||||
Available providers for auxiliary tasks: `auto`, `main`, plus any provider in the [provider registry](/docs/reference/environment-variables) — `openrouter`, `nous`, `openai-codex`, `copilot`, `copilot-acp`, `anthropic`, `gemini`, `google-gemini-cli`, `qwen-oauth`, `zai`, `kimi-coding`, `kimi-coding-cn`, `minimax`, `minimax-cn`, `minimax-oauth`, `deepseek`, `nvidia`, `xai`, `xai-oauth`, `ollama-cloud`, `alibaba`, `bedrock`, `huggingface`, `arcee`, `xiaomi`, `kilocode`, `opencode-zen`, `opencode-go`, `ai-gateway`, `azure-foundry` — or any named custom provider from your `custom_providers` list (e.g. `provider: "beans"`).
|
||||
|
||||
:::tip MiniMax OAuth
|
||||
`minimax-oauth` logs in via browser OAuth (no API key needed). Run `hermes model` and select **MiniMax (OAuth)** to authenticate. Auxiliary tasks use `MiniMax-M2.7-highspeed` automatically. See the [MiniMax OAuth guide](../guides/minimax-oauth.md).
|
||||
:::
|
||||
|
||||
:::tip xAI Grok OAuth
|
||||
`xai-oauth` logs in via browser OAuth for SuperGrok subscribers (no API key needed). Run `hermes model` and select **xAI Grok OAuth (SuperGrok Subscription)** to authenticate. The same OAuth token is reused for every direct-to-xAI surface (chat, auxiliary tasks, TTS, image gen, video gen, transcription). See the [xAI Grok OAuth guide](../guides/xai-grok-oauth.md).
|
||||
:::
|
||||
|
||||
:::warning `"main"` is for auxiliary tasks only
|
||||
The `"main"` provider option means "use whatever provider my main agent uses" — it's only valid inside `auxiliary:`, `compression:`, and `fallback_model:` configs. It is **not** a valid value for your top-level `model.provider` setting. If you use a custom OpenAI-compatible endpoint, set `provider: custom` in your `model:` section. See [AI Providers](/docs/integrations/providers) for all main model provider options.
|
||||
:::
|
||||
@@ -980,6 +984,7 @@ These options apply to **auxiliary task configs** (`auxiliary:`, `compression:`,
|
||||
| `"nous"` | Force Nous Portal | `hermes auth` |
|
||||
| `"codex"` | Force Codex OAuth (ChatGPT account). Supports vision (gpt-5.3-codex). | `hermes model` → Codex |
|
||||
| `"minimax-oauth"` | Force MiniMax OAuth (browser login, no API key). Uses MiniMax-M2.7-highspeed for auxiliary tasks. | `hermes model` → MiniMax (OAuth) |
|
||||
| `"xai-oauth"` | Force xAI Grok OAuth (browser login for SuperGrok subscribers, no API key). Same OAuth token covers chat, TTS, image, video, and transcription. | `hermes model` → xAI Grok OAuth (SuperGrok Subscription) |
|
||||
| `"main"` | Use your active custom/main endpoint. This can come from `OPENAI_BASE_URL` + `OPENAI_API_KEY` or from a custom endpoint saved via `hermes model` / `config.yaml`. Works with OpenAI, local models, or any OpenAI-compatible API. **Auxiliary tasks only — not valid for `model.provider`.** | Custom endpoint credentials + base URL |
|
||||
|
||||
Direct API-key providers from the main provider catalog also work here when you want side tasks to bypass your default router. `gmi` is valid once `GMI_API_KEY` is configured:
|
||||
@@ -1588,7 +1593,7 @@ security:
|
||||
```
|
||||
|
||||
- `redact_secrets` — when `true`, automatically detects and redacts patterns that look like API keys, tokens, and passwords in tool output before it enters the conversation context and logs. **Off by default** — enable if you commonly work with real credentials in tool output and want a safety net. Set to `true` explicitly to turn on.
|
||||
- `tirith_enabled` — when `true`, terminal commands are scanned by [Tirith](https://github.com/StackGuardian/tirith) before execution to detect potentially dangerous operations.
|
||||
- `tirith_enabled` — when `true`, terminal commands are scanned by [Tirith](https://github.com/sheeki03/tirith) before execution to detect potentially dangerous operations.
|
||||
- `tirith_path` — path to the tirith binary. Set this if tirith is installed in a non-standard location.
|
||||
- `tirith_timeout` — maximum seconds to wait for a tirith scan. Commands proceed if the scan times out.
|
||||
- `tirith_fail_open` — when `true` (default), commands are allowed to execute if tirith is unavailable or fails. Set to `false` to block commands when tirith cannot verify them.
|
||||
|
||||
@@ -45,6 +45,14 @@ This installs the `agent-client-protocol` dependency and enables:
|
||||
- `hermes-acp`
|
||||
- `python -m acp_adapter`
|
||||
|
||||
For Zed registry installs, Zed launches Hermes through the official ACP Registry entry. That entry uses a `uvx` distribution that runs:
|
||||
|
||||
```bash
|
||||
uvx --from 'hermes-agent[acp]==<version>' hermes-acp
|
||||
```
|
||||
|
||||
Make sure `uv` is available on `PATH` before using the registry install path.
|
||||
|
||||
## Launching the ACP server
|
||||
|
||||
Any of the following starts Hermes in ACP mode:
|
||||
@@ -63,6 +71,34 @@ python -m acp_adapter
|
||||
|
||||
Hermes logs to stderr so stdout remains reserved for ACP JSON-RPC traffic.
|
||||
|
||||
For non-interactive checks:
|
||||
|
||||
```bash
|
||||
hermes acp --version
|
||||
hermes acp --check
|
||||
```
|
||||
|
||||
### Browser tools (optional)
|
||||
|
||||
Browser tools (`browser_navigate`, `browser_click`, etc.) depend on the
|
||||
`agent-browser` npm package and Chromium, which aren't part of the Python
|
||||
wheel. Install them with:
|
||||
|
||||
```bash
|
||||
hermes acp --setup-browser # interactive (prompts before ~400 MB download)
|
||||
hermes acp --setup-browser --yes # accept the download non-interactively
|
||||
```
|
||||
|
||||
This is the standalone command. The Zed registry's terminal-auth flow (`hermes acp --setup`) also offers the browser bootstrap as a follow-up question after model selection, so most users never need to run `--setup-browser` directly.
|
||||
|
||||
What it does:
|
||||
|
||||
- Installs Node.js 22 LTS into `~/.hermes/node/` if missing
|
||||
- `npm install -g agent-browser @askjo/camofox-browser` into that prefix (no sudo needed — `npm`'s `--prefix` points at the user-writable Hermes-managed Node)
|
||||
- Installs Playwright Chromium, or uses a detected system Chrome/Chromium when available
|
||||
|
||||
The bootstrap is idempotent — re-running it is fast and skips work that's already done.
|
||||
|
||||
## Editor setup
|
||||
|
||||
### VS Code
|
||||
@@ -90,7 +126,19 @@ If you want to define Hermes manually, add it through VS Code settings under `ac
|
||||
|
||||
### Zed
|
||||
|
||||
Example settings snippet:
|
||||
Zed v0.221.x and newer installs external agents through the official ACP Registry.
|
||||
|
||||
1. Open the Agent Panel.
|
||||
2. Click **Add Agent**, or run the `zed: acp registry` command.
|
||||
3. Search for **Hermes Agent**.
|
||||
4. Install it and start a new Hermes external-agent thread.
|
||||
|
||||
Prerequisites:
|
||||
|
||||
- Configure Hermes provider credentials first with `hermes model`, or set them in `~/.hermes/.env` / `~/.hermes/config.yaml`.
|
||||
- Install `uv` so the registry launcher can run `uvx --from 'hermes-agent[acp]==<version>' hermes-acp`.
|
||||
|
||||
For local development before the registry entry is available, use a custom agent server in Zed settings:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -98,9 +146,9 @@ Example settings snippet:
|
||||
"hermes-agent": {
|
||||
"type": "custom",
|
||||
"command": "hermes",
|
||||
"args": ["acp"],
|
||||
},
|
||||
},
|
||||
"args": ["acp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
@@ -114,18 +162,23 @@ Use an ACP-compatible plugin and point it at:
|
||||
|
||||
## Registry manifest
|
||||
|
||||
The ACP registry manifest lives at:
|
||||
The source copy of Hermes' official ACP Registry metadata lives at:
|
||||
|
||||
```text
|
||||
acp_registry/agent.json
|
||||
acp_registry/icon.svg
|
||||
```
|
||||
|
||||
It advertises a command-based agent whose launch command is:
|
||||
The upstream registry PR copies those files into the top-level `hermes-agent/` directory in `agentclientprotocol/registry`.
|
||||
|
||||
The registry entry uses a `uvx` distribution that points directly at the `hermes-agent` PyPI release:
|
||||
|
||||
```text
|
||||
hermes acp
|
||||
uvx --from 'hermes-agent[acp]==<version>' hermes-acp
|
||||
```
|
||||
|
||||
The registry CI verifies that the pinned version exists on PyPI, so the manifest's `version` and uvx `package` pin must always match `pyproject.toml`. `scripts/release.py` keeps them in lockstep automatically.
|
||||
|
||||
## Configuration and credentials
|
||||
|
||||
ACP mode uses the same Hermes configuration as the CLI:
|
||||
@@ -135,7 +188,7 @@ ACP mode uses the same Hermes configuration as the CLI:
|
||||
- `~/.hermes/skills/`
|
||||
- `~/.hermes/state.db`
|
||||
|
||||
Provider resolution uses Hermes' normal runtime resolver, so ACP inherits the currently configured provider and credentials.
|
||||
Provider resolution uses Hermes' normal runtime resolver, so ACP inherits the currently configured provider and credentials. Hermes also advertises a terminal auth method (`--setup`) for first-run registry clients; this opens Hermes' interactive model/provider setup.
|
||||
|
||||
## Session behavior
|
||||
|
||||
@@ -171,29 +224,36 @@ On timeout or error, the approval bridge denies the request.
|
||||
|
||||
Check:
|
||||
|
||||
- the editor is pointed at the correct `acp_registry/` path
|
||||
- Hermes is installed and on your PATH
|
||||
- the ACP extra is installed (`pip install -e '.[acp]'`)
|
||||
- In Zed, open the ACP Registry with `zed: acp registry` and search for **Hermes Agent**.
|
||||
- For manual/local development, verify the custom `agent_servers` command points to `hermes acp`.
|
||||
- Hermes is installed and on your PATH.
|
||||
- The ACP extra is installed (`pip install -e '.[acp]'`).
|
||||
- `uv` is installed if launching from the official Zed registry entry.
|
||||
|
||||
### ACP starts but immediately errors
|
||||
|
||||
Try these checks:
|
||||
|
||||
```bash
|
||||
hermes acp --version
|
||||
hermes acp --check
|
||||
hermes doctor
|
||||
hermes status
|
||||
hermes acp
|
||||
```
|
||||
|
||||
### Missing credentials
|
||||
|
||||
ACP mode does not have its own login flow. It uses Hermes' existing provider setup. Configure credentials with:
|
||||
ACP mode uses Hermes' existing provider setup. Configure credentials with:
|
||||
|
||||
```bash
|
||||
hermes model
|
||||
```
|
||||
|
||||
or by editing `~/.hermes/.env`.
|
||||
or by editing `~/.hermes/.env`. Registry clients can also trigger Hermes' terminal auth flow, which runs the same interactive provider/model setup.
|
||||
|
||||
### Zed registry launcher cannot find uv
|
||||
|
||||
Install `uv` from the official uv installation docs, then retry the Hermes Agent thread from Zed.
|
||||
|
||||
## See also
|
||||
|
||||
|
||||
@@ -368,6 +368,13 @@ BROWSERBASE_SESSION_TIMEOUT=600000
|
||||
|
||||
# Inactivity timeout before auto-cleanup in seconds (default: 120)
|
||||
BROWSER_INACTIVITY_TIMEOUT=120
|
||||
|
||||
# Extra Chromium launch flags (comma- or newline-separated). Hermes auto-injects
|
||||
# `--no-sandbox,--disable-dev-shm-usage` when it detects root or AppArmor-restricted
|
||||
# unprivileged user namespaces (Ubuntu 23.10+, DGX Spark, many container images),
|
||||
# so most users don't need to set this. Set it manually only if you need a flag
|
||||
# Hermes doesn't add automatically; setting it disables the auto-injection.
|
||||
AGENT_BROWSER_ARGS=--no-sandbox
|
||||
```
|
||||
|
||||
### Install agent-browser CLI
|
||||
|
||||
@@ -0,0 +1,444 @@
|
||||
---
|
||||
title: Codex App-Server Runtime (optional)
|
||||
sidebar_label: Codex App-Server Runtime
|
||||
---
|
||||
|
||||
# Codex App-Server Runtime
|
||||
|
||||
Hermes can optionally hand `openai/*` and `openai-codex/*` turns to the [Codex CLI app-server](https://github.com/openai/codex) instead of running its own tool loop. When enabled, terminal commands, file edits, sandboxing, and MCP tool calls all execute inside Codex's runtime — Hermes becomes the shell around it (sessions DB, slash commands, gateway, memory and skill review).
|
||||
|
||||
This is **opt-in only**. Default Hermes behavior is unchanged unless you flip the flag. Hermes never auto-routes you onto this runtime.
|
||||
|
||||
## Why
|
||||
|
||||
- Run OpenAI agent turns against your **ChatGPT subscription** (no API key required) using the same auth flow Codex CLI uses.
|
||||
- Use **Codex's own toolset and sandbox** — `shell` for terminal/read/write/search, `apply_patch` for structured edits, `update_plan` for planning, all running inside seatbelt/landlock sandboxing.
|
||||
- **Native Codex plugins** — Linear, GitHub, Gmail, Calendar, Canva, etc. — installed via `codex plugin` are auto-migrated and active in your Hermes session.
|
||||
- **Hermes' richer tools come along** — web_search, web_extract, browser automation, vision, image generation, skills, and TTS work via an MCP callback. Codex calls back into Hermes for tools it doesn't have built in.
|
||||
- **Memory and skill nudges keep working** — Codex's events are projected into Hermes' message shape so the self-improvement loop sees a normal-looking transcript.
|
||||
|
||||
## What tools the model actually has
|
||||
|
||||
This is the part most users want to know up front. When this runtime is on, the model running your turn has three independent sources of tools:
|
||||
|
||||
### 1. Codex's built-in toolset (always on)
|
||||
|
||||
These ship with `codex app-server` itself — no Hermes involvement, no MCP, no plugins. All five are available the moment the runtime starts:
|
||||
|
||||
- **`shell`** — runs arbitrary shell commands inside the sandbox. This is how the model reads files (`cat`, `head`, `tail`), writes them (`echo > foo`, heredocs), searches them (`find`, `rg`, `grep`), navigates directories (`ls`, `cd`), runs builds, manages processes, and anything else you'd do in bash.
|
||||
- **`apply_patch`** — applies a structured multi-file diff in Codex's patch format. The model uses this for non-trivial code edits (adding a function, refactoring across files); shell heredocs are still available for one-off writes.
|
||||
- **`update_plan`** — codex's internal todo / plan tracker. Equivalent of Hermes' `todo` tool, but managed entirely inside codex's runtime.
|
||||
- **`view_image`** — load a local image file into the conversation so the model can see it.
|
||||
- **`web_search`** — codex has its own built-in web search when configured. Hermes also exposes `web_search` (Firecrawl-backed) via the callback below; the model picks whichever it prefers.
|
||||
|
||||
So **anything you'd do via terminal — read/write/search/find/run — codex does natively**. The sandbox profile (`:workspace` by default when you enable the runtime) controls what's writable.
|
||||
|
||||
### 2. Native Codex plugins (auto-migrated from your `codex plugin` install)
|
||||
|
||||
When you enable the runtime, Hermes queries codex's `plugin/list` RPC and writes a `[plugins."<name>@openai-curated"]` entry for every plugin you have installed. The plugins themselves are managed by codex and authorized once via codex's own UI.
|
||||
|
||||
Examples (the ones the OpenClaw thread highlighted as "YouTube-video-worthy"):
|
||||
|
||||
- **Linear** — find/update issues
|
||||
- **GitHub** — search code, view PRs, comment
|
||||
- **Gmail** — read/send mail
|
||||
- **Google Calendar** — create/find events
|
||||
- **Outlook calendar/email** — same shape via the Microsoft connector
|
||||
- **Canva** — design generation
|
||||
- ...whatever else you've installed via `codex plugin marketplace add openai-curated` + `codex plugin install ...`
|
||||
|
||||
What's NOT migrated:
|
||||
- Plugins you haven't installed yet — install them in Codex first.
|
||||
- ChatGPT app marketplace entries (`app/list`) — these are already enabled inside codex by virtue of your account auth.
|
||||
|
||||
### 3. Hermes tool callback (MCP server, registered in `~/.codex/config.toml`)
|
||||
|
||||
Hermes registers itself as an MCP server so codex can call back for tools codex doesn't ship with. Available via the callback:
|
||||
|
||||
- **`web_search`** / **`web_extract`** — Firecrawl-backed; tends to be cleaner than scraping for structured content.
|
||||
- **`browser_navigate` / `browser_click` / `browser_type` / `browser_press` / `browser_snapshot` / `browser_scroll` / `browser_back` / `browser_get_images` / `browser_console` / `browser_vision`** — full browser automation via Camofox or Browserbase.
|
||||
- **`vision_analyze`** — call a separate vision model to inspect an image (different from codex's `view_image` which loads it into the conversation).
|
||||
- **`image_generate`** — image generation through Hermes' image_gen plugin chain.
|
||||
- **`skill_view` / `skills_list`** — read from Hermes' skill library.
|
||||
- **`text_to_speech`** — TTS through Hermes' configured provider.
|
||||
|
||||
When the model wants one of these, codex spawns the `hermes_tools_mcp_server` subprocess via stdio MCP, the call is dispatched through `model_tools.handle_function_call()` (same code path as Hermes' default runtime), and the result is returned to codex like any other MCP response.
|
||||
|
||||
### What's NOT available on this runtime
|
||||
|
||||
These four Hermes tools require the running AIAgent context (mid-loop state) to dispatch, and a stateless MCP callback can't drive them. Switch back to the default runtime (`/codex-runtime auto`) when you need any of them:
|
||||
|
||||
- **`delegate_task`** — spawn subagents
|
||||
- **`memory`** — Hermes' persistent memory store
|
||||
- **`session_search`** — cross-session search
|
||||
- **`todo`** — Hermes' todo store (codex's `update_plan` is the in-runtime equivalent)
|
||||
|
||||
## Workflow features (`/goal`, kanban, cron)
|
||||
|
||||
### `/goal` (the Ralph loop)
|
||||
|
||||
**Works on this runtime.** Goals persist in `state_meta` keyed by session id, the continuation prompt feeds back as a normal user message through `run_conversation()`, and codex executes the next turn natively. The goal judge runs via the auxiliary client (configured via `auxiliary.goal_judge` in config.yaml), independent of which runtime is active. The judge's "blocked, needs user input" verdict is a clean escape if codex stalls on approvals.
|
||||
|
||||
**One thing to be aware of:** each continuation prompt is a fresh codex turn, which means codex re-evaluates command approval policy from scratch. If you're doing a long-running goal with lots of writes, expect more approval prompts than you'd see on a single in-session task. Set `default_permissions = ":workspace"` (which Hermes does automatically when you enable the runtime) so simple workspace writes don't require prompting.
|
||||
|
||||
### Kanban (multi-agent worktree dispatch)
|
||||
|
||||
**Works on this runtime, with one subtle dependency.** The kanban dispatcher spawns each worker as a separate `hermes chat -q` subprocess that reads the user's config — which means if `model.openai_runtime: codex_app_server` is set globally, workers also come up on the codex runtime.
|
||||
|
||||
What works inside a codex-runtime worker:
|
||||
- Codex's full toolset (shell, apply_patch, update_plan, view_image, web_search) — the worker does its actual task work natively
|
||||
- The migrated codex plugins — Linear, GitHub, etc.
|
||||
- The Hermes tool callback for browser_*, vision, image_gen, skills, TTS
|
||||
|
||||
What also works because the MCP callback exposes them:
|
||||
- **`kanban_complete` / `kanban_block` / `kanban_comment` / `kanban_heartbeat`** — the worker handoff tools. These read `HERMES_KANBAN_TASK` from env (set by the dispatcher), gate access correctly, and write to `~/.hermes/kanban.db`. Without these in the callback, a worker on this runtime could do its task but couldn't report back, hanging until the dispatcher's timeout.
|
||||
- **`kanban_show` / `kanban_list`** — read-only board queries for the worker to check its own context.
|
||||
- **`kanban_create` / `kanban_unblock` / `kanban_link`** — orchestrator-only operations. Available for orchestrator agents running on the codex runtime that need to dispatch new tasks.
|
||||
|
||||
The kanban tools are gated by `HERMES_KANBAN_TASK` env var the dispatcher sets — that var is propagated to the codex subprocess (codex inherits env) and from there to the spawned `hermes-tools` MCP server subprocess. So the tools see the right task id and gate correctly.
|
||||
|
||||
### Cron jobs
|
||||
|
||||
**Not specifically tested.** Cron jobs run via `cronjob` → `AIAgent.run_conversation`, the same code path as the CLI. If the cron job's config has `openai_runtime: codex_app_server` it'll run on codex. The same tool-availability rules apply — codex built-ins + plugins + MCP callback work, agent-loop tools (delegate_task, memory, session_search, todo) don't. If your cron job relies on those, scope the cron to a profile that uses the default runtime.
|
||||
|
||||
## Trade-offs
|
||||
|
||||
| | Hermes default runtime | Codex app-server (opt-in) |
|
||||
|---|---|---|
|
||||
| `delegate_task` subagents | yes | not available — needs agent loop context |
|
||||
| `memory`, `session_search`, `todo` | yes | not available — needs agent loop context |
|
||||
| `web_search`, `web_extract` | yes | yes (via MCP callback) |
|
||||
| Browser automation (Camofox/Browserbase) | yes | yes (via MCP callback) |
|
||||
| `vision_analyze`, `image_generate` | yes | yes (via MCP callback) |
|
||||
| `skill_view`, `skills_list` | yes | yes (via MCP callback) |
|
||||
| `text_to_speech` | yes | yes (via MCP callback) |
|
||||
| Codex `shell` (terminal/read/write/search/find/run) | — | yes (Codex built-in) |
|
||||
| Codex `apply_patch` (structured multi-file edits) | — | yes (Codex built-in) |
|
||||
| Codex `update_plan` (in-runtime todo) | — | yes (Codex built-in) |
|
||||
| Codex `view_image` (load image into conversation) | — | yes (Codex built-in) |
|
||||
| Codex sandbox (seatbelt/landlock, profiles) | — | yes (Codex built-in) |
|
||||
| ChatGPT subscription auth | — | yes (via `openai-codex` provider) |
|
||||
| Native Codex plugins (Linear, GitHub, etc.) | — | yes (auto-migrated) |
|
||||
| User MCP servers | yes | yes (auto-migrated to codex) |
|
||||
| Memory + skill review (background) | yes | yes (via item projection) |
|
||||
| Multi-turn conversations | yes | yes |
|
||||
| `/goal` (Ralph loop) | yes | yes |
|
||||
| Kanban worker dispatch | yes | yes (via callback) |
|
||||
| Kanban orchestrator tools | yes | yes (via callback) |
|
||||
| All gateway platforms | yes | yes |
|
||||
| Non-OpenAI providers | yes | n/a — OpenAI/Codex-scoped |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **Codex CLI installed:**
|
||||
```bash
|
||||
npm i -g @openai/codex
|
||||
codex --version # 0.130.0 or newer
|
||||
```
|
||||
2. **Codex OAuth login.** The codex subprocess reads `~/.codex/auth.json`. Two ways to populate it:
|
||||
```bash
|
||||
codex login # writes tokens to ~/.codex/auth.json
|
||||
```
|
||||
Hermes' own `hermes auth login codex` writes to `~/.hermes/auth.json` — that's a separate session. **Run `codex login` separately** if you haven't.
|
||||
|
||||
3. **(Optional) Install the Codex plugins you want.** When you enable the runtime, Hermes auto-migrates whichever curated plugins you've already installed via Codex CLI:
|
||||
```bash
|
||||
codex plugin marketplace add openai-curated
|
||||
# then via codex's TUI, install Linear / GitHub / Gmail / etc.
|
||||
```
|
||||
Hermes will discover them and write `[plugins."<name>@openai-curated"]` entries to `~/.codex/config.toml` automatically.
|
||||
|
||||
## Enabling
|
||||
|
||||
In a Hermes session:
|
||||
|
||||
```
|
||||
/codex-runtime codex_app_server
|
||||
```
|
||||
|
||||
That command:
|
||||
- Verifies the `codex` CLI is installed (blocks with an install hint if not).
|
||||
- Persists `model.openai_runtime: codex_app_server` to your config.yaml.
|
||||
- Migrates user MCP servers from `~/.hermes/config.yaml` to `~/.codex/config.toml`.
|
||||
- **Discovers and migrates installed native Codex plugins** (Linear, GitHub, Gmail, Calendar, Canva, etc.) by querying Codex's `plugin/list` RPC.
|
||||
- **Registers Hermes' own tools as an MCP server** so the codex subprocess can call back for tools codex doesn't ship with.
|
||||
- **Writes `default_permissions = ":workspace"`** so the sandbox allows writes within the workspace without prompting for every operation.
|
||||
- Tells you what was migrated. Takes effect on the **next** session — the current cached agent keeps the prior runtime so prompt caches stay valid.
|
||||
|
||||
Synonyms: `/codex-runtime on`, `/codex-runtime off`, `/codex-runtime auto`.
|
||||
|
||||
To check current state without changing anything:
|
||||
```
|
||||
/codex-runtime
|
||||
```
|
||||
|
||||
You can also set it manually in `~/.hermes/config.yaml`:
|
||||
```yaml
|
||||
model:
|
||||
openai_runtime: codex_app_server # default is "auto" (= Hermes runtime)
|
||||
```
|
||||
|
||||
## Self-improvement loop (memory + skill nudges)
|
||||
|
||||
Hermes' background self-improvement fires on counter thresholds:
|
||||
|
||||
- Every 10 user prompts → a forked review agent looks at the conversation and decides whether anything should be saved to memory.
|
||||
- Every 10 tool iterations within a single turn → same idea but for skills (`skill_manage` writes).
|
||||
|
||||
**Both keep working on the codex runtime.** The codex path projects each completed `commandExecution` / `fileChange` / `mcpToolCall` / `dynamicToolCall` item into a synthetic `assistant tool_call` + `tool` result message, so by the time the review runs it sees the same shape it sees on the default Hermes runtime.
|
||||
|
||||
How the wiring stays equivalent:
|
||||
|
||||
| | Default runtime | Codex runtime |
|
||||
|---|---|---|
|
||||
| `_turns_since_memory` increments | per user prompt, in run_conversation pre-loop | same code path, before the early-return |
|
||||
| `_iters_since_skill` increments | per tool iteration in the chat-completions loop | by `turn.tool_iterations` after the codex turn returns |
|
||||
| Memory trigger (`_turns_since_memory >= _memory_nudge_interval`) | computed in pre-loop, fires after response | computed in pre-loop, passed through to codex helper |
|
||||
| Skill trigger (`_iters_since_skill >= _skill_nudge_interval`) | computed after the loop | computed after the codex turn |
|
||||
| `_spawn_background_review(messages_snapshot=..., review_memory=..., review_skills=...)` | called when either trigger fires | called identically when either trigger fires |
|
||||
|
||||
One detail: the review fork itself needs to call Hermes' agent-loop tools (`memory`, `skill_manage`), which require Hermes' own dispatch. So when the parent agent is on `codex_app_server`, the review fork is **downgraded to `codex_responses`** — same OAuth credentials, same `openai-codex` provider, but talks to OpenAI's Responses API directly so Hermes owns the loop and the agent-loop tools work. This is invisible to the user.
|
||||
|
||||
Net effect: enable the codex runtime and your memory + skill nudges keep firing exactly as they would otherwise.
|
||||
|
||||
## How approvals work
|
||||
|
||||
Codex requests approval before executing commands or applying patches. These get translated into Hermes' standard "Dangerous Command" prompt:
|
||||
|
||||
```
|
||||
╭───────────────────────────────────────╮
|
||||
│ Dangerous Command │
|
||||
│ │
|
||||
│ /bin/bash -lc 'echo hello > foo.txt' │
|
||||
│ │
|
||||
│ ❯ 1. Allow once │
|
||||
│ 2. Allow for this session │
|
||||
│ 3. Deny │
|
||||
│ │
|
||||
│ Codex requests exec in /your/cwd │
|
||||
╰───────────────────────────────────────╯
|
||||
```
|
||||
|
||||
- **Allow once** → approve this single command.
|
||||
- **Allow for this session** → Codex won't re-prompt for similar commands.
|
||||
- **Deny** → command is rejected; Codex continues in read-only mode.
|
||||
|
||||
For `apply_patch` (file edit) approvals, Hermes shows a summary of what changed (`1 add, 1 update: /tmp/new.py, /tmp/old.py`) when codex provides the data via the corresponding `fileChange` item.
|
||||
|
||||
## Permission profiles
|
||||
|
||||
Codex has three built-in permission profiles:
|
||||
- `:read-only` — no writes; every shell command requires approval
|
||||
- `:workspace` — writes within the current workspace allowed without prompts (Hermes' default when you enable the runtime)
|
||||
- `:danger-no-sandbox` — no sandbox at all (don't use this unless you understand it)
|
||||
|
||||
You can override the default in `~/.codex/config.toml` outside Hermes' managed block:
|
||||
|
||||
```toml
|
||||
default_permissions = ":read-only"
|
||||
```
|
||||
|
||||
(Hermes will preserve your override on re-migration as long as it lives outside the `# managed by hermes-agent` markers.)
|
||||
|
||||
## Auxiliary tasks and ChatGPT subscription token cost
|
||||
|
||||
When this runtime is on with the `openai-codex` provider, **auxiliary tasks (title generation, context compression, vision auto-detect, session search summarization, the background self-improvement review fork) also flow through your ChatGPT subscription by default**, because Hermes' auxiliary client uses the main provider/model when no per-task override is set.
|
||||
|
||||
This isn't specific to `codex_app_server` — it's true for the existing `codex_responses` path too — but it's more visible here because you're explicitly opting in for the subscription billing.
|
||||
|
||||
To route specific aux tasks to a cheaper / different model, set explicit overrides in `~/.hermes/config.yaml`:
|
||||
|
||||
```yaml
|
||||
auxiliary:
|
||||
title_generation:
|
||||
provider: openrouter
|
||||
model: google/gemini-3-flash-preview
|
||||
context_compression:
|
||||
provider: openrouter
|
||||
model: google/gemini-3-flash-preview
|
||||
vision_detect:
|
||||
provider: openrouter
|
||||
model: google/gemini-3-flash-preview
|
||||
session_search:
|
||||
provider: openrouter
|
||||
model: google/gemini-3-flash-preview
|
||||
goal_judge:
|
||||
provider: openrouter
|
||||
model: google/gemini-3-flash-preview
|
||||
```
|
||||
|
||||
The self-improvement review fork inherits the main runtime via `_current_main_runtime()` and Hermes downgrades it from `codex_app_server` to `codex_responses` automatically (so the fork can actually call `memory` and `skill_manage` — Hermes' own agent-loop tools). That fork still uses your subscription auth unless you've routed aux tasks elsewhere.
|
||||
|
||||
## Editing `~/.codex/config.toml` safely
|
||||
|
||||
Hermes wraps everything it manages between two marker comments:
|
||||
|
||||
```toml
|
||||
# managed by hermes-agent — `hermes codex-runtime migrate` regenerates this section
|
||||
default_permissions = ":workspace"
|
||||
[mcp_servers.filesystem]
|
||||
...
|
||||
[plugins."github@openai-curated"]
|
||||
...
|
||||
# end hermes-agent managed section
|
||||
```
|
||||
|
||||
Anything **outside** that block is yours. Re-running migration (via `/codex-runtime codex_app_server` or whenever you toggle the runtime on) replaces the managed block in place but preserves user content above and below it verbatim. This means you can:
|
||||
|
||||
- Add your own MCP servers Hermes doesn't know about
|
||||
- Override `default_permissions` to `:read-only` if you prefer to be prompted
|
||||
- Configure codex-only options (model, providers, otel, etc.)
|
||||
- Add user-defined permission profiles in `[permissions.<name>]` tables
|
||||
|
||||
Anything you add **inside** the managed block will get clobbered on the next migration. If you need a tweak that requires editing the managed block, file an issue and we'll add the knob.
|
||||
|
||||
## Multi-profile / multi-tenant setups
|
||||
|
||||
By default, Hermes points the codex subprocess at `~/.codex/` regardless of which Hermes profile is active. This means `hermes -p work` and `hermes -p personal` share the same Codex auth, plugins, and config. For most users this is the right behavior — it matches what running `codex` CLI directly would do.
|
||||
|
||||
If you want per-profile Codex isolation (separate auth, separate installed plugins, separate config), set `CODEX_HOME` explicitly per profile. The cleanest way is to point at a directory under your `HERMES_HOME`:
|
||||
|
||||
```bash
|
||||
# Inside the work profile, you might wrap hermes:
|
||||
CODEX_HOME=~/.hermes/profiles/work/codex hermes chat
|
||||
```
|
||||
|
||||
You'll need to re-run `codex login` once with that `CODEX_HOME` set so the OAuth tokens land in the profile-scoped location. After that, `hermes -p work` will operate on isolated Codex state.
|
||||
|
||||
We don't auto-scope this because moving an existing user's `~/.codex/` would silently invalidate their Codex CLI auth — anyone who already ran `codex login` would have to re-authenticate. Opt-in feels safer than surprising users.
|
||||
|
||||
## HOME environment variable passthrough
|
||||
|
||||
Hermes does NOT rewrite `HOME` when spawning the codex app-server subprocess (we use `os.environ.copy()` and only overlay `CODEX_HOME` and `RUST_LOG`). This means:
|
||||
|
||||
- Commands codex runs via its `shell` tool see the real user `HOME` and find `~/.gitconfig`, `~/.gh/`, `~/.aws/`, `~/.npmrc`, etc. correctly.
|
||||
- Codex's internal state stays isolated through `CODEX_HOME` (which points at `~/.codex/` by default).
|
||||
|
||||
This matches the boundary OpenClaw arrived at after some early experimentation: isolate Codex's state, leave the user's home alone. (Cf. openclaw/openclaw#81562.)
|
||||
|
||||
## MCP server migration
|
||||
|
||||
Hermes' `mcp_servers` config is auto-translated to the TOML format Codex expects. The migration runs every time you enable the runtime and is idempotent — re-runs replace the managed section but preserve any user-edited Codex config.
|
||||
|
||||
What translates:
|
||||
|
||||
| Hermes (`config.yaml`) | Codex (`config.toml`) |
|
||||
|---|---|
|
||||
| `command` + `args` + `env` | stdio transport |
|
||||
| `url` + `headers` | streamable_http transport |
|
||||
| `timeout` | `tool_timeout_sec` |
|
||||
| `connect_timeout` | `startup_timeout_sec` |
|
||||
| `enabled: false` | `enabled = false` |
|
||||
|
||||
What's not migrated:
|
||||
- Hermes-specific keys like `sampling` (Codex's MCP client has no equivalent — these are dropped with a per-server warning).
|
||||
|
||||
## Native Codex plugin migration
|
||||
|
||||
Plugins installed via `codex plugin` (Linear, GitHub, Gmail, Calendar, Canva, etc.) are discovered through Codex's `plugin/list` RPC. For each plugin where `installed: true`, Hermes writes a `[plugins."<name>@openai-curated"]` block enabling it in your Hermes session.
|
||||
|
||||
This means: when your friend says "I have Calendar and GitHub set up in my Codex CLI" and they enable Hermes' codex runtime, Hermes activates those automatically. No re-configuration needed.
|
||||
|
||||
What's NOT migrated:
|
||||
- Plugins you haven't installed yet — install them in Codex first.
|
||||
- Plugins where codex reports `availability != AVAILABLE` (broken install, expired OAuth, removed from marketplace, etc.). These are skipped to avoid writing config that would fail at activation time.
|
||||
- ChatGPT app marketplace entries (the per-account `app/list` results — these are already enabled inside codex by virtue of your account auth).
|
||||
- Plugin OAuth — you authorize each plugin once in Codex itself; Hermes doesn't touch credentials.
|
||||
|
||||
## Hermes tool callback (the new MCP server)
|
||||
|
||||
Codex's built-in toolset covers shell/file ops/patches but doesn't have web search, browser automation, vision, image generation, etc. To keep those usable in a codex turn, Hermes registers itself as an MCP server in `~/.codex/config.toml`:
|
||||
|
||||
```toml
|
||||
[mcp_servers.hermes-tools]
|
||||
command = "/path/to/python"
|
||||
args = ["-m", "agent.transports.hermes_tools_mcp_server"]
|
||||
env = { HERMES_HOME = "/your/.hermes", PYTHONPATH = "...", HERMES_QUIET = "1" }
|
||||
startup_timeout_sec = 30.0
|
||||
tool_timeout_sec = 600.0
|
||||
```
|
||||
|
||||
When the model calls `web_search` (or another exposed Hermes tool), codex spawns the `hermes_tools_mcp_server` subprocess via stdio, the request is dispatched through `model_tools.handle_function_call()`, and the result is projected back to codex like any other MCP response.
|
||||
|
||||
**Tools available via the callback:** `web_search`, `web_extract`, `browser_navigate`, `browser_click`, `browser_type`, `browser_press`, `browser_snapshot`, `browser_scroll`, `browser_back`, `browser_get_images`, `browser_console`, `browser_vision`, `vision_analyze`, `image_generate`, `skill_view`, `skills_list`, `text_to_speech`.
|
||||
|
||||
**Tools NOT available:** `delegate_task`, `memory`, `session_search`, `todo`. These need the running AIAgent context to dispatch (mid-loop state) and a stateless MCP callback can't drive them. Use the default Hermes runtime (`/codex-runtime auto`) when you need these.
|
||||
|
||||
## Disabling
|
||||
|
||||
Switch back at any time:
|
||||
|
||||
```
|
||||
/codex-runtime auto
|
||||
```
|
||||
|
||||
Effective on the next session. The Codex managed block stays in `~/.codex/config.toml` so you can re-enable later without losing config — or remove it manually if you prefer.
|
||||
|
||||
## Limitations
|
||||
|
||||
This runtime is **opt-in beta**. Working as of Hermes Agent 2026.5 + Codex CLI 0.130.0:
|
||||
|
||||
- Multi-turn conversations
|
||||
- `commandExecution` and `fileChange` (apply_patch) approvals via Hermes UI
|
||||
- MCP tool calls (verified against `@modelcontextprotocol/server-filesystem` and the new `hermes-tools` callback)
|
||||
- Native Codex plugin migration (verified against Linear / GitHub / Calendar inventory)
|
||||
- Deny/cancel paths
|
||||
- Toggle on/off cycle
|
||||
- Memory and skill nudge counters (verified live via integration tests)
|
||||
- Hermes web_search through codex (verified live: "OpenAI Codex CLI – Getting Started" returned end-to-end)
|
||||
|
||||
Known limitations:
|
||||
|
||||
- **Hermes auth and codex auth are separate sessions.** You need both `codex login` AND `hermes auth login codex` for the cleanest UX (the runtime uses codex's session for the LLM call). This is a deliberate design choice in Hermes' `_import_codex_cli_tokens` — Hermes won't share OAuth state with codex CLI to avoid clobbering each other on token refresh.
|
||||
- **`delegate_task`, `memory`, `session_search`, `todo` are unavailable on this runtime.** They need the running AIAgent context which a stateless MCP callback can't provide. Use `/codex-runtime auto` when you need these.
|
||||
- **No inline patch preview in approval prompts when codex doesn't track the changeset.** Codex's `fileChange` approval params don't always carry the changeset. Hermes caches the data from the corresponding `item/started` notification when possible, but if approval arrives before the item has streamed, the prompt falls back to whatever `reason` codex provides.
|
||||
- **Sub-second cancellation isn't guaranteed.** Mid-stream interrupts (Ctrl+C while codex is responding) are sent via `turn/interrupt`, but if codex has already flushed the final message, you get the response anyway.
|
||||
|
||||
If you find a bug, [open an issue](https://github.com/NousResearch/hermes-agent/issues) with the output of `hermes logs --since 5m`. Mention `codex-runtime` in the title so it's easy to triage.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─── Hermes shell (CLI / TUI / gateway) ───┐
|
||||
│ sessions DB · slash commands · memory │
|
||||
│ & skill review · cron · session pickers │
|
||||
└──┬──────────────────────────────────────┬┘
|
||||
│ user_message final │
|
||||
▼ text + │
|
||||
┌──────────────────────────────────┐ projected │
|
||||
│ AIAgent.run_conversation() │ messages │
|
||||
│ if api_mode == codex_app_server │ │
|
||||
│ → CodexAppServerSession │ │
|
||||
│ else: chat_completions / codex_responses (default)
|
||||
└────┬─────────────────────────────┘ │
|
||||
│ JSON-RPC over stdio │
|
||||
▼ │
|
||||
┌──────────────────────────────────┐ │
|
||||
│ codex app-server (subprocess) │──────────────┘
|
||||
│ thread/start, turn/start │
|
||||
│ item/* notifications │
|
||||
│ shell + apply_patch + update_plan│
|
||||
│ view_image + sandbox │
|
||||
│ ┌─────────────────────────┐ │
|
||||
│ │ MCP client │ │
|
||||
│ │ ├─ user MCP servers │ │
|
||||
│ │ ├─ native plugins │ │
|
||||
│ │ │ (linear, github, │ │
|
||||
│ │ │ gmail, calendar, │ │
|
||||
│ │ │ canva, ...) │ │
|
||||
│ │ └─ hermes-tools ───────┼─────────────────┐
|
||||
│ │ (callback to │ │ │
|
||||
│ │ Hermes' richer │ │ │
|
||||
│ │ tools) │ │ │
|
||||
│ └─────────────────────────┘ │ │
|
||||
└──────────────────────────────────┘ │
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────┐
|
||||
│ hermes_tools_mcp_server.py (subprocess on demand) │
|
||||
│ web_search, web_extract, browser_*, vision_analyze, │
|
||||
│ image_generate, skill_view, skills_list, text_to_speech│
|
||||
└──────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
For implementation details, see [PR #24182](https://github.com/NousResearch/hermes-agent/pull/24182) and the [Codex app-server protocol README](https://github.com/openai/codex/blob/main/codex-rs/app-server/README.md).
|
||||
@@ -522,6 +522,86 @@ print(json.dumps({"wakeAgent": True, "context": {"new_issues": latest - prev}}))
|
||||
|
||||
When `wakeAgent` is omitted, the default is `true` (wake the agent as usual).
|
||||
|
||||
#### Recipes: cheap pre-run gates
|
||||
|
||||
The `wakeAgent` gate gives you a $0 way to decide whether a scheduled job should spend any LLM tokens at all. Three patterns cover most use cases.
|
||||
|
||||
**File-change gate** — only run when a watched file has new content since the last successful tick. The scheduler records each job's `last_run_at`; compare it against the file's mtime.
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# ~/.hermes/scripts/feed-changed.sh
|
||||
FEED="$HOME/data/feed.json"
|
||||
STATE="$HOME/.hermes/scripts/.feed-changed.last"
|
||||
test -f "$FEED" || { echo '{"wakeAgent": false}'; exit 0; }
|
||||
mtime=$(stat -c %Y "$FEED")
|
||||
last=$(cat "$STATE" 2>/dev/null || echo 0)
|
||||
if [ "$mtime" -le "$last" ]; then
|
||||
echo '{"wakeAgent": false}'
|
||||
else
|
||||
echo "$mtime" > "$STATE"
|
||||
echo '{"wakeAgent": true}'
|
||||
fi
|
||||
```
|
||||
|
||||
```text
|
||||
cronjob(action="create", name="process-feed",
|
||||
schedule="every 30m",
|
||||
script="feed-changed.sh",
|
||||
prompt="A new ~/data/feed.json has landed. Summarize what changed.")
|
||||
```
|
||||
|
||||
**External-flag gate** — only run when some other process has signalled readiness (e.g. a deploy hook drops a file, a CI job sets a value in your state store).
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# ~/.hermes/scripts/flag-ready.sh
|
||||
if test -f /tmp/new-data-ready; then
|
||||
rm -f /tmp/new-data-ready
|
||||
echo '{"wakeAgent": true}'
|
||||
else
|
||||
echo '{"wakeAgent": false}'
|
||||
fi
|
||||
```
|
||||
|
||||
```text
|
||||
cronjob(action="create", name="nightly-analysis",
|
||||
schedule="0 9 * * *",
|
||||
script="flag-ready.sh",
|
||||
prompt="Run the nightly analysis over today's batch.")
|
||||
```
|
||||
|
||||
**SQL-count gate** — only run when there are new rows to process in your own database. The script can also pass the count through to the agent via `context`, so the agent knows how much it's looking at without re-querying.
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python
|
||||
# ~/.hermes/scripts/new-rows.py
|
||||
import json, sqlite3
|
||||
conn = sqlite3.connect("/home/me/data/app.db")
|
||||
n = conn.execute(
|
||||
"SELECT COUNT(*) FROM messages WHERE ts > strftime('%s','now','-2 hours')"
|
||||
).fetchone()[0]
|
||||
if n < 1:
|
||||
print(json.dumps({"wakeAgent": False}))
|
||||
else:
|
||||
print(json.dumps({"wakeAgent": True, "context": {"new_rows": n}}))
|
||||
```
|
||||
|
||||
```text
|
||||
cronjob(action="create", name="summarize-new-msgs",
|
||||
schedule="every 2h",
|
||||
script="new-rows.py",
|
||||
prompt="Summarize the new messages from the last 2 hours.")
|
||||
```
|
||||
|
||||
The same pattern works for any data source you can query from a script — Postgres, an HTTP API, your own state store — without baking a SQL evaluator into the cron subsystem.
|
||||
|
||||
:::tip
|
||||
Hermes's own `~/.hermes/state.db` is an internal schema that changes between releases. Don't query it from a pre-run gate — point at your own database or feed instead.
|
||||
:::
|
||||
|
||||
Credit: this recipe set was prompted by @iankar8's exploration in [#2654](https://github.com/NousResearch/hermes-agent/pull/2654), which proposed adding sql/file/command triggers as a parallel mechanism. The `script` + `wakeAgent` gate already covers all three cases at $0, so the work landed as documentation instead.
|
||||
|
||||
### Chaining jobs: `context_from`
|
||||
|
||||
A cron job can consume the most recent successful output of one or more other jobs by listing their names (or IDs) in `context_from`:
|
||||
|
||||
@@ -66,6 +66,7 @@ Both `provider` and `model` are **required**. If either is missing, the fallback
|
||||
| Google Gemini (OAuth) | `google-gemini-cli` | `hermes model` (Google OAuth; optional: `HERMES_GEMINI_PROJECT_ID`) |
|
||||
| Google AI Studio | `gemini` | `GOOGLE_API_KEY` (alias: `GEMINI_API_KEY`) |
|
||||
| xAI (Grok) | `xai` (alias `grok`) | `XAI_API_KEY` (optional: `XAI_BASE_URL`) |
|
||||
| xAI Grok OAuth (SuperGrok) | `xai-oauth` (alias `grok-oauth`) | `hermes model` → xAI Grok OAuth (browser login; SuperGrok subscription) |
|
||||
| AWS Bedrock | `bedrock` | Standard boto3 auth (`AWS_REGION` + `AWS_PROFILE` or `AWS_ACCESS_KEY_ID`) |
|
||||
| Qwen Portal (OAuth) | `qwen-oauth` | `hermes model` (Qwen Portal OAuth; optional: `HERMES_QWEN_BASE_URL`) |
|
||||
| MiniMax (OAuth) | `minimax-oauth` | `hermes model` (MiniMax portal OAuth) |
|
||||
|
||||
@@ -21,7 +21,7 @@ install, no separate daemon to manage.
|
||||
## When LSP runs
|
||||
|
||||
LSP is gated on **git workspace detection**. When the agent's working
|
||||
directory (or the file being edited) is inside a git worktree, LSP
|
||||
directory (or the file being edited) is inside a git repository, LSP
|
||||
runs against that workspace. When neither is in a git repo, LSP
|
||||
stays dormant — useful for messaging gateways where the cwd is the
|
||||
user's home directory and there's no project to diagnose.
|
||||
@@ -249,5 +249,6 @@ the next edit re-spawns.
|
||||
|
||||
**Editing a file outside any git repo**
|
||||
|
||||
By design, LSP only runs inside git worktrees. Run `git init` in the
|
||||
project, or accept the in-process syntax-only fallback.
|
||||
By design, LSP only runs inside a git repository. If the project isn't
|
||||
yet initialized, run `git init` to enable LSP diagnostics. Otherwise the
|
||||
in-process syntax-only fallback applies.
|
||||
|
||||
@@ -109,6 +109,7 @@ Every `ctx.*` API below is available inside a plugin's `register(ctx)` function.
|
||||
| Distribute via pip | `[project.entry-points."hermes_agent.plugins"]` |
|
||||
| Register a gateway platform (Discord, Telegram, IRC, …) | `ctx.register_platform(name, label, adapter_factory, check_fn, ...)` — see [Adding Platform Adapters](/docs/developer-guide/adding-platform-adapters) |
|
||||
| Register an image-generation backend | `ctx.register_image_gen_provider(provider)` — see [Image Generation Provider Plugins](/docs/developer-guide/image-gen-provider-plugin) |
|
||||
| Register a video-generation backend | `ctx.register_video_gen_provider(provider)` — see [Video Generation Provider Plugins](/docs/developer-guide/video-gen-provider-plugin) |
|
||||
| Register a context-compression engine | `ctx.register_context_engine(engine)` — see [Context Engine Plugins](/docs/developer-guide/context-engine-plugin) |
|
||||
| Register a memory backend | Subclass `MemoryProvider` in `plugins/memory/<name>/__init__.py` — see [Memory Provider Plugins](/docs/developer-guide/memory-provider-plugin) (uses a separate discovery system) |
|
||||
| Run a host-owned LLM call | `ctx.llm.complete(...)` / `ctx.llm.complete_structured(...)` — borrow the user's active model + auth for a one-shot completion with optional JSON schema validation. See [Plugin LLM Access](/docs/developer-guide/plugin-llm-access) |
|
||||
@@ -230,6 +231,7 @@ The table above shows the four plugin categories, but within "General plugins" t
|
||||
| A **memory backend** (Honcho, Mem0, Supermemory, …) | Memory plugin — subclass `MemoryProvider` in `plugins/memory/<name>/` | [Memory Provider Plugins](/docs/developer-guide/memory-provider-plugin) |
|
||||
| A **context-compression strategy** | Context-engine plugin — `ctx.register_context_engine()` | [Context Engine Plugins](/docs/developer-guide/context-engine-plugin) |
|
||||
| An **image-generation backend** (DALL·E, SDXL, …) | Backend plugin — `ctx.register_image_gen_provider()` | [Image Generation Provider Plugins](/docs/developer-guide/image-gen-provider-plugin) |
|
||||
| A **video-generation backend** (Veo, Kling, Pixverse, Grok-Imagine, Runway, …) | Backend plugin — `ctx.register_video_gen_provider()` | [Video Generation Provider Plugins](/docs/developer-guide/video-gen-provider-plugin) |
|
||||
| A **TTS backend** (any CLI — Piper, VoxCPM, Kokoro, xtts, voice-cloning scripts, …) | Config-driven — declare under `tts.providers.<name>` with `type: command` in `config.yaml` | [TTS setup](/docs/user-guide/features/tts#custom-command-providers) |
|
||||
| An **STT backend** (custom whisper binary, local ASR CLI) | Config-driven — set `HERMES_LOCAL_STT_COMMAND` env var to a shell template | [Voice Message Transcription (STT)](/docs/user-guide/features/tts#voice-message-transcription-stt) |
|
||||
| **External tools via MCP** (filesystem, GitHub, Linear, Notion, any MCP server) | Config-driven — declare `mcp_servers.<name>` with `command:` / `url:` in `config.yaml`. Hermes auto-discovers the server's tools and registers them alongside built-ins. | [MCP](/docs/user-guide/features/mcp) |
|
||||
|
||||
@@ -1,234 +0,0 @@
|
||||
---
|
||||
sidebar_position: 13
|
||||
title: "RL Training"
|
||||
description: "Reinforcement learning on agent behaviors with Tinker-Atropos — environment discovery, training, and evaluation"
|
||||
---
|
||||
|
||||
# RL Training
|
||||
|
||||
Hermes Agent includes an integrated RL (Reinforcement Learning) training pipeline built on **Tinker-Atropos**. This enables training language models on environment-specific tasks using GRPO (Group Relative Policy Optimization) with LoRA adapters, orchestrated entirely through the agent's tool interface.
|
||||
|
||||
## Overview
|
||||
|
||||
The RL training system consists of three components:
|
||||
|
||||
1. **[Atropos](https://github.com/NousResearch/atropos)** — A trajectory API server that coordinates environment interactions, manages rollout groups, and computes advantages
|
||||
2. **[Tinker](https://thinkingmachines.ai/tinker/)** — A training service that handles model weights, LoRA training, sampling/inference, and optimizer steps
|
||||
3. **Environments** — Python classes that define tasks, scoring, and reward functions (e.g., GSM8K math problems)
|
||||
|
||||
The agent can discover environments, configure training parameters, launch training runs, and monitor metrics — all through a set of `rl_*` tools.
|
||||
|
||||
## Requirements
|
||||
|
||||
RL training requires:
|
||||
|
||||
- **Python >= 3.11** (Tinker package requirement)
|
||||
- **TINKER_API_KEY** — API key for the Tinker training service
|
||||
- **WANDB_API_KEY** — API key for [Weights & Biases](https://wandb.ai/) metrics tracking
|
||||
- The `tinker-atropos` submodule (at `tinker-atropos/` relative to the Hermes root)
|
||||
|
||||
```bash
|
||||
# Set up API keys
|
||||
hermes config set TINKER_API_KEY your-tinker-key
|
||||
hermes config set WANDB_API_KEY your-wandb-key
|
||||
```
|
||||
|
||||
When both keys are present and Python >= 3.11 is available, the `rl` toolset is automatically enabled.
|
||||
|
||||
## Available Tools
|
||||
|
||||
| Tool | Description |
|
||||
|------|-------------|
|
||||
| `rl_list_environments` | Discover available RL environments |
|
||||
| `rl_select_environment` | Select an environment and load its config |
|
||||
| `rl_get_current_config` | View configurable and locked fields |
|
||||
| `rl_edit_config` | Modify configurable training parameters |
|
||||
| `rl_start_training` | Launch a training run (spawns 3 processes) |
|
||||
| `rl_check_status` | Monitor training progress and WandB metrics |
|
||||
| `rl_stop_training` | Stop a running training job |
|
||||
| `rl_get_results` | Get final metrics and model weights path |
|
||||
| `rl_list_runs` | List all active and completed runs |
|
||||
| `rl_test_inference` | Quick inference test using OpenRouter |
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Discover Environments
|
||||
|
||||
```
|
||||
List the available RL environments
|
||||
```
|
||||
|
||||
The agent calls `rl_list_environments()` which scans `tinker-atropos/tinker_atropos/environments/` using AST parsing to find Python classes inheriting from `BaseEnv`. Each environment defines:
|
||||
|
||||
- **Dataset loading** — where training data comes from (e.g., HuggingFace datasets)
|
||||
- **Prompt construction** — how to format items for the model
|
||||
- **Scoring/verification** — how to evaluate model outputs and assign rewards
|
||||
|
||||
### 2. Select and Configure
|
||||
|
||||
```
|
||||
Select the GSM8K environment and show me the configuration
|
||||
```
|
||||
|
||||
The agent calls `rl_select_environment("gsm8k_tinker")`, then `rl_get_current_config()` to see all parameters.
|
||||
|
||||
Configuration fields are divided into two categories:
|
||||
|
||||
**Configurable fields** (can be modified):
|
||||
- `group_size` — Number of completions per item (default: 16)
|
||||
- `batch_size` — Training batch size (default: 128)
|
||||
- `wandb_name` — WandB run name (auto-set to `{env}-{timestamp}`)
|
||||
- Other environment-specific parameters
|
||||
|
||||
**Locked fields** (infrastructure settings, cannot be changed):
|
||||
- `tokenizer_name` — Model tokenizer (e.g., `Qwen/Qwen3-8B`)
|
||||
- `rollout_server_url` — Atropos API URL (`http://localhost:8000`)
|
||||
- `max_token_length` — Maximum token length (8192)
|
||||
- `max_num_workers` — Maximum parallel workers (2048)
|
||||
- `total_steps` — Total training steps (2500)
|
||||
- `lora_rank` — LoRA adapter rank (32)
|
||||
- `learning_rate` — Learning rate (4e-5)
|
||||
- `max_token_trainer_length` — Max tokens for trainer (9000)
|
||||
|
||||
### 3. Start Training
|
||||
|
||||
```
|
||||
Start the training run
|
||||
```
|
||||
|
||||
The agent calls `rl_start_training()` which:
|
||||
|
||||
1. Generates a YAML config file merging locked settings with configurable overrides
|
||||
2. Creates a unique run ID
|
||||
3. Spawns three processes:
|
||||
- **Atropos API server** (`run-api`) — trajectory coordination
|
||||
- **Tinker trainer** (`launch_training.py`) — LoRA training + FastAPI inference server on port 8001
|
||||
- **Environment** (`environment.py serve`) — the selected environment connecting to Atropos
|
||||
|
||||
The processes start with staggered delays (5s for API, 30s for trainer, 90s more for environment) to ensure proper initialization order.
|
||||
|
||||
### 4. Monitor Progress
|
||||
|
||||
```
|
||||
Check the status of training run abc12345
|
||||
```
|
||||
|
||||
The agent calls `rl_check_status(run_id)` which reports:
|
||||
|
||||
- Process status (running/exited for each of the 3 processes)
|
||||
- Running time
|
||||
- WandB metrics (step, reward mean, percent correct, eval accuracy)
|
||||
- Log file locations for debugging
|
||||
|
||||
:::note Rate Limiting
|
||||
Status checks are rate-limited to once every **30 minutes** per run ID. This prevents excessive polling during long-running training jobs that take hours.
|
||||
:::
|
||||
|
||||
### 5. Stop or Get Results
|
||||
|
||||
```
|
||||
Stop the training run
|
||||
# or
|
||||
Get the final results for run abc12345
|
||||
```
|
||||
|
||||
`rl_stop_training()` terminates all three processes in reverse order (environment → trainer → API). `rl_get_results()` retrieves final WandB metrics and training history.
|
||||
|
||||
## Inference Testing
|
||||
|
||||
Before committing to a full training run, you can test if an environment works correctly using `rl_test_inference`. This runs a few steps of inference and scoring using OpenRouter — no Tinker API needed, just an `OPENROUTER_API_KEY`.
|
||||
|
||||
```
|
||||
Test the selected environment with inference
|
||||
```
|
||||
|
||||
Default configuration:
|
||||
- **3 steps × 16 completions = 48 rollouts per model**
|
||||
- Tests 3 models at different scales for robustness:
|
||||
- `qwen/qwen3-8b` (small)
|
||||
- `z-ai/glm-4.7-flash` (medium)
|
||||
- `minimax/minimax-m2.7` (large)
|
||||
- Total: ~144 rollouts
|
||||
|
||||
This validates:
|
||||
- Environment loads correctly
|
||||
- Prompt construction works
|
||||
- Inference response parsing is robust across model scales
|
||||
- Verifier/scoring logic produces valid rewards
|
||||
|
||||
## Tinker API Integration
|
||||
|
||||
The trainer uses the [Tinker](https://tinker.computer) API for model training operations:
|
||||
|
||||
- **ServiceClient** — Creates training and sampling clients
|
||||
- **Training client** — Handles forward-backward passes with importance sampling loss, optimizer steps (Adam), and weight checkpointing
|
||||
- **Sampling client** — Provides inference using the latest trained weights
|
||||
|
||||
The training loop:
|
||||
1. Fetches a batch of rollouts from Atropos (prompt + completions + scores)
|
||||
2. Converts to Tinker Datum objects with padded logprobs and advantages
|
||||
3. Runs forward-backward pass with importance sampling loss
|
||||
4. Takes an optimizer step (Adam: lr=4e-5, β1=0.9, β2=0.95)
|
||||
5. Saves weights and creates a new sampling client for next-step inference
|
||||
6. Logs metrics to WandB
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
api["Atropos API<br/>run-api<br/>port 8000"]
|
||||
env["Environment<br/>BaseEnv implementation"]
|
||||
infer["OpenAI / sglang<br/>inference API<br/>port 8001"]
|
||||
trainer["Tinker Trainer<br/>LoRA training + FastAPI"]
|
||||
|
||||
env <--> api
|
||||
env --> infer
|
||||
api -->|"batches: tokens, scores, logprobs"| trainer
|
||||
trainer -->|"serves inference"| infer
|
||||
```
|
||||
|
||||
## Creating Custom Environments
|
||||
|
||||
To create a new RL environment:
|
||||
|
||||
1. Create a Python file in `tinker-atropos/tinker_atropos/environments/`
|
||||
2. Define a class that inherits from `BaseEnv`
|
||||
3. Implement the required methods:
|
||||
- `load_dataset()` — Load your training data
|
||||
- `get_next_item()` — Provide the next item to the model
|
||||
- `score_answer()` — Score model outputs and assign rewards
|
||||
- `collect_trajectories()` — Collect and return trajectories
|
||||
4. Optionally define a custom config class inheriting from `BaseEnvConfig`
|
||||
|
||||
Study the existing `gsm8k_tinker.py` as a template. The agent can help you create new environments — it can read existing environment files, inspect HuggingFace datasets, and write new environment code.
|
||||
|
||||
## WandB Metrics
|
||||
|
||||
Training runs log to Weights & Biases with these key metrics:
|
||||
|
||||
| Metric | Description |
|
||||
|--------|-------------|
|
||||
| `train/loss` | Training loss (importance sampling) |
|
||||
| `train/learning_rate` | Current learning rate |
|
||||
| `reward/mean` | Mean reward across groups |
|
||||
| `logprobs/mean` | Mean reference logprobs |
|
||||
| `logprobs/mean_training` | Mean training logprobs |
|
||||
| `logprobs/diff` | Logprob drift (reference - training) |
|
||||
| `advantages/mean` | Mean advantage values |
|
||||
| `advantages/std` | Advantage standard deviation |
|
||||
|
||||
## Log Files
|
||||
|
||||
Each training run generates log files in `~/.hermes/logs/rl_training/`:
|
||||
|
||||
```
|
||||
logs/
|
||||
├── api_{run_id}.log # Atropos API server logs
|
||||
├── trainer_{run_id}.log # Tinker trainer logs
|
||||
├── env_{run_id}.log # Environment process logs
|
||||
└── inference_tests/ # Inference test results
|
||||
├── test_{env}_{model}.jsonl
|
||||
└── test_{env}_{model}.log
|
||||
```
|
||||
|
||||
These are invaluable for debugging when training fails or produces unexpected results.
|
||||
@@ -351,6 +351,7 @@ Hermes can install directly from GitHub repositories and GitHub-based taps. This
|
||||
Default taps (browsable without any setup):
|
||||
- [openai/skills](https://github.com/openai/skills)
|
||||
- [anthropics/skills](https://github.com/anthropics/skills)
|
||||
- [huggingface/skills](https://github.com/huggingface/skills)
|
||||
- [VoltAgent/awesome-agent-skills](https://github.com/VoltAgent/awesome-agent-skills)
|
||||
- [garrytan/gstack](https://github.com/garrytan/gstack)
|
||||
|
||||
@@ -445,7 +446,7 @@ Important behavior:
|
||||
|-------|--------|--------|
|
||||
| `builtin` | Ships with Hermes | Always trusted |
|
||||
| `official` | `optional-skills/` in the repo | Builtin trust, no third-party warning |
|
||||
| `trusted` | Trusted registries/repos such as `openai/skills`, `anthropics/skills` | More permissive policy than community sources |
|
||||
| `trusted` | Trusted registries/repos such as `openai/skills`, `anthropics/skills`, `huggingface/skills` | More permissive policy than community sources |
|
||||
| `community` | Everything else (`skills.sh`, well-known endpoints, custom GitHub repos, most marketplaces) | Non-dangerous findings can be overridden with `--force`; `dangerous` verdicts stay blocked |
|
||||
|
||||
### Update lifecycle
|
||||
|
||||
@@ -0,0 +1,203 @@
|
||||
---
|
||||
sidebar_position: 15
|
||||
title: "Subscription Proxy"
|
||||
description: "Use your Nous Portal subscription (or other OAuth provider) as an OpenAI-compatible endpoint for external apps"
|
||||
---
|
||||
|
||||
# Subscription Proxy
|
||||
|
||||
The subscription proxy is a local HTTP server that lets external apps —
|
||||
OpenViking, Karakeep, Open WebUI, anything that speaks OpenAI-compatible
|
||||
chat completions — use your Hermes-managed provider subscription as their
|
||||
LLM endpoint. The proxy attaches the right credentials (refreshing them
|
||||
automatically) so the app never needs a static API key.
|
||||
|
||||
This is different from the [API server](./api-server.md):
|
||||
|
||||
| | API server | Subscription proxy |
|
||||
|---|---|---|
|
||||
| What it serves | Your agent (full toolset, memory, skills) | Raw model inference |
|
||||
| Use case | "Use Hermes as a chat backend" | "Use my Portal sub from another app" |
|
||||
| Auth | Your `API_SERVER_KEY` | Any bearer (proxy attaches the real one) |
|
||||
| Tool calls | Yes — the agent runs tools | No — passthrough only |
|
||||
|
||||
Use the API server when you want the **agent** as a backend. Use the
|
||||
proxy when you just want **the model** through your subscription.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Log into your provider (one-time)
|
||||
|
||||
```bash
|
||||
hermes login nous
|
||||
```
|
||||
|
||||
This opens your browser for the Nous Portal OAuth flow. Hermes stores
|
||||
the refresh token in `~/.hermes/auth.json` — the same place all Hermes
|
||||
provider logins live.
|
||||
|
||||
### 2. Start the proxy
|
||||
|
||||
```bash
|
||||
hermes proxy start
|
||||
```
|
||||
|
||||
```
|
||||
Starting Hermes proxy for Nous Portal
|
||||
Listening on: http://127.0.0.1:8645/v1
|
||||
Forwarding to: (resolved per-request from your subscription)
|
||||
Use any bearer token in the client — the proxy attaches your real credential.
|
||||
```
|
||||
|
||||
Leave this running in the foreground. Use `tmux`, `nohup`, or a systemd
|
||||
unit if you want it to survive logout.
|
||||
|
||||
### 3. Point your app at it
|
||||
|
||||
Any OpenAI-compatible app config takes the same triple:
|
||||
|
||||
```
|
||||
Base URL: http://127.0.0.1:8645/v1
|
||||
API key: anything (e.g. "sk-unused")
|
||||
Model: Hermes-4-70B # or Hermes-4.3-36B, Hermes-4-405B
|
||||
```
|
||||
|
||||
The proxy ignores the `Authorization` header from your app and attaches
|
||||
your real Portal credential to the upstream request. Refreshes happen
|
||||
automatically when the bearer approaches expiry.
|
||||
|
||||
## Available providers
|
||||
|
||||
```bash
|
||||
hermes proxy providers
|
||||
```
|
||||
|
||||
Currently shipped: `nous` (Nous Portal). More OAuth providers can be
|
||||
added by implementing the `UpstreamAdapter` interface in
|
||||
`hermes_cli/proxy/adapters/`.
|
||||
|
||||
## Check status
|
||||
|
||||
```bash
|
||||
hermes proxy status
|
||||
```
|
||||
|
||||
```
|
||||
Hermes proxy upstream adapters
|
||||
|
||||
[nous ] Nous Portal — ready (bearer expires 2026-05-15T06:43:21Z)
|
||||
```
|
||||
|
||||
If you see `not logged in`, run `hermes login nous`. If you see
|
||||
`credentials need attention`, your refresh token was revoked (rare —
|
||||
happens if you signed out from the Portal web UI) — just re-run
|
||||
`hermes login nous`.
|
||||
|
||||
## Allowed paths
|
||||
|
||||
The proxy only forwards paths the upstream actually serves. For Nous
|
||||
Portal:
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `/v1/chat/completions` | Chat completions (streaming + non-streaming) |
|
||||
| `/v1/completions` | Legacy text completions |
|
||||
| `/v1/embeddings` | Embeddings |
|
||||
| `/v1/models` | Model list |
|
||||
|
||||
Other paths (`/v1/images/generations`, `/v1/audio/speech`, etc.) return
|
||||
404 with a clear error pointing at the allowed paths. This keeps stray
|
||||
clients from leaking weird requests to the upstream.
|
||||
|
||||
## Configuring OpenViking to use Portal
|
||||
|
||||
[OpenViking](https://github.com/volcengine/OpenViking) is a context
|
||||
database that needs an LLM provider for its VLM (vision/language model
|
||||
used to extract memories) and embedding model. With the proxy, you can
|
||||
point its `vlm.api_base` at your local proxy:
|
||||
|
||||
Edit `~/.openviking/ov.conf`:
|
||||
|
||||
```json
|
||||
{
|
||||
"vlm": {
|
||||
"provider": "openai",
|
||||
"model": "Hermes-4-70B",
|
||||
"api_base": "http://127.0.0.1:8645/v1",
|
||||
"api_key": "unused-proxy-attaches-real-creds"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Then start your proxy in a terminal alongside `openviking-server`:
|
||||
|
||||
```bash
|
||||
# Terminal 1
|
||||
hermes proxy start
|
||||
|
||||
# Terminal 2
|
||||
openviking-server
|
||||
```
|
||||
|
||||
OpenViking's VLM calls now flow through your Portal subscription. The
|
||||
embedding model side still needs its own provider — Portal does serve
|
||||
`/v1/embeddings` but the model selection depends on what your tier
|
||||
supports; check `portal.nousresearch.com/models`.
|
||||
|
||||
## Configuring Karakeep (or any bookmark/summarizer app)
|
||||
|
||||
[Karakeep](https://karakeep.app/) takes an OpenAI-compatible API for
|
||||
bookmark summarization. In its config:
|
||||
|
||||
```bash
|
||||
# Karakeep .env
|
||||
OPENAI_API_BASE_URL=http://127.0.0.1:8645/v1
|
||||
OPENAI_API_KEY=any-non-empty-string
|
||||
INFERENCE_TEXT_MODEL=Hermes-4-70B
|
||||
```
|
||||
|
||||
Same pattern works for Open WebUI, LobeChat, NextChat, or any other
|
||||
OpenAI-compatible client.
|
||||
|
||||
## Exposing on LAN
|
||||
|
||||
By default the proxy binds `127.0.0.1` (localhost only). To let other
|
||||
machines on your network use it:
|
||||
|
||||
```bash
|
||||
hermes proxy start --host 0.0.0.0 --port 8645
|
||||
```
|
||||
|
||||
⚠ **Be aware:** anyone on your network can now use your Portal
|
||||
subscription. The proxy has no auth of its own — it accepts any bearer.
|
||||
Use a firewall, VPN, or reverse proxy with proper auth if you expose
|
||||
this beyond your trusted network.
|
||||
|
||||
## Rate limits
|
||||
|
||||
Your Portal tier's RPM/TPM limits apply across the whole proxy. The
|
||||
proxy doesn't fan out or pool — it's a single bearer with your full
|
||||
subscription quota. Monitor usage at
|
||||
[portal.nousresearch.com](https://portal.nousresearch.com).
|
||||
|
||||
## Architecture
|
||||
|
||||
The proxy is intentionally minimal. Per request:
|
||||
|
||||
1. Receive `POST /v1/chat/completions` from your app
|
||||
2. Look up the adapter's current credential (refresh if expiring)
|
||||
3. Forward the request body verbatim, with `Authorization: Bearer <minted-key>`
|
||||
4. Stream the response back unchanged (SSE preserved)
|
||||
|
||||
No transformation. No logging of request bodies. No agent loop. The
|
||||
proxy is a credential-attaching pass-through.
|
||||
|
||||
## Future: more OAuth providers
|
||||
|
||||
The adapter system is pluggable. Adding a new provider (e.g.
|
||||
HuggingFace, GitHub Copilot's chat endpoint, Anthropic via OAuth)
|
||||
requires implementing `UpstreamAdapter` in
|
||||
`hermes_cli/proxy/adapters/<provider>.py` and registering it in
|
||||
`adapters/__init__.py`. Providers that aren't OpenAI-compatible at the
|
||||
protocol level (Anthropic Messages API, for example) would need a
|
||||
transformation layer, which is out of scope for the current shape.
|
||||
@@ -277,6 +277,7 @@ Discord behavior is controlled through two files: **`~/.hermes/.env`** for crede
|
||||
| `DISCORD_HOME_CHANNEL_NAME` | No | `"Home"` | Display name for the home channel in logs and status output. |
|
||||
| `DISCORD_COMMAND_SYNC_POLICY` | No | `"safe"` | Controls native slash-command startup sync. `"safe"` diffs existing global commands and only updates what changed, recreating commands when Discord metadata changes cannot be applied via patch. `"bulk"` preserves the old `tree.sync()` behavior. `"off"` skips startup sync entirely. |
|
||||
| `DISCORD_REQUIRE_MENTION` | No | `true` | When `true`, the bot only responds in server channels when `@mentioned`. Set to `false` to respond to all messages in every channel. |
|
||||
| `DISCORD_THREAD_REQUIRE_MENTION` | No | `false` | When `true`, the in-thread mention shortcut is disabled — threads are gated the same as channels, requiring `@mention` even after the bot has already participated. Use this when multiple bots share a thread and you want each to fire only on explicit `@mention`. |
|
||||
| `DISCORD_FREE_RESPONSE_CHANNELS` | No | — | Comma-separated channel IDs where the bot responds without requiring an `@mention`, even when `DISCORD_REQUIRE_MENTION` is `true`. |
|
||||
| `DISCORD_IGNORE_NO_MENTION` | No | `true` | When `true`, the bot stays silent if a message `@mentions` other users but does **not** mention the bot. Prevents the bot from jumping into conversations directed at other people. Only applies in server channels, not DMs. |
|
||||
| `DISCORD_AUTO_THREAD` | No | `true` | When `true`, automatically creates a new thread for every `@mention` in a text channel, so each conversation is isolated (similar to Slack behavior). Messages already inside threads or DMs are unaffected. |
|
||||
@@ -285,6 +286,8 @@ Discord behavior is controlled through two files: **`~/.hermes/.env`** for crede
|
||||
| `DISCORD_IGNORED_CHANNELS` | No | — | Comma-separated channel IDs where the bot **never** responds, even when `@mentioned`. Takes priority over all other channel settings. |
|
||||
| `DISCORD_ALLOWED_CHANNELS` | No | — | Comma-separated channel IDs. When set, the bot **only** responds in these channels (plus DMs if allowed). Overrides `config.yaml` `discord.allowed_channels`. Combine with `DISCORD_IGNORED_CHANNELS` to express allow/deny rules. |
|
||||
| `DISCORD_NO_THREAD_CHANNELS` | No | — | Comma-separated channel IDs where the bot responds directly in the channel instead of creating a thread. Only relevant when `DISCORD_AUTO_THREAD` is `true`. |
|
||||
| `DISCORD_HISTORY_BACKFILL` | No | `true` | When `true`, prepend recent channel scrollback (since the bot's last response) to the user message when the bot is mentioned. Recovers context the bot would otherwise miss with `require_mention`. Skipped in DMs and free-response channels. Set to `false` to disable. |
|
||||
| `DISCORD_HISTORY_BACKFILL_LIMIT` | No | `50` | Maximum number of messages to scan backwards when assembling the backfill block. In practice the scan usually stops earlier — at the bot's own last message in the channel. |
|
||||
| `DISCORD_REPLY_TO_MODE` | No | `"first"` | Controls reply-reference behavior: `"off"` — never reply to the original message, `"first"` — reply-reference on the first message chunk only (default), `"all"` — reply-reference on every chunk. |
|
||||
| `DISCORD_ALLOW_MENTION_EVERYONE` | No | `false` | When `false` (default), the bot cannot ping `@everyone` or `@here` even if its response contains those tokens. Set to `true` to opt back in. See [Mention Control](#mention-control) below. |
|
||||
| `DISCORD_ALLOW_MENTION_ROLES` | No | `false` | When `false` (default), the bot cannot ping `@role` mentions. Set to `true` to allow. |
|
||||
@@ -302,11 +305,14 @@ The `discord` section in `~/.hermes/config.yaml` mirrors the env vars above. Con
|
||||
# Discord-specific settings
|
||||
discord:
|
||||
require_mention: true # Require @mention in server channels
|
||||
thread_require_mention: false # If true, require @mention in threads too (multi-bot threads)
|
||||
free_response_channels: "" # Comma-separated channel IDs (or YAML list)
|
||||
auto_thread: true # Auto-create threads on @mention
|
||||
reactions: true # Add emoji reactions during processing
|
||||
ignored_channels: [] # Channel IDs where bot never responds
|
||||
no_thread_channels: [] # Channel IDs where bot responds without threading
|
||||
history_backfill: true # Prepend recent channel scrollback on mention (default: true)
|
||||
history_backfill_limit: 50 # Max messages to scan backwards (default: 50)
|
||||
channel_prompts: {} # Per-channel ephemeral system prompts
|
||||
allow_mentions: # What the bot is allowed to ping (safe defaults)
|
||||
everyone: false # @everyone / @here pings (default: false)
|
||||
@@ -324,6 +330,20 @@ group_sessions_per_user: true # Isolate sessions per user in shared channels
|
||||
|
||||
When enabled, the bot only responds in server channels when directly `@mentioned`. DMs always get a response regardless of this setting.
|
||||
|
||||
#### `discord.thread_require_mention`
|
||||
|
||||
**Type:** boolean — **Default:** `false`
|
||||
|
||||
By default, once the bot has participated in a thread (auto-created on `@mention` or replied in once), it keeps responding to every subsequent message in that thread without needing to be `@mentioned` again. That's the right default for one-on-one conversations.
|
||||
|
||||
In **multi-bot threads** where users address one bot per turn, this default becomes a footgun — every other bot in the thread also fires on every message, burning credits and spamming the channel. Set `thread_require_mention: true` to disable the in-thread shortcut and gate threads the same way channels are gated. Explicit `@mentions` still work as before.
|
||||
|
||||
```yaml
|
||||
discord:
|
||||
require_mention: true
|
||||
thread_require_mention: true # multi-bot setup
|
||||
```
|
||||
|
||||
#### `discord.free_response_channels`
|
||||
|
||||
**Type:** string or list — **Default:** `""`
|
||||
@@ -350,7 +370,7 @@ Free-response channels also **skip auto-threading** — the bot replies inline r
|
||||
|
||||
**Type:** boolean — **Default:** `true`
|
||||
|
||||
When enabled, every `@mention` in a regular text channel automatically creates a new thread for the conversation. This keeps the main channel clean and gives each conversation its own isolated session history. Once a thread is created, subsequent messages in that thread don't require `@mention` — the bot knows it's already participating.
|
||||
When enabled, every `@mention` in a regular text channel automatically creates a new thread for the conversation. This keeps the main channel clean and gives each conversation its own isolated session history. Once a thread is created, subsequent messages in that thread don't require `@mention` — the bot knows it's already participating. Set [`thread_require_mention`](#discordthread_require_mention) to `true` to disable this in-thread shortcut for multi-bot setups.
|
||||
|
||||
Messages sent in existing threads or DMs are unaffected by this setting. Channels listed in `discord.free_response_channels` or `discord.no_thread_channels` also bypass auto-threading and get inline replies instead.
|
||||
|
||||
@@ -421,6 +441,47 @@ Behavior:
|
||||
- If a message arrives inside a thread or forum post and that thread has no explicit entry, Hermes falls back to the parent channel/forum ID.
|
||||
- Prompts are applied ephemerally at runtime, so changing them affects future turns immediately without rewriting past session history.
|
||||
|
||||
#### `discord.history_backfill`
|
||||
|
||||
**Type:** boolean — **Default:** `true`
|
||||
|
||||
When enabled, the bot recovers missed channel messages on each `@mention`. With `require_mention: true`, the bot only processes messages that tag it directly — everything else in the channel is invisible to the session transcript. History backfill scans backwards through recent channel history when triggered, collecting messages between the bot's last response and the current mention, and includes them as context.
|
||||
|
||||
Behavior by surface:
|
||||
|
||||
- **Server channels** (with `require_mention: true`): backfill scans the channel since the bot's last response. Useful when other participants posted while the bot wasn't addressed.
|
||||
- **Threads**: backfill scans the thread only — Discord's `channel.history()` on a thread returns only that thread's messages, not the parent channel. This is the right scope because threads are usually self-contained conversations.
|
||||
- **DMs**: skipped. Every DM message triggers the bot, so the session transcript is already complete — there's no mention gap to fill.
|
||||
- **Free-response channels** and **bot's own auto-created threads**: skipped for the same reason — no mention gating means no gap.
|
||||
|
||||
Per-user sessions (`group_sessions_per_user: true`, the default) also benefit: a user's session is missing the context posted by other channel participants and the user's own messages from before they tagged the bot. Backfill fills both gaps.
|
||||
|
||||
```yaml
|
||||
discord:
|
||||
history_backfill: true # default
|
||||
```
|
||||
|
||||
To turn it off:
|
||||
|
||||
```yaml
|
||||
discord:
|
||||
history_backfill: false
|
||||
```
|
||||
|
||||
> **Note:** Messages that arrive *while* the bot is processing (between a trigger and its response) are not captured. This is an accepted simplification — the user can re-send or tag again.
|
||||
|
||||
#### `discord.history_backfill_limit`
|
||||
|
||||
**Type:** integer — **Default:** `50`
|
||||
|
||||
Maximum number of messages to scan backwards when recovering channel context. In practice the scan usually stops much earlier — at the bot's own last message in the channel, which is the natural boundary between turns. This limit is a safety cap for cold starts and long gaps where no prior bot message exists in recent history.
|
||||
|
||||
```yaml
|
||||
discord:
|
||||
history_backfill: true
|
||||
history_backfill_limit: 50
|
||||
```
|
||||
|
||||
#### `group_sessions_per_user`
|
||||
|
||||
**Type:** boolean — **Default:** `true`
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
# SimpleX Chat
|
||||
|
||||
[SimpleX Chat](https://simplex.chat/) is a private, decentralised messaging platform where users own their contacts and groups. Unlike other platforms, SimpleX assigns no persistent user IDs — every contact is identified by an opaque internal ID generated at connection time, which makes it one of the most private messengers available.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- The **simplex-chat** CLI installed and running as a daemon
|
||||
- Python package **websockets** (`pip install websockets`)
|
||||
|
||||
## Install simplex-chat
|
||||
|
||||
Download the latest release from the [simplex-chat GitHub releases](https://github.com/simplex-chat/simplex-chat/releases) page, or via Docker:
|
||||
|
||||
```bash
|
||||
# Linux / macOS binary
|
||||
curl -L https://github.com/simplex-chat/simplex-chat/releases/latest/download/simplex-chat-ubuntu-22_04-x86-64 -o simplex-chat
|
||||
chmod +x simplex-chat
|
||||
|
||||
# Or Docker
|
||||
docker run -p 5225:5225 simplexchat/simplex-chat -p 5225
|
||||
```
|
||||
|
||||
## Start the daemon
|
||||
|
||||
```bash
|
||||
simplex-chat -p 5225
|
||||
```
|
||||
|
||||
The daemon listens on WebSocket at `ws://127.0.0.1:5225` by default.
|
||||
|
||||
## Configure Hermes
|
||||
|
||||
### Via setup wizard
|
||||
|
||||
```bash
|
||||
hermes setup gateway
|
||||
```
|
||||
|
||||
Select **SimpleX Chat** and follow the prompts.
|
||||
|
||||
### Via environment variables
|
||||
|
||||
Add these to `~/.hermes/.env`:
|
||||
|
||||
```
|
||||
SIMPLEX_WS_URL=ws://127.0.0.1:5225
|
||||
SIMPLEX_ALLOWED_USERS=<contact-id-1>,<contact-id-2>
|
||||
SIMPLEX_HOME_CHANNEL=<contact-id>
|
||||
```
|
||||
|
||||
| Variable | Required | Description |
|
||||
|---|---|---|
|
||||
| `SIMPLEX_WS_URL` | Yes | WebSocket URL of the simplex-chat daemon |
|
||||
| `SIMPLEX_ALLOWED_USERS` | Recommended | Comma-separated contact IDs allowed to use the agent |
|
||||
| `SIMPLEX_ALLOW_ALL_USERS` | Optional | Set `true` to allow every contact (use carefully) |
|
||||
| `SIMPLEX_HOME_CHANNEL` | Optional | Default contact ID for cron job delivery |
|
||||
| `SIMPLEX_HOME_CHANNEL_NAME` | Optional | Human label for the home channel |
|
||||
|
||||
## Find your contact ID
|
||||
|
||||
After starting the daemon, open a conversation with your agent contact. The contact ID will appear in session logs or via `hermes send_message action=list`.
|
||||
|
||||
## Authorization
|
||||
|
||||
By default **all contacts are denied**. You must either:
|
||||
|
||||
1. Set `SIMPLEX_ALLOWED_USERS` to a comma-separated list of contact IDs, or
|
||||
2. Use **DM pairing** — send any message to the bot and it will reply with a pairing code. Enter that code via `hermes gateway pair`.
|
||||
|
||||
## Using SimpleX with cron jobs
|
||||
|
||||
```python
|
||||
cronjob(
|
||||
action="create",
|
||||
schedule="every 1h",
|
||||
deliver="simplex", # uses SIMPLEX_HOME_CHANNEL
|
||||
prompt="Check for alerts and summarise."
|
||||
)
|
||||
```
|
||||
|
||||
Or target a specific contact:
|
||||
|
||||
```python
|
||||
send_message(target="simplex:<contact-id>", message="Done!")
|
||||
```
|
||||
|
||||
## Privacy notes
|
||||
|
||||
- SimpleX never reveals phone numbers or email addresses — contacts use opaque IDs
|
||||
- The connection between Hermes and the daemon is local WebSocket (`ws://127.0.0.1:5225`) — no data leaves your machine
|
||||
- Messages are end-to-end encrypted by the SimpleX protocol before reaching the daemon
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**"Cannot reach daemon"** — Ensure `simplex-chat -p 5225` is running and the port matches `SIMPLEX_WS_URL`.
|
||||
|
||||
**"websockets not installed"** — Run `pip install websockets`.
|
||||
|
||||
**Messages not received** — Check that the contact's ID is in `SIMPLEX_ALLOWED_USERS` or approve them via DM pairing.
|
||||
@@ -264,6 +264,22 @@ For backward compatibility with older manifests, you can still type
|
||||
run the tests`. Free-form questions also work: `/hermes what's the
|
||||
weather?` is treated as a regular message.
|
||||
|
||||
### Using commands inside threads (the `!cmd` prefix)
|
||||
|
||||
Slack itself blocks native slash commands inside thread replies — try
|
||||
`/queue` in a thread and Slack responds with *"/queue is not supported
|
||||
in threads. Sorry!"* There is no app-side setting that re-enables them;
|
||||
Slack never delivers them to Hermes.
|
||||
|
||||
As a workaround, Hermes recognises a leading `!` as an alternate
|
||||
command prefix that works in threads (and anywhere else). Type
|
||||
`!queue`, `!stop`, `!model gpt-5.4`, etc. as a regular thread reply —
|
||||
Hermes treats it identically to the slash form and replies in the same
|
||||
thread.
|
||||
|
||||
Only the first token is checked against the known command list, so
|
||||
casual messages like `!nice work` pass through to the agent unchanged.
|
||||
|
||||
### Advanced: emit only the slash-commands array
|
||||
|
||||
If you maintain your Slack manifest by hand and just want the slash
|
||||
|
||||
@@ -25,6 +25,43 @@ The SQLite database stores:
|
||||
- Timestamps (started_at, ended_at)
|
||||
- Parent session ID (for compression-triggered session splitting)
|
||||
|
||||
### What Counts Toward Context
|
||||
|
||||
Hermes stores session history so it can resume conversations, but it does not
|
||||
keep re-sending every byte it has ever handled. On each turn, the model sees
|
||||
the selected system prompt, the current conversation window, and any content
|
||||
Hermes explicitly injects for that turn.
|
||||
|
||||
Media attachments are handled as turn-scoped inputs:
|
||||
|
||||
- Images may be attached natively to the next model call, or pre-analyzed into
|
||||
a text description when the active model does not support native vision.
|
||||
- Audio is transcribed into text when speech-to-text is configured.
|
||||
- Text documents can have their extracted text included; other document types
|
||||
are usually represented by a saved local path and a short note.
|
||||
- Attachment paths and extracted/derived text can appear in the transcript, but
|
||||
the raw image, audio, or binary file bytes are not repeatedly copied into
|
||||
future prompts.
|
||||
|
||||
For example, if a user sends an image and asks Hermes to make a meme from it,
|
||||
Hermes may inspect that image once with vision and run an image-processing
|
||||
script. Future turns do not automatically carry the original JPEG in context.
|
||||
They carry only whatever was written into the conversation, such as the user's
|
||||
request, a short image description, a local cache path, or the final assistant
|
||||
response.
|
||||
|
||||
The most common cause of context growth is not the media file itself. It is
|
||||
verbose text: pasted transcripts, full logs, large tool outputs, long diffs,
|
||||
repeated status reports, and detailed proof dumps. Prefer summaries, file
|
||||
paths, focused excerpts, and tool-backed lookups over copying large artifacts
|
||||
into chat.
|
||||
|
||||
:::tip
|
||||
Use `/compress` when a session gets long, `/new` for a fresh thread, and
|
||||
`hermes sessions prune` only when you want to delete old ended sessions from
|
||||
storage. Compression reduces the active context; it is not a privacy delete.
|
||||
:::
|
||||
|
||||
### Session Sources
|
||||
|
||||
Each session is tagged with its source platform:
|
||||
|
||||
@@ -1,249 +0,0 @@
|
||||
---
|
||||
title: "Base"
|
||||
sidebar_label: "Base"
|
||||
description: "Query Base (Ethereum L2) blockchain data with USD pricing — wallet balances, token info, transaction details, gas analysis, contract inspection, whale detect..."
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Base
|
||||
|
||||
Query Base (Ethereum L2) blockchain data with USD pricing — wallet balances, token info, transaction details, gas analysis, contract inspection, whale detection, and live network stats. Uses Base RPC + CoinGecko. No API key required.
|
||||
|
||||
## Skill metadata
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Source | Optional — install with `hermes skills install official/blockchain/base` |
|
||||
| Path | `optional-skills/blockchain/base` |
|
||||
| Version | `0.1.0` |
|
||||
| Author | youssefea |
|
||||
| License | MIT |
|
||||
| Platforms | linux, macos, windows |
|
||||
| Tags | `Base`, `Blockchain`, `Crypto`, `Web3`, `RPC`, `DeFi`, `EVM`, `L2`, `Ethereum` |
|
||||
|
||||
## Reference: full SKILL.md
|
||||
|
||||
:::info
|
||||
The following is the complete skill definition that Hermes loads when this skill is triggered. This is what the agent sees as instructions when the skill is active.
|
||||
:::
|
||||
|
||||
# Base Blockchain Skill
|
||||
|
||||
Query Base (Ethereum L2) on-chain data enriched with USD pricing via CoinGecko.
|
||||
8 commands: wallet portfolio, token info, transactions, gas analysis,
|
||||
contract inspection, whale detection, network stats, and price lookup.
|
||||
|
||||
No API key needed. Uses only Python standard library (urllib, json, argparse).
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks for a Base wallet balance, token holdings, or portfolio value
|
||||
- User wants to inspect a specific transaction by hash
|
||||
- User wants ERC-20 token metadata, price, supply, or market cap
|
||||
- User wants to understand Base gas costs and L1 data fees
|
||||
- User wants to inspect a contract (ERC type detection, proxy resolution)
|
||||
- User wants to find large ETH transfers (whale detection)
|
||||
- User wants Base network health, gas price, or ETH price
|
||||
- User asks "what's the price of USDC/AERO/DEGEN/ETH?"
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
The helper script uses only Python standard library (urllib, json, argparse).
|
||||
No external packages required.
|
||||
|
||||
Pricing data comes from CoinGecko's free API (no key needed, rate-limited
|
||||
to ~10-30 requests/minute). For faster lookups, use `--no-prices` flag.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
RPC endpoint (default): https://mainnet.base.org
|
||||
Override: export BASE_RPC_URL=https://your-private-rpc.com
|
||||
|
||||
Helper script path: ~/.hermes/skills/blockchain/base/scripts/base_client.py
|
||||
|
||||
```
|
||||
python3 base_client.py wallet <address> [--limit N] [--all] [--no-prices]
|
||||
python3 base_client.py tx <hash>
|
||||
python3 base_client.py token <contract_address>
|
||||
python3 base_client.py gas
|
||||
python3 base_client.py contract <address>
|
||||
python3 base_client.py whales [--min-eth N]
|
||||
python3 base_client.py stats
|
||||
python3 base_client.py price <contract_address_or_symbol>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Procedure
|
||||
|
||||
### 0. Setup Check
|
||||
|
||||
```bash
|
||||
python3 --version
|
||||
|
||||
# Optional: set a private RPC for better rate limits
|
||||
export BASE_RPC_URL="https://mainnet.base.org"
|
||||
|
||||
# Confirm connectivity
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py stats
|
||||
```
|
||||
|
||||
### 1. Wallet Portfolio
|
||||
|
||||
Get ETH balance and ERC-20 token holdings with USD values.
|
||||
Checks ~15 well-known Base tokens (USDC, WETH, AERO, DEGEN, etc.)
|
||||
via on-chain `balanceOf` calls. Tokens sorted by value, dust filtered.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py \
|
||||
wallet 0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045
|
||||
```
|
||||
|
||||
Flags:
|
||||
- `--limit N` — show top N tokens (default: 20)
|
||||
- `--all` — show all tokens, no dust filter, no limit
|
||||
- `--no-prices` — skip CoinGecko price lookups (faster, RPC-only)
|
||||
|
||||
Output includes: ETH balance + USD value, token list with prices sorted
|
||||
by value, dust count, total portfolio value in USD.
|
||||
|
||||
Note: Only checks known tokens. Unknown ERC-20s are not discovered.
|
||||
Use the `token` command with a specific contract address for any token.
|
||||
|
||||
### 2. Transaction Details
|
||||
|
||||
Inspect a full transaction by its hash. Shows ETH value transferred,
|
||||
gas used, fee in ETH/USD, status, and decoded ERC-20/ERC-721 transfers.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py \
|
||||
tx 0xabc123...your_tx_hash_here
|
||||
```
|
||||
|
||||
Output: hash, block, from, to, value (ETH + USD), gas price, gas used,
|
||||
fee, status, contract creation address (if any), token transfers.
|
||||
|
||||
### 3. Token Info
|
||||
|
||||
Get ERC-20 token metadata: name, symbol, decimals, total supply, price,
|
||||
market cap, and contract code size.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py \
|
||||
token 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913
|
||||
```
|
||||
|
||||
Output: name, symbol, decimals, total supply, price, market cap.
|
||||
Reads name/symbol/decimals directly from the contract via eth_call.
|
||||
|
||||
### 4. Gas Analysis
|
||||
|
||||
Detailed gas analysis with cost estimates for common operations.
|
||||
Shows current gas price, base fee trends over 10 blocks, block
|
||||
utilization, and estimated costs for ETH transfers, ERC-20 transfers,
|
||||
and swaps.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py gas
|
||||
```
|
||||
|
||||
Output: current gas price, base fee, block utilization, 10-block trend,
|
||||
cost estimates in ETH and USD.
|
||||
|
||||
Note: Base is an L2 — actual transaction costs include an L1 data
|
||||
posting fee that depends on calldata size and L1 gas prices. The
|
||||
estimates shown are for L2 execution only.
|
||||
|
||||
### 5. Contract Inspection
|
||||
|
||||
Inspect an address: determine if it's an EOA or contract, detect
|
||||
ERC-20/ERC-721/ERC-1155 interfaces, resolve EIP-1967 proxy
|
||||
implementation addresses.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py \
|
||||
contract 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913
|
||||
```
|
||||
|
||||
Output: is_contract, code size, ETH balance, detected interfaces
|
||||
(ERC-20, ERC-721, ERC-1155), ERC-20 metadata, proxy implementation
|
||||
address.
|
||||
|
||||
### 6. Whale Detector
|
||||
|
||||
Scan the most recent block for large ETH transfers with USD values.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py \
|
||||
whales --min-eth 1.0
|
||||
```
|
||||
|
||||
Note: scans the latest block only — point-in-time snapshot, not historical.
|
||||
Default threshold is 1.0 ETH (lower than Solana's default since ETH
|
||||
values are higher).
|
||||
|
||||
### 7. Network Stats
|
||||
|
||||
Live Base network health: latest block, chain ID, gas price, base fee,
|
||||
block utilization, transaction count, and ETH price.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py stats
|
||||
```
|
||||
|
||||
### 8. Price Lookup
|
||||
|
||||
Quick price check for any token by contract address or known symbol.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py price ETH
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py price USDC
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py price AERO
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py price DEGEN
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py price 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913
|
||||
```
|
||||
|
||||
Known symbols: ETH, WETH, USDC, cbETH, AERO, DEGEN, TOSHI, BRETT,
|
||||
WELL, wstETH, rETH, cbBTC.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **CoinGecko rate-limits** — free tier allows ~10-30 requests/minute.
|
||||
Price lookups use 1 request per token. Use `--no-prices` for speed.
|
||||
- **Public RPC rate-limits** — Base's public RPC limits requests.
|
||||
For production use, set BASE_RPC_URL to a private endpoint
|
||||
(Alchemy, QuickNode, Infura).
|
||||
- **Wallet shows known tokens only** — unlike Solana, EVM chains have no
|
||||
built-in "get all tokens" RPC. The wallet command checks ~15 popular
|
||||
Base tokens via `balanceOf`. Unknown ERC-20s won't appear. Use the
|
||||
`token` command for any specific contract.
|
||||
- **Token names read from contract** — if a contract doesn't implement
|
||||
`name()` or `symbol()`, these fields may be empty. Known tokens have
|
||||
hardcoded labels as fallback.
|
||||
- **Gas estimates are L2 only** — Base transaction costs include an L1
|
||||
data posting fee (depends on calldata size and L1 gas prices). The gas
|
||||
command estimates L2 execution cost only.
|
||||
- **Whale detector scans latest block only** — not historical. Results
|
||||
vary by the moment you query. Default threshold is 1.0 ETH.
|
||||
- **Proxy detection** — only EIP-1967 proxies are detected. Other proxy
|
||||
patterns (EIP-1167 minimal proxy, custom storage slots) are not checked.
|
||||
- **Retry on 429** — both RPC and CoinGecko calls retry up to 2 times
|
||||
with exponential backoff on rate-limit errors.
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# Should print Base chain ID (8453), latest block, gas price, and ETH price
|
||||
python3 ~/.hermes/skills/blockchain/base/scripts/base_client.py stats
|
||||
```
|
||||
@@ -0,0 +1,227 @@
|
||||
---
|
||||
title: "Evm — Read-only EVM client: wallets, tokens, gas across 8 chains"
|
||||
sidebar_label: "Evm"
|
||||
description: "Read-only EVM client: wallets, tokens, gas across 8 chains"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Evm
|
||||
|
||||
Read-only EVM client: wallets, tokens, gas across 8 chains.
|
||||
|
||||
## Skill metadata
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Source | Optional — install with `hermes skills install official/blockchain/evm` |
|
||||
| Path | `optional-skills/blockchain/evm` |
|
||||
| Version | `1.0.0` |
|
||||
| Author | Mibayy (@Mibayy), youssefea (@youssefea), ethernet8023 (@ethernet8023), Hermes Agent |
|
||||
| License | MIT |
|
||||
| Platforms | linux, macos, windows |
|
||||
| Tags | `EVM`, `Ethereum`, `BNB`, `BSC`, `Base`, `Arbitrum`, `Polygon`, `Optimism`, `Avalanche`, `zkSync`, `Blockchain`, `Crypto`, `Web3`, `DeFi`, `NFT`, `ENS`, `Whale`, `Security` |
|
||||
| Related skills | [`solana`](/docs/user-guide/skills/optional/blockchain/blockchain-solana) |
|
||||
|
||||
## Reference: full SKILL.md
|
||||
|
||||
:::info
|
||||
The following is the complete skill definition that Hermes loads when this skill is triggered. This is what the agent sees as instructions when the skill is active.
|
||||
:::
|
||||
|
||||
# EVM Blockchain Skill
|
||||
|
||||
Query EVM-compatible blockchain data across 8 chains with USD pricing.
|
||||
14 commands: wallet portfolio, token info, transactions, activity, gas tracker,
|
||||
network stats, price lookup, multi-chain scan, whale detection, ENS resolution,
|
||||
allowance checker, contract inspector, and transaction decoder.
|
||||
|
||||
Supports 8 chains: Ethereum, BNB Chain (BSC), Base, Arbitrum One, Polygon,
|
||||
Optimism, Avalanche (C-Chain), zkSync Era.
|
||||
|
||||
No API key needed. Zero external dependencies — Python standard library only
|
||||
(urllib, json, argparse, threading).
|
||||
|
||||
> **Supersedes the standalone `base` skill.** Base-specific tokens (AERO, DEGEN,
|
||||
> TOSHI, BRETT, WELL, cbETH, cbBTC, wstETH, rETH) and all Base RPC functionality
|
||||
> previously living under `optional-skills/blockchain/base/` have been folded
|
||||
> into this skill. Pass `--chain base` to any command for Base coverage.
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
- User asks for a wallet balance or portfolio on any EVM chain
|
||||
- User wants to check the same wallet across ALL chains at once
|
||||
- User wants to inspect a transaction by hash (or decode what it did)
|
||||
- User wants ERC-20 token metadata, price, supply, or market cap
|
||||
- User wants recent transaction history for an address
|
||||
- User wants current gas prices or to compare fees across chains
|
||||
- User wants to find large whale transfers in recent blocks
|
||||
- User asks to resolve an ENS name (vitalik.eth) or reverse-lookup an address
|
||||
- User wants to check if a contract has dangerous token approvals
|
||||
- User wants to inspect a smart contract (proxy? ERC-20? ERC-721? bytecode size?)
|
||||
- User wants to compare gas costs across chains before a transaction
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
Python 3.8+ standard library only. No pip installs required.
|
||||
Pricing: CoinGecko free API (rate-limited, ~10-30 req/min).
|
||||
ENS: ensideas.com public API.
|
||||
Tx decoding: 4byte.directory public API.
|
||||
|
||||
Override RPC endpoint: `export EVM_RPC_URL=https://your-rpc.com`
|
||||
|
||||
Helper script path: `~/.hermes/skills/blockchain/evm/scripts/evm_client.py`
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
```
|
||||
SCRIPT=~/.hermes/skills/blockchain/evm/scripts/evm_client.py
|
||||
|
||||
# Network & prices
|
||||
python3 $SCRIPT stats # Ethereum stats
|
||||
python3 $SCRIPT stats --chain arbitrum # Arbitrum stats
|
||||
python3 $SCRIPT compare # Gas + prices ALL 8 chains
|
||||
|
||||
# Wallet
|
||||
python3 $SCRIPT wallet 0xd8dA...96045 # Portfolio (ETH + ERC-20)
|
||||
python3 $SCRIPT wallet 0xd8dA...96045 --chain bsc
|
||||
python3 $SCRIPT multichain 0xd8dA...96045 # Same wallet on ALL chains
|
||||
|
||||
# Tokens & prices
|
||||
python3 $SCRIPT price ETH
|
||||
python3 $SCRIPT price 0xdAC1...1ec7 # By contract address
|
||||
python3 $SCRIPT token 0xdAC1...1ec7 # ERC-20 metadata + market cap
|
||||
|
||||
# Transactions
|
||||
python3 $SCRIPT tx 0x5c50...f060 # Transaction details
|
||||
python3 $SCRIPT decode 0x5c50...f060 # Decode input data (4byte.directory)
|
||||
python3 $SCRIPT activity 0xd8dA...96045 # Recent transactions
|
||||
|
||||
# Gas
|
||||
python3 $SCRIPT gas # Gas prices + cost estimates
|
||||
python3 $SCRIPT gas --chain optimism
|
||||
|
||||
# Security
|
||||
python3 $SCRIPT allowance 0xd8dA...96045 # Dangerous ERC-20 approvals
|
||||
python3 $SCRIPT contract 0xdAC1...1ec7 # Contract inspection (proxy? standards?)
|
||||
|
||||
# ENS
|
||||
python3 $SCRIPT ens vitalik.eth # Name -> address + profile
|
||||
python3 $SCRIPT ens 0xd8dA...96045 # Address -> ENS name
|
||||
|
||||
# Whale detection
|
||||
python3 $SCRIPT whale # Large transfers (last 20 blocks, >$10k)
|
||||
python3 $SCRIPT whale --blocks 50 --min-usd 100000 --chain arbitrum
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Procedure
|
||||
|
||||
### 0. Setup Check
|
||||
```bash
|
||||
python3 --version # 3.8+ required
|
||||
python3 ~/.hermes/skills/blockchain/evm/scripts/evm_client.py stats
|
||||
```
|
||||
|
||||
### 1. Wallet Portfolio
|
||||
Native balance + known ERC-20 tokens, sorted by USD value.
|
||||
```bash
|
||||
python3 $SCRIPT wallet 0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045
|
||||
python3 $SCRIPT wallet 0xd8dA... --chain bsc --no-prices # faster
|
||||
```
|
||||
|
||||
### 2. Multi-Chain Scan
|
||||
Scans all 8 chains simultaneously for the same address using threads.
|
||||
```bash
|
||||
python3 $SCRIPT multichain 0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045
|
||||
```
|
||||
Output: per-chain native balance + token holdings + grand total USD.
|
||||
|
||||
### 3. Compare (Gas + Prices)
|
||||
All 8 chains queried in parallel. Shows cheapest/most expensive chain.
|
||||
```bash
|
||||
python3 $SCRIPT compare
|
||||
```
|
||||
|
||||
### 4. Transaction Details & Decode
|
||||
```bash
|
||||
python3 $SCRIPT tx 0x5c504ed432cb51138bcf09aa5e8a410dd4a1e204ef84bfed1be16dfba1b22060
|
||||
python3 $SCRIPT decode 0x5c504ed... # Shows human-readable function signature
|
||||
```
|
||||
Decode uses 4byte.directory to translate 0xa9059cbb -> transfer(address,uint256).
|
||||
|
||||
### 5. ENS Resolution
|
||||
```bash
|
||||
python3 $SCRIPT ens vitalik.eth # -> 0xd8dA... + avatar + social links
|
||||
python3 $SCRIPT ens 0xd8dA...96045 # -> vitalik.eth
|
||||
```
|
||||
|
||||
### 6. Allowance Checker (Security)
|
||||
Checks ERC-20 approvals granted to known DEX/bridge contracts.
|
||||
```bash
|
||||
python3 $SCRIPT allowance 0xYourWallet
|
||||
```
|
||||
Flags UNLIMITED approvals as HIGH risk.
|
||||
|
||||
### 7. Contract Inspector
|
||||
```bash
|
||||
python3 $SCRIPT contract 0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48 # USDC (proxy)
|
||||
python3 $SCRIPT contract 0xdAC17F958D2ee523a2206206994597C13D831ec7 # USDT (ERC-20)
|
||||
```
|
||||
Detects: proxy (EIP-1967/EIP-1167), ERC-20, ERC-721, ERC-165. Shows bytecode size and implementation address for proxies.
|
||||
|
||||
### 8. Whale Detection
|
||||
```bash
|
||||
python3 $SCRIPT whale # ETH, last 20 blocks, >$10k
|
||||
python3 $SCRIPT whale --blocks 50 --min-usd 50000 --chain bsc
|
||||
```
|
||||
|
||||
### 9. Gas Tracker
|
||||
```bash
|
||||
python3 $SCRIPT gas
|
||||
python3 $SCRIPT gas --chain polygon
|
||||
```
|
||||
Shows gwei price + USD cost for: transfer, ERC-20 transfer, approve, swap, NFT mint, NFT transfer.
|
||||
|
||||
---
|
||||
|
||||
## Supported Chains
|
||||
| Key | Name | Native | Chain ID |
|
||||
|-----------|----------------|--------|----------|
|
||||
| ethereum | Ethereum | ETH | 1 |
|
||||
| bsc | BNB Chain | BNB | 56 |
|
||||
| base | Base | ETH | 8453 |
|
||||
| arbitrum | Arbitrum One | ETH | 42161 |
|
||||
| polygon | Polygon | POL | 137 |
|
||||
| optimism | Optimism | ETH | 10 |
|
||||
| avalanche | Avalanche C | AVAX | 43114 |
|
||||
| zksync | zkSync Era | ETH | 324 |
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
- CoinGecko free tier: ~10-30 req/min. Use `--no-prices` for faster wallet scans.
|
||||
- Public RPCs may throttle. Set EVM_RPC_URL to a private endpoint for production.
|
||||
- `wallet` and `allowance` only check known token list (~30 tokens per chain). Use a block explorer for complete token discovery.
|
||||
- `activity` scans recent blocks only (max 200). For full history, use Etherscan API.
|
||||
- `multichain` runs 8 parallel threads — can trigger rate limits on public RPCs.
|
||||
- ENS resolution depends on a single public endpoint (ensideas.com / ens.vitalik.ca) with no fallback. If that endpoint is down, `ens` will fail — re-run later or use a block explorer.
|
||||
- Tx decoding depends on a single public endpoint (4byte.directory) with no fallback. Selectors not in their database show up as `unknown`.
|
||||
- **L2 gas estimates are L2-execution only.** On rollups like Base, Arbitrum, Optimism, and zkSync, the actual transaction cost also includes an L1 data-posting fee that depends on calldata size and current L1 gas prices. The `gas` command does not estimate that L1 component. For Base specifically, see the network's L1 fee oracle (contract `0x420000000000000000000000000000000000000F`).
|
||||
- Address / tx-hash inputs are validated for 0x-prefix + correct length + hex, but EIP-55 checksum casing is **not** enforced (RPC endpoints accept any-case hex).
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
```bash
|
||||
# Should print current block, gas price, ETH price
|
||||
python3 ~/.hermes/skills/blockchain/evm/scripts/evm_client.py stats
|
||||
|
||||
# Should resolve vitalik.eth to 0xd8dA...
|
||||
python3 ~/.hermes/skills/blockchain/evm/scripts/evm_client.py ens vitalik.eth
|
||||
```
|
||||
@@ -1,323 +0,0 @@
|
||||
---
|
||||
title: "Hermes Atropos Environments — Build, test, and debug Hermes Agent RL environments for Atropos training"
|
||||
sidebar_label: "Hermes Atropos Environments"
|
||||
description: "Build, test, and debug Hermes Agent RL environments for Atropos training"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Hermes Atropos Environments
|
||||
|
||||
Build, test, and debug Hermes Agent RL environments for Atropos training. Covers the HermesAgentBaseEnv interface, reward functions, agent loop integration, evaluation with tools, wandb logging, and the three CLI modes (serve/process/evaluate). Use when creating, reviewing, or fixing RL environments in the hermes-agent repo.
|
||||
|
||||
## Skill metadata
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Source | Optional — install with `hermes skills install official/mlops/hermes-atropos-environments` |
|
||||
| Path | `optional-skills/mlops/hermes-atropos-environments` |
|
||||
| Version | `1.1.0` |
|
||||
| Author | Hermes Agent |
|
||||
| License | MIT |
|
||||
| Platforms | linux, macos, windows |
|
||||
| Tags | `atropos`, `rl`, `environments`, `training`, `reinforcement-learning`, `reward-functions` |
|
||||
| Related skills | [`axolotl`](/docs/user-guide/skills/optional/mlops/mlops-training-axolotl), [`fine-tuning-with-trl`](/docs/user-guide/skills/optional/mlops/mlops-training-trl-fine-tuning), `lm-evaluation-harness` |
|
||||
|
||||
## Reference: full SKILL.md
|
||||
|
||||
:::info
|
||||
The following is the complete skill definition that Hermes loads when this skill is triggered. This is what the agent sees as instructions when the skill is active.
|
||||
:::
|
||||
|
||||
# Hermes Agent Atropos Environments
|
||||
|
||||
Guide for building RL environments in the hermes-agent repo that integrate with the Atropos training framework.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
<!-- ascii-guard-ignore -->
|
||||
```
|
||||
Atropos BaseEnv (atroposlib/envs/base.py)
|
||||
└── HermesAgentBaseEnv (environments/hermes_base_env.py)
|
||||
├── Handles agent loop orchestration
|
||||
├── Handles tool resolution per group
|
||||
├── Handles ToolContext for reward verification
|
||||
└── YOUR ENVIRONMENT (environments/your_env.py)
|
||||
Only implements: setup, get_next_item, format_prompt,
|
||||
compute_reward, evaluate, wandb_log
|
||||
```
|
||||
<!-- ascii-guard-ignore-end -->
|
||||
|
||||
Hermes environments are special because they run a **multi-turn agent loop with tool calling** — not just single-turn completions. The base env handles the loop; you implement the task and scoring.
|
||||
|
||||
## File Locations
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `environments/hermes_base_env.py` | Base class with agent loop + tool resolution |
|
||||
| `environments/agent_loop.py` | `HermesAgentLoop` + `AgentResult` dataclass |
|
||||
| `environments/tool_context.py` | `ToolContext` for reward verification |
|
||||
| `environments/tool_call_parsers.py` | Phase 2 tool call parsers (hermes, mistral, etc.) |
|
||||
| `environments/your_env.py` | Your environment implementation |
|
||||
|
||||
## Inference Setup — Ask the User First
|
||||
|
||||
**IMPORTANT:** Before running any test, evaluation, or data generation command, always ask the user how they want to handle inference. Do NOT assume OpenRouter or any specific endpoint. Present these options:
|
||||
|
||||
1. **OpenRouter** — Ask which model they want to use (e.g., `anthropic/claude-sonnet-4.5`, `google/gemini-2.5-pro`, `meta-llama/llama-3.3-70b-instruct`, etc.). Requires `OPENROUTER_API_KEY` in environment.
|
||||
2. **Self-hosted VLLM endpoint** — Ask for their base URL (e.g., `http://localhost:8000/v1`) and model name. Set `--openai.server_type vllm`.
|
||||
3. **Other OpenAI-compatible API** — Ask for the base URL, model name, and any required API key. Set `--openai.server_type openai` and `--openai.health_check false`.
|
||||
4. **Local Atropos training server** — For `serve` mode with a live training loop. Default `http://localhost:8000/v1`.
|
||||
|
||||
Once the user tells you their setup, use those values in all CLI commands for that session. Example prompts:
|
||||
|
||||
> "Before I run this, how would you like to handle inference?
|
||||
> 1. OpenRouter (I'll need your preferred model, e.g. claude-sonnet-4.5)
|
||||
> 2. A self-hosted VLLM endpoint (give me the URL and model name)
|
||||
> 3. Another OpenAI-compatible API (give me the URL, model, and any auth details)
|
||||
> 4. Local Atropos training server (serve mode)"
|
||||
|
||||
### Key flags by provider:
|
||||
|
||||
| Provider | `--openai.server_type` | `--openai.health_check` | `--openai.api_key` |
|
||||
|----------|----------------------|------------------------|-------------------|
|
||||
| OpenRouter | `openai` | `false` | `$OPENROUTER_API_KEY` |
|
||||
| VLLM (self-hosted) | `vllm` | (default) | (not needed) |
|
||||
| Other OpenAI-compatible | `openai` | `false` | As needed |
|
||||
| Local Atropos | (default) | (default) | (not needed) |
|
||||
|
||||
## Required Methods
|
||||
|
||||
### 1. `setup()` — Load dataset and initialize state
|
||||
|
||||
```python
|
||||
async def setup(self) -> None:
|
||||
"""Called once at startup. Load datasets, initialize state."""
|
||||
# Try HuggingFace first, fallback to built-in samples
|
||||
try:
|
||||
from datasets import load_dataset
|
||||
ds = load_dataset("your/dataset", split="test")
|
||||
self._items = [...]
|
||||
except Exception:
|
||||
self._items = BUILTIN_SAMPLES
|
||||
|
||||
# Always split into train/eval
|
||||
random.shuffle(self._items)
|
||||
eval_size = max(20, int(len(self._items) * 0.1))
|
||||
self._eval_items = self._items[:eval_size]
|
||||
self._items = self._items[eval_size:]
|
||||
```
|
||||
|
||||
### 2. `get_next_item()` — Return next training item
|
||||
|
||||
```python
|
||||
async def get_next_item(self) -> dict:
|
||||
"""Return next item, cycling through dataset."""
|
||||
item = self._items[self._index % len(self._items)]
|
||||
self._index += 1
|
||||
return item
|
||||
```
|
||||
|
||||
### 3. `format_prompt(item)` — Convert item to user message
|
||||
|
||||
```python
|
||||
def format_prompt(self, item: dict) -> str:
|
||||
"""Convert a dataset item into the user-facing prompt."""
|
||||
return f"Research this question: {item['question']}"
|
||||
```
|
||||
|
||||
### 4. `compute_reward(item, result, ctx)` — Score the rollout
|
||||
|
||||
**CRITICAL**: `result` is an `AgentResult`, NOT a dict. It has these attributes:
|
||||
- `result.messages` — List of message dicts (OpenAI format)
|
||||
- `result.turns_used` — Number of LLM calls made
|
||||
- `result.finished_naturally` — True if model stopped voluntarily
|
||||
- `result.tool_errors` — List of ToolError objects
|
||||
|
||||
**AgentResult does NOT have**: `final_response`, `tool_calls`, `tools_used`.
|
||||
You must extract these from `result.messages`:
|
||||
|
||||
```python
|
||||
async def compute_reward(self, item, result: AgentResult, ctx: ToolContext) -> float:
|
||||
# Extract final response (last assistant message with content)
|
||||
final_response = ""
|
||||
tools_used = []
|
||||
for msg in reversed(result.messages):
|
||||
if msg.get("role") == "assistant" and msg.get("content") and not final_response:
|
||||
final_response = msg["content"]
|
||||
if msg.get("role") == "assistant" and msg.get("tool_calls"):
|
||||
for tc in msg["tool_calls"]:
|
||||
fn = tc.get("function", {}) if isinstance(tc, dict) else {}
|
||||
name = fn.get("name", "")
|
||||
if name:
|
||||
tools_used.append(name)
|
||||
|
||||
# Score using LLM judge, heuristic, or ToolContext verification
|
||||
correctness = await self._llm_judge(item, final_response)
|
||||
return correctness
|
||||
```
|
||||
|
||||
`ctx` (ToolContext) gives you terminal/file access to the agent's sandbox for verification:
|
||||
```python
|
||||
# Run tests in the agent's sandbox
|
||||
result = ctx.terminal("pytest /workspace/test.py")
|
||||
return 1.0 if result["exit_code"] == 0 else 0.0
|
||||
```
|
||||
|
||||
### 5. `evaluate()` — Periodic evaluation with full agent loop
|
||||
|
||||
**MUST use the full agent loop with tools**, not single-turn chat_completion.
|
||||
The whole point of hermes-agent environments is agentic evaluation:
|
||||
|
||||
```python
|
||||
async def evaluate(self, *args, **kwargs) -> None:
|
||||
import time, uuid
|
||||
from environments.agent_loop import HermesAgentLoop
|
||||
from environments.tool_context import ToolContext
|
||||
|
||||
start_time = time.time()
|
||||
tools, valid_names = self._resolve_tools_for_group()
|
||||
samples = []
|
||||
|
||||
for item in self._eval_items[:self.config.eval_size]:
|
||||
task_id = str(uuid.uuid4())
|
||||
messages = []
|
||||
if self.config.system_prompt:
|
||||
messages.append({"role": "system", "content": self.config.system_prompt})
|
||||
messages.append({"role": "user", "content": self.format_prompt(item)})
|
||||
|
||||
agent = HermesAgentLoop(
|
||||
server=self.server,
|
||||
tool_schemas=tools,
|
||||
valid_tool_names=valid_names,
|
||||
max_turns=self.config.max_agent_turns,
|
||||
task_id=task_id,
|
||||
temperature=0.0, # Deterministic for eval
|
||||
max_tokens=self.config.max_token_length,
|
||||
extra_body=self.config.extra_body,
|
||||
)
|
||||
result = await agent.run(messages)
|
||||
|
||||
ctx = ToolContext(task_id)
|
||||
try:
|
||||
reward = await self.compute_reward(item, result, ctx)
|
||||
finally:
|
||||
ctx.cleanup()
|
||||
|
||||
samples.append({"prompt": ..., "response": ..., "reward": reward})
|
||||
|
||||
eval_metrics = {"eval/mean_reward": ...}
|
||||
await self.evaluate_log(metrics=eval_metrics, samples=samples,
|
||||
start_time=start_time, end_time=time.time())
|
||||
```
|
||||
|
||||
### 6. `wandb_log()` — Custom metrics logging
|
||||
|
||||
Always call `super().wandb_log()` at the end:
|
||||
|
||||
```python
|
||||
async def wandb_log(self, wandb_metrics=None):
|
||||
if wandb_metrics is None:
|
||||
wandb_metrics = {}
|
||||
if self._reward_buffer:
|
||||
n = len(self._reward_buffer)
|
||||
wandb_metrics["train/mean_reward"] = sum(self._reward_buffer) / n
|
||||
self._reward_buffer.clear()
|
||||
await super().wandb_log(wandb_metrics) # MUST call super
|
||||
```
|
||||
|
||||
**Pitfall**: `compute_reward` appends to metric buffers. During eval, this pollutes training metrics. Roll back buffer entries added during eval.
|
||||
|
||||
## Config Class
|
||||
|
||||
Always create a custom config subclass with Pydantic Field descriptors. Key inherited fields you can tune: `enabled_toolsets`, `max_agent_turns`, `agent_temperature`, `system_prompt`, `terminal_backend`, `group_size`, `steps_per_eval`, `total_steps`.
|
||||
|
||||
## config_init() — Default Configuration
|
||||
|
||||
Classmethod returning `(YourEnvConfig, [APIServerConfig(...)])`. Set server_type to "openai" for OpenRouter/external APIs. Load API key from environment variable.
|
||||
|
||||
## Three CLI Modes
|
||||
|
||||
```bash
|
||||
# SERVE — Full training loop (connects to Atropos API server)
|
||||
python environments/my_env.py serve --openai.base_url http://localhost:8000/v1
|
||||
|
||||
# PROCESS — Offline data generation (saves JSONL)
|
||||
python environments/my_env.py process --env.total_steps 10 --env.group_size 1 \
|
||||
--env.use_wandb false --env.data_path_to_save_groups output.jsonl \
|
||||
--openai.base_url "<USER_BASE_URL>" \
|
||||
--openai.model_name "<USER_MODEL>" \
|
||||
--openai.server_type <USER_SERVER_TYPE> --openai.health_check false
|
||||
|
||||
# EVALUATE — Standalone eval (runs setup + evaluate only)
|
||||
python environments/my_env.py evaluate --env.eval_size 20 \
|
||||
--env.data_dir_to_save_evals /tmp/eval_results \
|
||||
--openai.base_url "<USER_BASE_URL>" \
|
||||
--openai.model_name "<USER_MODEL>" \
|
||||
--openai.server_type <USER_SERVER_TYPE> --openai.health_check false
|
||||
```
|
||||
|
||||
Config priority: CLI args > YAML file > config_init() defaults.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **AgentResult has .messages, not .final_response** — Extract the final response by iterating reversed(result.messages) looking for the last assistant message with content.
|
||||
|
||||
2. **evaluate() must use HermesAgentLoop, not chat_completion** — Single-turn chat_completion has no tools. The whole point of hermes-agent benchmarks is agentic evaluation with tool use.
|
||||
|
||||
3. **Don't call _llm_judge twice** — If compute_reward already calls it, extract the score from the buffer instead of calling judge separately in evaluate().
|
||||
|
||||
4. **Eval pollutes training buffers** — compute_reward appends to metric buffers. During eval, roll back buffer entries to keep training metrics clean.
|
||||
|
||||
5. **Always set health_check=false for OpenRouter** — OpenRouter has no /health endpoint.
|
||||
|
||||
6. **Set data_dir_to_save_evals in evaluate mode** — Without it, results aren't saved.
|
||||
|
||||
7. **default_toolsets class variable vs enabled_toolsets config** — The class variable is a hint; the config field is what actually controls tool resolution.
|
||||
|
||||
8. **Tool call parsing in messages** — Tool calls are dicts with `{"function": {"name": ..., "arguments": ...}}`. Always check `isinstance(tc, dict)`.
|
||||
|
||||
9. **ToolContext.cleanup()** — Always call in a finally block to release sandbox resources.
|
||||
|
||||
10. **server_type must be "openai" for external APIs** — Without it, Atropos assumes a local VLLM server.
|
||||
|
||||
11. **Always ask the user for their inference setup** — Never hardcode or assume a specific provider/model. See the "Inference Setup" section above.
|
||||
|
||||
## Reward Function Patterns
|
||||
|
||||
### LLM Judge (for open-ended tasks)
|
||||
Use `self.server.chat_completion()` with a scoring prompt. Parse JSON response for score float. Always include a heuristic fallback (keyword overlap) for when the judge call fails.
|
||||
|
||||
### Binary Verification (for code/terminal tasks)
|
||||
Use `ctx.terminal("pytest test.py -q")` to run tests in the agent's sandbox. Return 1.0 for pass, 0.0 for fail.
|
||||
|
||||
### Multi-Signal (combine multiple indicators)
|
||||
Weight correctness (0.6) + tool usage (0.2) + efficiency (0.2) + optional bonuses. Clamp to [0, 1].
|
||||
|
||||
## Testing Your Environment
|
||||
|
||||
1. **Import test**: `python -c "from environments.my_env import MyEnv; print('OK')"`
|
||||
2. **Ask the user for inference setup** (see "Inference Setup" section above)
|
||||
3. **Process mode** (1 item): Verify JSONL output has valid tokens, masks, scores
|
||||
4. **Evaluate mode**: Verify full agent loop runs with tools, metrics logged correctly
|
||||
5. **Check reward range**: Scores should be in [0, 1], not all identical
|
||||
|
||||
## Minimum Implementation Checklist
|
||||
|
||||
```python
|
||||
class MyEnv(HermesAgentBaseEnv):
|
||||
name = "my-env"
|
||||
env_config_cls = MyEnvConfig
|
||||
|
||||
@classmethod
|
||||
def config_init(cls): ... # Default server + env config
|
||||
async def setup(self): ... # Load dataset + train/eval split
|
||||
async def get_next_item(self): ... # Cycle through training items
|
||||
def format_prompt(self, item): ... # Item → user message string
|
||||
async def compute_reward(self, item, result, ctx): ... # Score rollout
|
||||
async def evaluate(self, *args, **kwargs): ... # Full agent loop eval
|
||||
async def wandb_log(self, metrics=None): ... # Custom metrics + super()
|
||||
|
||||
if __name__ == "__main__":
|
||||
MyEnv.cli()
|
||||
```
|
||||
Reference in New Issue
Block a user