fix(computer-use): cap AX elements array to prevent context blowup (#22865)

`computer_use(action='capture', mode='ax')` returned the full AX element
list verbatim in the JSON response. Dense Electron / Obsidian / JetBrains
UIs publish 500+ AX nodes (one reproduction in #22865 returned 597
elements against Obsidian), so a single capture could consume enough
context to trigger compression failures or render the session unusable.
The human-readable `_format_elements` summary is already capped at 40
lines, so the truncation gap was invisible to anyone reading the summary
output.

Add a `max_elements` argument to the tool schema, default 100, that
trims the AX `elements` array. When the cap fires, the response surfaces
`total_elements` and `truncated_elements` and appends a "raise
max_elements or pass app= to narrow" hint to the summary so the model
knows the JSON view is partial and can re-issue with a tighter scope.

Validation is centralized in `_coerce_max_elements`: missing /
non-integer / sub-1 inputs fall back to the default cap, so the
protection can never be silently disabled by a malformed tool-call
argument. The cap only affects AX-mode JSON; `mode='som'` and
`mode='vision'` keep returning a screenshot + image-aware summary
unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
briandevans
2026-05-21 19:07:32 -07:00
committed by Teknium
co-authored by Claude Opus 4.7
parent 9e30ef224d
commit bb694bad42
3 changed files with 168 additions and 7 deletions
+17
View File
@@ -75,6 +75,23 @@ COMPUTER_USE_SCHEMA: Dict[str, Any] = {
"frontmost app's window or the whole screen."
),
},
"max_elements": {
"type": "integer",
"description": (
"Optional cap on the AX `elements` array returned by "
"`action='capture'`. Default 100. Dense UIs (Electron "
"apps such as Obsidian or VS Code, JetBrains IDEs) can "
"publish 500+ AX nodes — capping prevents a single "
"capture from blowing session context. When the cap "
"trims the response, `total_elements` and "
"`truncated_elements` are surfaced in the result so "
"you can re-call with `app=` to narrow scope or raise "
"`max_elements` when the full tree is required. Has no "
"effect on `mode='som'` / `mode='vision'` (those return "
"a screenshot, not the elements array)."
),
"minimum": 1,
},
# ── click / drag / scroll targeting ────────────────────
"element": {
"type": "integer",