fix(agent): stop reporting broken streams as output-length truncation (#36705)

A stream that drops mid-response after tokens are delivered (peer-closed
connection, stale-stream reconnect) is converted into a synthetic
finish_reason="length" stub. The conversation loop treated that network
stall as a max-output-tokens truncation: when the dropped content was a
tool call it retried exactly once, then hard-failed with "Response
truncated due to output length limit" — even on large-output models that
never hit any cap (e.g. Opus).

- Tool-call truncation now retries up to 3 times (was 1) with a
  progressive max_tokens boost, and is stub-aware: a PARTIAL_STREAM_STUB_ID
  stall prints "Stream interrupted mid tool-call — retrying (n/3)" instead
  of the false "model hit max output tokens", and the give-up message
  distinguishes a network drop from a real truncation.
- Length-continuation retries preserve the original request's output cap
  as a floor, so a high provider/model default isn't silently downshifted
  to 8K/12K on retry.
- Added _requested_output_cap_from_api_kwargs() helper.

Tests: stub-stall mid-tool-call recovery within 3 retries; continuation
preserves a large provider-default output cap.

Fixes #26425. Salvages the substance of #26427 (cap floor) and #9525
(retry bump), adapted to the post-refactor conversation_loop.py which
handles all three api_modes uniformly.

Co-authored-by: LeonSGP43 <cine.dreamer.one@gmail.com>
Co-authored-by: ygd58 <ygd58@users.noreply.github.com>
This commit is contained in:
Teknium
2026-06-01 03:01:20 -07:00
committed by GitHub
co-authored by LeonSGP43 ygd58
parent b571ec298d
commit 023149f665
3 changed files with 143 additions and 11 deletions
+15
View File
@@ -1206,6 +1206,21 @@ class AIAgent:
return {"max_completion_tokens": value}
return {"max_tokens": value}
@staticmethod
def _requested_output_cap_from_api_kwargs(api_kwargs: Any) -> Optional[int]:
"""Extract the outgoing response token cap from a prepared request."""
if not isinstance(api_kwargs, dict):
return None
for key in ("max_output_tokens", "max_completion_tokens", "max_tokens"):
raw = api_kwargs.get(key)
try:
value = int(raw)
except (TypeError, ValueError):
continue
if value > 0:
return value
return None
def _has_content_after_think_block(self, content: str) -> bool:
"""
Check if content has actual text after any reasoning/thinking blocks.