revert: drop cumulative-resend tool-arg heuristic from shared streaming path (#35718) (#35860)

PR #35718 added a per-slot "cumulative-resend" latch to the universal
streaming tool-call accumulator to fix DeepSeek / Baidu Qianfan (#35592).
The latch fires when a delta is a strict superset of the accumulated
buffer (len(_new) > len(_prev) and _new.startswith(_prev)) and then
REPLACES the buffer instead of appending.

That superset test is not an unambiguous cumulative signature. A normal
incremental stream can emit a single fragment that restates an already-
accumulated prefix — trivially common in large code-patch arguments with
repeated lines / indentation — which trips the latch and clobbers the
accumulated buffer, corrupting the tool call. Observed in the wild on
Anthropic Opus (the primary model) building a large patch: corrupted /
short arguments → finish_reason='length' dead-end → session killed.

A guessing heuristic that can silently clobber a tool-call buffer has no
place on the path every provider and model shares. Reverting restores the
known-good plain `+=` accumulator. The #35592 narrow provider bug should
be re-addressed provider-gated so it is structurally impossible to touch
Anthropic / OpenAI incremental streams, rather than via a heuristic on the
shared path.

Reverts ca03486b6.
This commit is contained in:
Teknium
2026-05-31 06:14:32 -07:00
committed by GitHub
parent f2d4cf4f76
commit 2b5268f716
4 changed files with 1 additions and 295 deletions
+1 -44
View File
@@ -1750,12 +1750,6 @@ def interruptible_streaming_api_call(agent, api_kwargs: dict, *, on_first_delta=
# call starting at the same index and redirect it to a fresh slot.
_last_id_at_idx: dict = {} # raw_index -> last seen non-empty id
_active_slot_by_idx: dict = {} # raw_index -> current slot in tool_calls_acc
# Per-slot latch: set once a slot is positively identified as a
# cumulative-resend stream (a delta that is a strict superset of the
# accumulated buffer). Until latched, deltas are appended normally;
# after latching, the buffer is replaced and exact-duplicate deltas
# are dropped. See the argument-accumulation block below (#35592).
_cumulative_args_slot: set = set()
finish_reason = None
model_name = None
role = "assistant"
@@ -1873,44 +1867,7 @@ def interruptible_streaming_api_call(agent, api_kwargs: dict, *, on_first_delta=
# Vercel AI patterns) is immune to this.
entry["function"]["name"] = tc_delta.function.name
if tc_delta.function.arguments:
# Argument deltas are normally incremental
# fragments (OpenAI spec), so the default is to
# concatenate. But some OpenAI-compatible
# providers (DeepSeek / Baidu Qianfan, #35592)
# operate in *cumulative* mode: each chunk
# resends the full arguments-so-far instead of
# the new fragment. Blind += turns that into
# '{...}{...}{...}', corrupting the tool call.
#
# Detect cumulative mode per-slot: in cumulative
# mode the new delta is a superset that starts
# with everything accumulated so far (monotonic
# growth), and an exact resend equals it.
# Incremental fragments are JSON suffixes that do
# NOT restate the accumulated prefix, so this is
# unambiguous on the full buffer (not a partial
# per-chunk guess).
_new = tc_delta.function.arguments
_prev = entry["function"]["arguments"]
if not _prev:
entry["function"]["arguments"] = _new
elif len(_new) > len(_prev) and _new.startswith(_prev):
# Strict superset of the accumulated buffer —
# the unambiguous cumulative-resend signature.
# Latch the slot and replace (don't append).
_cumulative_args_slot.add(idx)
entry["function"]["arguments"] = _new
elif idx in _cumulative_args_slot and _new == _prev:
# Already a confirmed cumulative slot and this
# is an exact full resend — drop the duplicate.
pass
else:
# Incremental fragment — normal append. Note
# an exact-equal delta on a NON-latched slot is
# treated as a real fragment, never silently
# dropped, so genuine incremental streams are
# untouched.
entry["function"]["arguments"] = _prev + _new
entry["function"]["arguments"] += tc_delta.function.arguments
extra = getattr(tc_delta, "extra_content", None)
if extra is None and hasattr(tc_delta, "model_extra"):
extra = (tc_delta.model_extra or {}).get("extra_content")