fix(streaming): stop duplicating tool-call args from cumulative-resend providers (#35718)

DeepSeek / Baidu Qianfan stream tool-call arguments in cumulative mode:
each chunk resends the full arguments-so-far instead of the new fragment.
The stream accumulator blindly concatenated arg deltas with +=, turning
that into '{...}{...}{...}', which failed json.loads and got nuked to '{}'
— a silently corrupted tool call (#35592). Worse on multi-param tools
(search_files, session_search, memory replace) because longer args take
more chunks, giving more resend opportunities.

- Per-slot cumulative latch in the stream accumulator: a delta that is a
  strict superset of the accumulated buffer marks the slot cumulative and
  replaces (not appends); exact duplicates are dropped only after latching.
  Incremental fragments are untouched (default += path).
- Backstop _collapse_repeated_json_arguments() in the repair pipeline
  collapses pure identical-resend buffers (K exact repeats of a valid-JSON
  unit) for providers that resend the complete object from chunk 1. Only
  reached after json.loads already failed, so compliant single objects are
  never touched.

Not a gateway or DeepSeek-model bug — any OpenAI-wire provider in
cumulative streaming mode is affected.
This commit is contained in:
Teknium
2026-05-31 00:19:39 -07:00
committed by GitHub
parent 0ffbcbbe7d
commit ca03486b6a
4 changed files with 295 additions and 1 deletions
+44 -1
View File
@@ -1750,6 +1750,12 @@ def interruptible_streaming_api_call(agent, api_kwargs: dict, *, on_first_delta=
# call starting at the same index and redirect it to a fresh slot.
_last_id_at_idx: dict = {} # raw_index -> last seen non-empty id
_active_slot_by_idx: dict = {} # raw_index -> current slot in tool_calls_acc
# Per-slot latch: set once a slot is positively identified as a
# cumulative-resend stream (a delta that is a strict superset of the
# accumulated buffer). Until latched, deltas are appended normally;
# after latching, the buffer is replaced and exact-duplicate deltas
# are dropped. See the argument-accumulation block below (#35592).
_cumulative_args_slot: set = set()
finish_reason = None
model_name = None
role = "assistant"
@@ -1867,7 +1873,44 @@ def interruptible_streaming_api_call(agent, api_kwargs: dict, *, on_first_delta=
# Vercel AI patterns) is immune to this.
entry["function"]["name"] = tc_delta.function.name
if tc_delta.function.arguments:
entry["function"]["arguments"] += tc_delta.function.arguments
# Argument deltas are normally incremental
# fragments (OpenAI spec), so the default is to
# concatenate. But some OpenAI-compatible
# providers (DeepSeek / Baidu Qianfan, #35592)
# operate in *cumulative* mode: each chunk
# resends the full arguments-so-far instead of
# the new fragment. Blind += turns that into
# '{...}{...}{...}', corrupting the tool call.
#
# Detect cumulative mode per-slot: in cumulative
# mode the new delta is a superset that starts
# with everything accumulated so far (monotonic
# growth), and an exact resend equals it.
# Incremental fragments are JSON suffixes that do
# NOT restate the accumulated prefix, so this is
# unambiguous on the full buffer (not a partial
# per-chunk guess).
_new = tc_delta.function.arguments
_prev = entry["function"]["arguments"]
if not _prev:
entry["function"]["arguments"] = _new
elif len(_new) > len(_prev) and _new.startswith(_prev):
# Strict superset of the accumulated buffer —
# the unambiguous cumulative-resend signature.
# Latch the slot and replace (don't append).
_cumulative_args_slot.add(idx)
entry["function"]["arguments"] = _new
elif idx in _cumulative_args_slot and _new == _prev:
# Already a confirmed cumulative slot and this
# is an exact full resend — drop the duplicate.
pass
else:
# Incremental fragment — normal append. Note
# an exact-equal delta on a NON-latched slot is
# treated as a real fragment, never silently
# dropped, so genuine incremental streams are
# untouched.
entry["function"]["arguments"] = _prev + _new
extra = getattr(tc_delta, "extra_content", None)
if extra is None and hasattr(tc_delta, "model_extra"):
extra = (tc_delta.model_extra or {}).get("extra_content")