fix(compaction): prevent infinite loop when transcript fits in tail budget

When summary_target_ratio is large (e.g. 0.45) and the context_length is
moderate (e.g. 96000), the soft_ceiling (token_budget * 1.5) can exceed
the total transcript size.  _find_tail_cut_by_tokens walks the entire
transcript without breaking early, and the resulting compress window is
either empty (compress_start >= compress_end) or a single message whose
summary-of-one overhead saves ~0 tokens.

Both outcomes cause a no-op compression that does not increment
_ineffective_compression_count, so should_compress() returns True on
every subsequent turn and the loop repeats endlessly.

Fix (two layers):
1. _find_tail_cut_by_tokens: when the backward walk consumed the entire
   transcript without breaking (cut_idx <= head_end and accumulated <=
   soft_ceiling), re-walk with the raw (non-inflated) token budget to
   find a meaningful cut that gives the summarizer a useful middle window.
2. compress(): when compress_start >= compress_end, increment
   _ineffective_compression_count and log a warning so the existing
   anti-thrashing guard in should_compress() can break the loop.

Fixes #40803
This commit is contained in:
islam666
2026-06-07 21:50:57 -07:00
committed by Teknium
parent 38d1a414a1
commit b18490b890
2 changed files with 300 additions and 0 deletions
+50
View File
@@ -1818,6 +1818,41 @@ The user has requested that this compaction PRIORITISE preserving all informatio
accumulated += msg_tokens
cut_idx = i
# If the backward walk never broke early because the entire transcript
# fits within soft_ceiling, accumulated now holds the total transcript
# size. Without intervention _ensure_last_user_message_in_tail pushes
# cut_idx forward to include the last user message, and the caller's
# compress_start >= compress_end guard either returns unchanged (no-op)
# or compresses a single message — both of which trigger the infinite
# compaction loop described in #40803.
#
# Fix: when the whole transcript fits in soft_ceiling, compute a
# meaningful cut point using the raw (non-inflated) budget so that
# compression actually summarizes a worthwhile middle section.
if cut_idx <= head_end and accumulated <= soft_ceiling and accumulated > 0:
# The entire compressable region fits in the soft ceiling.
# Re-walk with the raw budget (no 1.5x multiplier) to find a
# split that gives the summarizer something useful.
raw_budget = token_budget
raw_accumulated = 0
for j in range(n - 1, head_end - 1, -1):
raw_msg = messages[j]
raw_content = raw_msg.get("content") or ""
raw_len = _content_length_for_budget(raw_content)
raw_tok = raw_len // _CHARS_PER_TOKEN + 10
for tc in raw_msg.get("tool_calls") or []:
if isinstance(tc, dict):
args = tc.get("function", {}).get("arguments", "")
raw_tok += len(args) // _CHARS_PER_TOKEN
if raw_accumulated + raw_tok > raw_budget and (n - j) >= min_tail:
cut_idx = j
break
raw_accumulated += raw_tok
cut_idx = j
# If the raw-budget walk also consumed everything (very small
# transcript), fall through — the existing fallback logic below
# will still force a minimal cut after head_end.
# Ensure we protect at least min_tail messages
fallback_cut = n - min_tail
cut_idx = min(cut_idx, fallback_cut)
@@ -1920,6 +1955,21 @@ The user has requested that this compaction PRIORITISE preserving all informatio
compress_end = self._find_tail_cut_by_tokens(messages, compress_start)
if compress_start >= compress_end:
# No compressable window — the entire transcript fits within
# the tail budget (soft_ceiling). Without recording this as
# an ineffective compression the anti-thrashing guard in
# should_compress() never fires and every subsequent turn
# re-triggers a no-op compression loop. (#40803)
self._ineffective_compression_count += 1
self._last_compression_savings_pct = 0.0
if not self.quiet_mode:
logger.warning(
"Compression skipped: compress_start (%d) >= compress_end (%d) "
"— transcript fits within tail budget, nothing to compress. "
"ineffective_compression_count=%d",
compress_start, compress_end,
self._ineffective_compression_count,
)
return messages
turns_to_summarize = messages[compress_start:compress_end]