fix(gemini): default native maxOutputTokens + strip OpenAI extra_body on Gemini endpoints (#39730)

* fix: respect disabled auto-compaction on context overflow

Port from anomalyco/opencode#30749.

When compression.enabled is false, NO automatic compaction trigger may
fire. The proactive token-threshold paths (preflight + post-response
should_compress gate) already honoured the setting, but the three
provider-overflow recovery paths in the agent loop — long-context-tier
429, 413 payload-too-large, and context-overflow — called
_compress_context() unconditionally, silently compressing and rotating
the session against the user's explicit choice.

Add a single guard at the top of the overflow-recovery dispatch: when
compression is disabled and the error is one of those three overflow
classes, surface a terminal error (compaction_disabled: True) telling the
user to /compress manually, /new, switch to a larger-context model, or
reduce attachments. Manual /compress (force=True) is unaffected — it never
enters this loop.

Tests: new TestOverflowWithCompactionDisabled (413 + 400 overflow don't
compress when disabled; control case still compresses when enabled).
Existing overflow-recovery tests updated to enable compaction explicitly
(they verify the recovery fires); fixture defaults flipped to True to
match production (compression.enabled defaults to True).

* fix(gemini): default native maxOutputTokens + strip OpenAI extra_body on Gemini endpoints

Two distinct failures hit users on the gemini provider with only Google
AI Studio keys set.

1. Truncation loop: build_gemini_request() only set maxOutputTokens when
   max_tokens was non-None. Hermes passes None to mean "unlimited", but
   Gemini's native generateContent does NOT treat an absent maxOutputTokens
   as full budget — it applies a low internal default and stops early with
   finishReason=MAX_TOKENS, truncating tool calls. The agent then retries
   3x and refuses the incomplete call. Now default to the published 65,535
   ceiling (shared by all current Gemini text models) when max_tokens=None.

2. HTTP 400 on Gemini endpoint: the chat_completions transport assembles
   profile extra_body (Nous portal 'tags', reasoning, provider prefs) and
   sends it via the OpenAI client to whatever base_url is resolved. When a
   profile that emits extra_body (e.g. Nous) is active but the endpoint is a
   native Gemini base_url — typical when only Google creds exist and a
   fallback/aux call lands on Gemini — Google rejects the unknown 'tags'
   field with a non-retryable 400. Strip all non-thinking_config extra_body
   keys when the resolved endpoint is native Gemini.

Verified E2E against real transport code: tags stripped on native Gemini,
preserved on Nous and the /openai compat endpoint; maxOutputTokens=65535
on None, explicit values respected.
This commit is contained in:
Teknium
2026-06-05 03:53:59 -07:00
committed by GitHub
parent 6bf55a473e
commit ec46f5912e
7 changed files with 277 additions and 2 deletions
+55
View File
@@ -2720,6 +2720,61 @@ def run_conversation(
# compress history and retry, not abort immediately.
status_code = getattr(api_error, "status_code", None)
# ── Respect disabled auto-compaction on overflow ──────
# Ported from anomalyco/opencode#30749. When the user has
# turned auto-compaction off (``compression.enabled: false``),
# NO automatic compaction trigger may fire — including the
# provider/request-size overflow recovery paths below
# (long-context-tier 429, 413 payload-too-large, and
# context-overflow). Without this guard the proactive
# threshold path correctly honours the setting (see the
# preflight check and the post-response ``should_compress``
# gate) but a provider overflow error would still silently
# compress + rotate the session, bypassing the user's
# explicit choice. Surface a terminal error instead so the
# user can compact manually (``/compress``), start fresh
# (``/new``), switch to a larger-context model, or reduce
# attachments. Forced compaction via ``/compress``
# (``force=True``) is unaffected — it never reaches this loop.
_overflow_reasons = {
FailoverReason.long_context_tier,
FailoverReason.payload_too_large,
FailoverReason.context_overflow,
}
if (
classified.reason in _overflow_reasons
and not getattr(agent, "compression_enabled", True)
):
agent._flush_status_buffer()
agent._vprint(
f"{agent.log_prefix}❌ Context overflow, but auto-compaction is disabled "
f"(compression.enabled: false).",
force=True,
)
agent._vprint(
f"{agent.log_prefix} 💡 Run /compress to compact manually, /new to start fresh, "
f"switch to a larger-context model, or reduce attachments.",
force=True,
)
logger.error(
f"{agent.log_prefix}Context overflow ({classified.reason.value}) with "
f"auto-compaction disabled — not compressing."
)
agent._persist_session(messages, conversation_history)
return {
"messages": messages,
"completed": False,
"api_calls": api_call_count,
"error": (
"Context overflow and auto-compaction is disabled "
"(compression.enabled: false). Run /compress to compact manually, "
"/new to start fresh, or switch to a larger-context model."
),
"partial": True,
"failed": True,
"compaction_disabled": True,
}
# ── Anthropic Sonnet long-context tier gate ───────────
# Anthropic returns HTTP 429 "Extra usage is required for
# long context requests" when a Claude Max (or similar)
+19
View File
@@ -33,6 +33,13 @@ logger = logging.getLogger(__name__)
DEFAULT_GEMINI_BASE_URL = "https://generativelanguage.googleapis.com/v1beta"
# Published max output-token ceiling shared by every current Gemini text model
# (2.5 + 3.x: flash, flash-lite, pro). Used as the default when the caller
# passes max_tokens=None, because Gemini's native API otherwise applies a low
# internal default and truncates output (unlike OpenAI-compat endpoints where
# an omitted limit means full budget).
GEMINI_DEFAULT_MAX_OUTPUT_TOKENS = 65535
def is_native_gemini_base_url(base_url: str) -> bool:
"""Return True when the endpoint speaks Gemini's native REST API."""
@@ -414,6 +421,18 @@ def build_gemini_request(
generation_config["temperature"] = temperature
if max_tokens is not None:
generation_config["maxOutputTokens"] = max_tokens
else:
# Gemini's native generateContent does NOT treat an omitted
# maxOutputTokens as "use the model's full output budget" — it applies
# a low internal default and the model stops early with
# finishReason=MAX_TOKENS, truncating tool calls mid-stream (Hermes
# then retries 3× and refuses the incomplete call). Every current
# Gemini text model (2.5 + 3.x, flash / flash-lite / pro) caps at
# 65,535 output tokens, so default to that ceiling when the caller
# passes None ("unlimited"). See the OpenAI-compat path where omitting
# the field genuinely means full budget — that assumption does not
# hold on the native API.
generation_config["maxOutputTokens"] = GEMINI_DEFAULT_MAX_OUTPUT_TOKENS
if top_p is not None:
generation_config["topP"] = top_p
if stop:
+22 -1
View File
@@ -571,7 +571,28 @@ class ChatCompletionsTransport(ProviderTransport):
api_kwargs[k] = v
if extra_body:
api_kwargs["extra_body"] = extra_body
# Native Gemini (generativelanguage.googleapis.com, non-/openai)
# speaks Google's REST schema, not OpenAI's. OpenAI-style extra_body
# keys (tags, reasoning, provider, plugins, …) are unknown fields
# there and Gemini rejects the whole request with a non-retryable
# HTTP 400 ("Invalid JSON payload received. Unknown name 'tags'").
# This happens when a profile that emits extra_body (e.g. the Nous
# profile's portal `tags`) is active but the resolved endpoint is a
# Gemini base_url — typical when only Google credentials are set and
# a fallback/aux call lands on Gemini. The native client only reads
# thinking_config from extra_body, so drop everything else here.
try:
from agent.gemini_native_adapter import is_native_gemini_base_url
_native_gemini = is_native_gemini_base_url(params.get("base_url"))
except Exception:
_native_gemini = False
if _native_gemini:
extra_body = {
k: v for k, v in extra_body.items()
if k in ("thinking_config", "thinkingConfig")
}
if extra_body:
api_kwargs["extra_body"] = extra_body
return api_kwargs