fix(agent): surface model refusals instead of retrying them as errors

A Claude refusal (HTTP 200, stop_reason="refusal", empty content) was
laundered into a generic retry loop and surfaced as a misleading
"rate limited / invalid response" or "no content after retries" error,
burning paid attempts reproducing a deterministic refusal.

This hit two distinct paths:

- Direct Anthropic (anthropic_messages): validate_response rejected the
  empty-content refusal *before* normalize_response mapped refusal ->
  content_filter, so it fell into the invalid-response retry loop.
- Nous Portal / OpenAI-compatible (chat_completions): the portal surfaces
  a Claude refusal via message.refusal with empty content, which sailed
  past validation and died in the empty-response retry loop.

Fix (one unified content_filter dispatch for all backends):
- AnthropicTransport.validate_response: accept empty content when
  stop_reason == "refusal" so it flows to normalize_response.
- ChatCompletionsTransport.normalize_response: promote message.refusal to
  content + a content_filter finish reason.
- conversation_loop: handle finish_reason == "content_filter" - fire the
  api_request_error hook (content_policy_blocked), try a configured
  fallback once, else return a clear terminal refusal message. Never retry
  a deterministic refusal.

Supersedes #43084, which fixed only the direct-Anthropic path and could
not reach the chat_completions/portal path.

Tests: transport-level (validate_response refusal, message.refusal
promotion) + end-to-end loop (refusal surfaced, exactly one API call).

(cherry picked from commit 01f546f92c)
This commit is contained in:
SHL0MS
2026-06-14 12:10:08 +05:30
committed by kshitijk4poor
parent 4b5ba112ad
commit bb46bf8ce4
6 changed files with 276 additions and 6 deletions
+41
View File
@@ -4660,6 +4660,47 @@ class TestRetryExhaustion:
assert "error" in result
assert "Invalid API response" in result["error"]
def test_content_filter_refusal_surfaced_not_retried(self, agent):
"""A model refusal must be surfaced immediately, NOT laundered into
the empty-response retry loop and reported as "rate limited" / "no
content after retries".
Regression: running a Claude refusal through an OpenAI-compatible
portal (Nous Portal fronting Anthropic) returns ``message.refusal``
with empty content. The transport now promotes that to a
``content_filter`` finish reason and the loop surfaces it as a terminal
``content_policy_blocked`` result instead of retrying a deterministic
refusal three times.
"""
self._setup_agent(agent)
refusal_resp = SimpleNamespace(
choices=[SimpleNamespace(
message=SimpleNamespace(
content=None, tool_calls=None, reasoning=None,
reasoning_content=None, refusal="I won't help with that.",
),
finish_reason="stop",
)],
model="test/model",
usage=None,
id="resp_1",
)
agent.client.chat.completions.create.return_value = refusal_resp
with (
patch.object(agent, "_persist_session"),
patch.object(agent, "_save_trajectory"),
patch.object(agent, "_cleanup_task_resources"),
):
result = agent.run_conversation("please do something disallowed")
assert result.get("completed") is False
assert result.get("failed") is True
assert "content_policy_blocked" in result.get("error", "")
# The model's refusal text is surfaced to the user, not swallowed.
assert "I won't help with that." in (result.get("final_response") or "")
# Crucial regression guard: a deterministic refusal is NOT retried —
# exactly one API call, no empty-response retry loop.
assert agent.client.chat.completions.create.call_count == 1
def test_api_error_returns_gracefully_after_retries(self, agent):
"""Exhausted retries on API errors must return error result, not crash."""
self._setup_agent(agent)