fix(model_metadata): consult DEFAULT_CONTEXT_LENGTHS before 256K fallback on custom endpoints
Problem: get_model_context_length() had an early return at the end of the custom-endpoint probe branch (step 3) that returned DEFAULT_FALLBACK_CONTEXT (256K) without ever consulting the hardcoded DEFAULT_CONTEXT_LENGTHS catalog (step 8). Models served through a custom/proxied gateway (e.g. corporate Anthropic proxy) that didn't expose Ollama or local-server endpoints would hit this path and get capped at 256K, even when the model name clearly matched a known entry in the catalog (e.g. claude-opus-4-8 → 1M). Changes: - agent/model_metadata.py: Before returning DEFAULT_FALLBACK_CONTEXT at the end of the custom-endpoint branch, consult DEFAULT_CONTEXT_LENGTHS using the same longest-key-first fuzzy matching as step 8. Only fall through to 256K if no catalog entry matches. - tests/agent/test_model_metadata.py: Updated existing test and added new test covering the custom-endpoint → catalog fallback behavior. Fixes #38865
This commit is contained in:
@@ -1684,6 +1684,26 @@ def get_model_context_length(
|
||||
"in config.yaml to override.",
|
||||
model, base_url, f"{DEFAULT_FALLBACK_CONTEXT:,}",
|
||||
)
|
||||
# 3b. Before falling back to the hard 256K default, consult the
|
||||
# hardcoded catalog as a last resort. A proxied/custom Anthropic
|
||||
# gateway (e.g. corporate proxy) fails the Ollama/local probes
|
||||
# above, but the model name may still match an entry in
|
||||
# DEFAULT_CONTEXT_LENGTHS (e.g. "claude-opus-4-8" → 1M).
|
||||
# Without this, the early return here short-circuits the catalog
|
||||
# lookup at step 8 and silently caps context at 256K.
|
||||
model_lower = model.lower()
|
||||
for default_model, length in sorted(
|
||||
DEFAULT_CONTEXT_LENGTHS.items(),
|
||||
key=lambda x: len(x[0]),
|
||||
reverse=True,
|
||||
):
|
||||
if default_model in model_lower:
|
||||
logger.info(
|
||||
"Using hardcoded context length %s for model %r "
|
||||
"(custom endpoint, catalog match on %r)",
|
||||
f"{length:,}", model, default_model,
|
||||
)
|
||||
return length
|
||||
return DEFAULT_FALLBACK_CONTEXT
|
||||
|
||||
# 4. Anthropic /v1/models API (only for regular API keys, not OAuth)
|
||||
|
||||
Reference in New Issue
Block a user