fix(bedrock): forward anthropic-beta header to LiteLLM (1M context)#1484
fix(bedrock): forward anthropic-beta header to LiteLLM (1M context)#1484KennethWKZ wants to merge 1 commit into
Conversation
The client's anthropic-beta header (notably context-1m-2025-08-07) was dropped before the LiteLLM acompletion call. Headroom's LiteLLM backend extracts only auth (x-api-key/authorization) from the incoming headers, so beta features negotiated by the client never reached the upstream. LiteLLM's Bedrock converse transform reads beta features from the headers kwarg (via litellm.anthropic_beta_headers_manager) and context-1m-2025-08-07 is in the bedrock_converse allowlist, so the header would be forwarded to Bedrock if it reached LiteLLM — but without this change it never does, and the 1M window silently caps at 200k. Add _apply_anthropic_beta() and call it before all four acompletion sites (send_message, stream_message, send_openai_message, stream_openai_message). LiteLLM filters provider-unsupported betas, so forwarding the raw value is safe.
PR governanceThis PR does not yet satisfy the required template fields:
Please update the PR body, or move the PR back to draft while it is still in progress. |
|
Closing — verified against the official Claude Platform docs that Opus 4.8 has the 1M-token context window by default (no The underlying header-forwarding gap (Bedrock litellm path drops the client |
Description
The client's
anthropic-betaheader (notablycontext-1m-2025-08-07) is dropped before the LiteLLMacompletioncall, so the 1M context window silently caps at 200k on the Bedrock path.Headroom's LiteLLM backend extracts only auth (
x-api-key/authorization) from the incoming request headers. Beta features negotiated by the client never reach the upstream.Root cause (verified against LiteLLM source)
LiteLLM does support forwarding
context-1mto Bedrock:context-1m-2025-08-07is in LiteLLM's per-provider allowlist (anthropic_beta_headers_config.json,bedrock_conversesection).headers/extra_headerskwarg vialitellm.anthropic_beta_headers_manager.Without this change → header dropped → 1M never engages (200k cap).
With this change → header reaches LiteLLM → allowed for
bedrock_converse→ forwarded to Bedrock.Type of Change
Changes Made
_apply_anthropic_beta(kwargs, headers)inheadroom/backends/litellm.py.acompletionsites:send_message,stream_message,send_openai_message,stream_openai_message.TestApplyAnthropicBeta(4 cases: forward, no-op, preserve-existing, title-case key).Scoped to the beta-forwarding fix only. ARN→
bedrock/converse/routing and named-profile forwarding are handled in #1456; this is complementary and touches neither.Testing
```text
$ uv run --extra dev --with litellm pytest tests/test_bedrock_region.py -q
31 passed in 1.36s
$ uv run --extra dev --with litellm ruff check headroom/backends/litellm.py tests/test_bedrock_region.py
All checks passed!
```
Checklist