Skip to content

fix(bedrock): forward anthropic-beta header to LiteLLM (1M context)#1484

Closed
KennethWKZ wants to merge 1 commit into
headroomlabs-ai:mainfrom
KennethWKZ:fix/bedrock-anthropic-beta-forwarding
Closed

fix(bedrock): forward anthropic-beta header to LiteLLM (1M context)#1484
KennethWKZ wants to merge 1 commit into
headroomlabs-ai:mainfrom
KennethWKZ:fix/bedrock-anthropic-beta-forwarding

Conversation

@KennethWKZ

Copy link
Copy Markdown
Contributor

Description

The client's anthropic-beta header (notably context-1m-2025-08-07) is dropped before the LiteLLM acompletion call, so the 1M context window silently caps at 200k on the Bedrock path.

Headroom's LiteLLM backend extracts only auth (x-api-key/authorization) from the incoming request headers. Beta features negotiated by the client never reach the upstream.

Root cause (verified against LiteLLM source)

LiteLLM does support forwarding context-1m to Bedrock:

  1. context-1m-2025-08-07 is in LiteLLM's per-provider allowlist (anthropic_beta_headers_config.json, bedrock_converse section).
  2. LiteLLM's Bedrock converse transform reads betas from the headers/extra_headers kwarg via litellm.anthropic_beta_headers_manager.
  3. But Headroom never populates that kwarg — so the header is lost before LiteLLM sees it.

Without this change → header dropped → 1M never engages (200k cap).
With this change → header reaches LiteLLM → allowed for bedrock_converse → forwarded to Bedrock.

Note: not yet physically exercised with a >200k-token request, but the full path is traced and confirmed against LiteLLM's real allowlist. Independent of model-id routing — works for short IDs and ARNs alike.

Type of Change

  • Bug fix (non-breaking)

Changes Made

  • Add _apply_anthropic_beta(kwargs, headers) in headroom/backends/litellm.py.
  • Call it before all four acompletion sites: send_message, stream_message, send_openai_message, stream_openai_message.
  • LiteLLM filters provider-unsupported betas, so forwarding the raw header value is safe.
  • Add TestApplyAnthropicBeta (4 cases: forward, no-op, preserve-existing, title-case key).

Scoped to the beta-forwarding fix only. ARN→bedrock/converse/ routing and named-profile forwarding are handled in #1456; this is complementary and touches neither.

Testing

```text
$ uv run --extra dev --with litellm pytest tests/test_bedrock_region.py -q
31 passed in 1.36s

$ uv run --extra dev --with litellm ruff check headroom/backends/litellm.py tests/test_bedrock_region.py
All checks passed!
```

Checklist

  • Self-review performed
  • New unit tests added
  • New + existing unit tests pass locally
  • Linting passes

The client's anthropic-beta header (notably context-1m-2025-08-07) was
dropped before the LiteLLM acompletion call. Headroom's LiteLLM backend
extracts only auth (x-api-key/authorization) from the incoming headers, so
beta features negotiated by the client never reached the upstream.

LiteLLM's Bedrock converse transform reads beta features from the headers
kwarg (via litellm.anthropic_beta_headers_manager) and context-1m-2025-08-07
is in the bedrock_converse allowlist, so the header would be forwarded to
Bedrock if it reached LiteLLM — but without this change it never does, and
the 1M window silently caps at 200k.

Add _apply_anthropic_beta() and call it before all four acompletion sites
(send_message, stream_message, send_openai_message, stream_openai_message).
LiteLLM filters provider-unsupported betas, so forwarding the raw value is
safe.
@github-actions

Copy link
Copy Markdown
Contributor

PR governance

This PR does not yet satisfy the required template fields:

  • Missing required section Real Behavior Proof.
  • Missing required section Review Readiness.
  • Check at least one verification item in Testing.
  • Paste real command output or artifact links in TestingTest Output.
  • Check I have performed a self-review before requesting human review.
  • Check This PR is ready for human review or convert the PR back to draft.

Please update the PR body, or move the PR back to draft while it is still in progress.

@KennethWKZ

Copy link
Copy Markdown
Contributor Author

Closing — verified against the official Claude Platform docs that Opus 4.8 has the 1M-token context window by default (no context-1m beta header required): https://platform.claude.com/docs/en/about-claude/models/whats-new-claude-4-8 . The 1M motivation for this change is therefore moot on the current model.

The underlying header-forwarding gap (Bedrock litellm path drops the client anthropic-beta header, which also carries compaction / context-management betas) may still be worth addressing separately if compaction misbehaves on Bedrock, but that's a distinct, unverified concern — not the 1M fix this PR was framed around.

@KennethWKZ KennethWKZ closed this Jun 27, 2026
@KennethWKZ KennethWKZ deleted the fix/bedrock-anthropic-beta-forwarding branch June 27, 2026 02:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: needs author action Pull request body or readiness checklist still needs author updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant