Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions src/oss/python/integrations/chat/openai.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1887,6 +1887,79 @@ response1 = llm.invoke(messages)
response2 = llm.invoke(messages, prompt_cache_key="override-cache-v1")
```

### Explicit caching with breakpoints (Responses API)

<Note>
Requires `langchain-openai>=1.3.5` and `openai>=2.45.0`. Only supported when using the [Responses API](/oss/python/integrations/chat/openai#responses-api).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Requires `langchain-openai>=1.3.5` and `openai>=2.45.0`. Only supported when using the [Responses API](/oss/python/integrations/chat/openai#responses-api).
Requires `langchain-openai>=1.3.5`. Only supported when using the [Responses API](/oss/python/integrations/chat/openai#responses-api).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is done transitively

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only supported when using the Responses API.

need to verify this claim

</Note>

OpenAI's Responses API supports [explicit prompt-cache breakpoints](https://developers.openai.com/api/docs/guides/prompt-caching?prompt-cache-api=responses#prompt-cache-breakpoints), which let you designate specific content blocks as cache boundaries. This gives you fine-grained control over which parts of a prompt are cached, rather than relying solely on automatic prefix caching.

To mark a content block as a cache breakpoint, add `"prompt_cache_breakpoint": True` to the block:

```python
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4.1", use_responses_api=True)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
llm = ChatOpenAI(model="gpt-4.1", use_responses_api=True)
llm = ChatOpenAI(model="gpt-5.6-sol", use_responses_api=True)

need to verify model string

don't think 4.1 supports


messages = [
{
"role": "system",
"content": [
{
"type": "text",
"text": "You are a helpful assistant with access to a large knowledge base.",
"prompt_cache_breakpoint": True, # [!code highlight]
}
],
},
{"role": "user", "content": "Summarize the key points."},
]

response = llm.invoke(messages)
```

Breakpoints are supported on text, image, and file content blocks. You can also nest `prompt_cache_breakpoint` inside an `extras` dict if you prefer to keep the LangChain content block structure clean:

```python
content_block = {
"type": "text",
"text": "Long system prompt...",
"extras": {"prompt_cache_breakpoint": True},
}
```

### Request-level cache options

<Note>
Requires `langchain-openai>=1.3.5`.
</Note>

You can pass request-level prompt cache options to the Responses API using the `prompt_cache_options` parameter on the model:

```python
llm = ChatOpenAI(
model="gpt-4.1",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change

use_responses_api=True,
prompt_cache_options={"cache_type": "ephemeral"},
)
```

### Cache write tokens

When OpenAI writes new content to the prompt cache, it reports `cache_write_tokens` in the response. `ChatOpenAI` surfaces this as `cache_creation` in `input_token_details`:

```python
response = llm.invoke(messages)

cache_read = response.usage_metadata["input_token_details"].get("cache_read")
cache_creation = response.usage_metadata["input_token_details"].get("cache_creation")
print(f"Cache read tokens: {cache_read}")
print(f"Cache creation tokens: {cache_creation}")
```

On the `"priority"` and `"flex"` service tiers, these keys are prefixed with the tier name — for example, `"priority_cache_read"` and `"priority_cache_creation"`.

---

## Flex processing
Expand Down
Loading