News

Amazon Bedrock adds explicit prompt caching for open-weight models

Published 4 min readAI-assisted · human approved
TL;DR

AWS published a how-to on 16 November 2026 covering implicit and explicit prompt caching for open-weight models on Amazon Bedrock. Cache reads cost less than standard input; explicit mode lets you mark the reusable prefix.

Why it matters

If your app resends a long system prompt, manual or tool list on every call, caching can lower input cost and latency. The explicit mode helps when only a small part of the prompt changes per request, because you control what gets written to cache.

Image from AWS Machine Learning Blog
Image: AWS Machine Learning Blog. It belongs to its owner; see our image policy.

Original source

AWS published a technical how-to on its Machine Learning Blog on 16 November 2026 about prompt caching for open-weight models served through Amazon Bedrock. It says explicit caching is now available for these models, alongside the automatic implicit mode, and walks through both with the Chat Completions API.

What happened

The post, written by four AWS authors, explains how Amazon Bedrock prompt caching for open-weight models works. When a request begins with the same content as an earlier one, Bedrock reads that prefix from cache and processes only the remainder. The cache is managed by Bedrock. Typical reusable content is a system prompt, reference documents, tool definitions or the conversation so far.

AWS says prompts and outputs are not shared with the model provider or used to train models, as with other Bedrock models.

The post then does three things: it shows both modes through the Chat Completions API, shows how to read cache usage from each response, and applies implicit caching to an OpenCode coding agent session. It closes with best practices. The excerpt reviewed here covers the concepts, prerequisites and setup of the first example; the later sections are not reproduced below.

Implicit vs. explicit caching

Implicit Explicit
Request changes None On the Chat Completions API, prompt_cache_options with mode set to explicit, plus one or more prompt_cache_breakpoint markers
What gets cached Eligible prefixes, chosen automatically on a best-effort basis Content up to and including each breakpoint
Suited to Prompts that already begin with a stable prefix, such as conversations and agent loops A fixed shared prefix with a changing question, or layered prefixes you want to control

Which mode a model offers depends on the model and the API. AWS points to its "Models at a glance" page for each model's modes, minimum prefix length and cache lifetime.

The practical difference is about waste. With implicit caching, Bedrock may write content that later requests never reuse, such as a question that differs every time. Explicit caching limits writes to the shared prefix, and anything after the last breakpoint is billed at the standard input rate. That matters on models that charge more for cache writes.

Key details

  • Cache reads are billed at a lower rate than standard input. On some models, cache writes cost more than standard input; AWS directs readers to Bedrock pricing for per-model rates.
  • For implicit caching, AWS advises putting static content first and dynamic content last. If an early part of the prompt changes, the cache is rebuilt from that point.
  • With explicit caching you can add several breakpoints, up to the model's maximum, to cache layers that change at different rates.
  • The model card lists the minimum token count before a breakpoint, the maximum number of breakpoints and the time to live (TTL), which resets with each cache hit.
  • Model availability and caching support vary by AWS Region.
  • The samples authenticate with a short-term Bedrock API key, valid for up to 12 hours, generated from your AWS credentials.

Try it

Prerequisites in the post: an AWS account with access to your chosen model, an IAM identity allowed to invoke it, Python 3.12 with openai 3.26.0 or later and aws-bedrock-token-generator 1.1.0 or later, and npm for the OpenCode example (OpenCode 1.18.35).

python3.12 -m venv .venv
source .venv/bin/activate
pip install "openai>=3.26.0" "aws-bedrock-token-generator>=1.1.0"

The client is the OpenAI SDK pointed at the Bedrock endpoint:

from aws_bedrock_token_generator import provide_token
from openai import OpenAI

REGION = "<AWS_REGION>"
MODEL_ID = "<MODEL_ID_OR_INFERENCE_PROFILE_ID>"

client = OpenAI(
    api_key=provide_token(region=REGION),
    base_url=f"https://bedrock-runtime.{REGION}.amazonaws.com/openai/v1",
)

The sample use case is a support assistant for a fictional water filter. The system prompt holds a short instruction plus a roughly 4,000-token owner's manual, and each request asks a different customer question. AWS notes that placeholder text works for padding the manual past your model's minimum prefix length, since the cache only needs an identical prefix. The post sets max_tokens=1024 because reasoning tokens, if the model returns them, count toward that limit.

Why it matters

Agent loops, chat assistants and document Q&A all resend large, stable context. For those workloads, the choice is between letting Bedrock cache automatically and marking the shared prefix yourself. Explicit breakpoints are the more predictable option when the tail of the prompt varies on every call. Check your model's card for minimum prefix size, TTL and write pricing before relying on savings; AWS gives no specific savings figure in the excerpt reviewed. For related cost work, see how four Bedrock request settings change inference spend and how to measure Bedrock agent cost per task.

Checked claims

ClaimStatusEvidence
AWS says explicit prompt caching is now supported for open-weight models on Amazon Bedrock.confirmedOpen-weight models on Amazon Bedrock now support explicit prompt caching.
Cache reads are billed at a lower rate than standard input.confirmedCache reads are billed at a lower rate than standard input
On some models, cache writes cost more than standard input.confirmedDepending on the model, cache writes can be billed at a higher rate than standard input.
Explicit caching limits cache writes to the shared prefix, avoiding write charges for content never read back.confirmedWhen the content after a shared prefix changes on every request, use explicit caching to limit cache writes to the shared prefix.
The samples require Python 3.12 with specific minimum package versions.confirmedPython 3.12 with the openai (3.26.0 or later) and aws-bedrock-token-generator (1.1.0 or later) packages.
The short-term Bedrock API key used in the samples is valid for up to 12 hours.confirmedvalid for up to 12 hours

Found a mistake? Use Report above; an editor reads every one. Editorial policy

Question for readersWhich part of your prompts is stable enough to cache, and would you let Bedrock choose it or mark the breakpoints yourself?

Discussion · 0

No comments yet. Start the discussion.

Sign in to join the discussion.