AWS shows how four Bedrock request settings change inference spend

AWS published a post on its Machine Learning Blog on 12 November 2026 describing four request-level settings in Amazon Bedrock that change what an inference call costs: prompt cache duration, service tier, adaptive thinking effort and the maxTokens limit. The authors say the right value depends on model and workload, and they report their own test results for each.

What happened

The post, written by AWS staff, targets teams whose Bedrock bill keeps growing after prompt and application tuning. Its argument is that each response carries usage metadata, with separate counters for cache reads and writes, and that reasoning tokens are counted as output tokens. That makes request parameters a cost lever. AWS also points to sample notebooks (Mixed TTL Caching, Tenant Isolation with Prompt Caching) and a workshop repository folder, 01-low-effort, for trying the settings. Each sample sets its own model ID, so you need to swap in the model you actually call.

Key details

Prompt cache duration

• The prompt caching guide lists 5-minute and 1-hour TTLs for supported Anthropic models, and a 30-minute minimum TTL for supported GPT-5.6 models. Support varies by model.
• Implicit prompt caching is best effort with no hit guarantee, so the duration choice applies only to explicit caching.
• For Claude Sonnet 5 and Claude Opus 5, a 5-minute write costs 1.25 times the input rate, a 1-hour write 2 times, and a cache read 0.1 times.
• Assuming exact prefix reuse and TTL refresh on each read, a 5-minute cache pays back on the first reuse inside 5 minutes. If requests arrive more than 5 minutes apart, it rewrites at 1.25 times each time, and the 1-hour cache wins from the second request within the hour.
• AWS advises moving to a longer TTL when the same prefix recurs at least twice an hour with gaps over 5 minutes. To beat no caching at all, the 1-hour cache needs three requests an hour.
• Changing a system instruction or tool definition causes a miss and a new write. In the authors' six-call test on Claude Sonnet 5, each of the two changes produced zero cache reads.
• Caching requires on-demand inference; it does not work with batch inference.
• Cache entries are scoped to your account and Region, not your tenants. For multi-tenant apps, AWS suggests prepending a tenant-specific SHA-256 hash to the cacheable prefix and enforcing tenant authorization before building the prompt. For a hard data boundary, use separate AWS accounts.

Service tiers

• Flex suits background jobs, evaluation harnesses, nightly summarization and backfills. Standard is requested with default, not standard, or by omitting the field.
• Support is model-specific. The post says Nova 2 Lite and gpt-oss-120b list Flex as supported, while Claude Sonnet 5 and Claude Opus 5 do not.
• A Flex request to an unsupported model fails with a ValidationException rather than falling back to Standard. If policy allows Standard as a fallback, retry once with default, but don't retry every ValidationException because malformed requests raise it too.
• In the authors' test on Nova 2 Lite, 200 default and 200 flex requests were interleaved. Token counts matched, all 400 resolved to the requested tier, and the median latency difference was within noise. At the time of writing, the Flex rate in us-west-2 was half Standard, and their bill reflected it.
• You can confirm the resolved tier from the response or the CloudWatch ResolvedServiceTier metric.

Thinking effort

• Claude Sonnet 5 and Opus 5 default to adaptive thinking at high effort, even when the request omits the thinking field. Claude Haiku 5.5 defaults to medium. All three support low, medium, high, xhigh and max.
• An app moved unchanged from Claude Sonnet 4.6 starts billing thinking tokens, and maxTokens now caps thinking plus response text.
• Effort belongs in output_config, not inside thinking.

The authors tested a tool-calling agent on Claude Opus 5 across three tasks, each with one correct answer, running each task eight times per effort level:

Effort	Correct runs	Median output tokens	Change from low (95% interval)
low	24 of 24	936	baseline
medium	24 of 24	1,484	+59% (50% to 78%)
high (default)	24 of 24	2,521	+169% (112% to 190%)
xhigh	24 of 24	2,964	+217% (170% to 260%)
max	24 of 24	3,762	+302% (180% to 402%)

On a code-generation sample, median output was 676 tokens at low and 743 at high, within run-to-run variation, and graded programs passed at every level. AWS cautions that accuracy at lower effort depends on the model and task, so measure on tasks like yours.

maxTokens

The post treats maxTokens as an application-level ceiling that stops a bad prompt, regression or loop from generating unbounded output. Bedrock deducts input tokens plus maxTokens from quota when a request starts, so an oversized value reduces concurrency. AWS says it is not a safety control and points to Amazon Bedrock Guardrails for that.

Try it

These are the request fragments the post shows. Each goes in the matching field of your request, for models that support it:

{ "cachePoint": { "type": "default", "ttl": "1h" } }

{ "serviceTier": { "type": "flex" } }

{
  "thinking": { "type": "adaptive" },
  "output_config": { "effort": "low" }
}

The first requests a 1-hour cache entry in the Converse API; for supported Anthropic models, InvokeModel puts the same ttl in cache_control.

Why it matters

Two points stand out for developers. First, model migrations can change cost silently: a move to Claude Sonnet 5 or Opus 5 brings high-effort thinking by default. Second, the post ties each setting to something you can check, namely cache read and write tokens, resolved tier, per-request output counts, CloudWatch OutputTokenCount and a task pass rate. The results come from AWS's own tests on specific models, so rerun them on yours. If you are building agents on Bedrock, the earlier look at four layers of context engineering in AWS's Deep Insight agent covers a different side of token use, and the addition of Claude Fable 5.2 and Mythos 5.2 to Bedrock is relevant when you re-check model support. The original write-up is on the AWS Machine Learning Blog.

FAQ

Is a longer Bedrock cache TTL always cheaper? No. For Claude Sonnet 5 and Opus 5, a 1-hour write costs 2 times the input rate versus 1.25 times for 5 minutes, so it pays off only when the same prefix is reused often enough with gaps over 5 minutes.

What happens if I request Flex on a model that doesn't support it? According to AWS, the request fails with a ValidationException instead of running on Standard. Retry once with default only if your policy permits that fallback.