AWS and Accenture lay out six levers to cut Amazon Bedrock token spend
AWS published a post with Accenture on 19 November 2026 describing six Amazon Bedrock levers for cutting token spend. Where Accenture measured savings in client engagements, token spend fell by about 40–85 percent.
If your Bedrock bill is growing, the post gives an order of work: tag and measure first, then caching and batching, then routing and RAG tuning, then semantic caching and distillation. The savings figures are Accenture's, from client engagements, so treat them as ranges to test on your own workloads.

AWS published a post on 19 November 2026, written with Accenture, that describes six levers for reducing token spend on Amazon Bedrock. They are grouped into three phases by effort. Accenture says it applied them in four client engagements and, where savings were measured, token spend fell by about 40–85 percent.
What happened
The AWS Machine Learning Blog post starts from a cost problem. Per-token prices keep falling, but total spend can still climb, because cheaper tokens drive adoption and agentic workloads trigger many more model calls. Without cost allocation tags, the invoice shows AI as one line item rather than splitting it by team, workload, agent or model.
The authors recommend measuring first, then working through the levers in order. Each phase can also be applied on its own to the workloads where it helps most.
Step zero: attribute cost before optimizing
The post says tagging comes before any tuning. In Bedrock, you create application inference profiles, one per application or workload, and tag them by team, cost center and environment. You then activate those tags as cost allocation tags in the AWS Billing and Cost Management console.
Three AWS services supply the data:
- AWS Cost and Usage Reports (CUR 2.0), delivered through AWS Data Exports: billed inference cost by workload and team, once tags are active.
- Amazon CloudWatch: operational signals such as invocation counts and latency.
- Amazon Bedrock model invocation logging: request-level detail, including the model used and input and output token counts.
Accenture adds a business-level metric called Cost per Business Transaction (CpBT). It sums every model call, retry, tool call and agent handoff behind one successful business output. That lets a team tell a bill that grows because the product is succeeding from one that grows because of waste. For related reading, see OpenBeat's coverage of measuring Bedrock agent cost per task and per-identity cost reporting and spend limits.
Key details: the six levers
| Phase | Lever | What it does | Figure given in the post |
|---|---|---|---|
| 1. Configuration only | Prompt caching | Reuses repeated prompt prefixes such as system instructions and tool definitions | Designed to cut cached input token cost by up to 90 percent on supported models |
| 1. Configuration only | Batch inference | Runs non-urgent jobs asynchronously, with results written to Amazon S3 | 50 percent lower cost than on-demand for select models |
| 2. Architecture | Intelligent prompt routing | Sends each request to a model in the same family that fits its complexity and risk | Up to 30 percent lower cost |
| 2. Architecture | RAG optimization | Hierarchical chunking, metadata filtering and reranking to shrink retrieved context | None given |
| 3. Infrastructure and training | Semantic caching | Reuses answers to semantically similar requests, for example via Amazon ElastiCache | AWS benchmark: 74.5 percent hit rate at a 0.90 similarity threshold |
| 3. Infrastructure and training | Model distillation | Trains a smaller student model from a larger teacher | Up to 75 percent cheaper, under 2 percent accuracy loss for uses such as RAG |
The figures are AWS's own. The post also says distilled models are up to 500 percent faster.
Phase 1: configuration changes
Prompt caching needs cache checkpoint markers in your prompts. Batch inference means submitting eligible requests as a batch job instead of calling the on-demand API. The post suggests caching for interactive traffic and batching for work like nightly reports, document pipelines and evaluation runs. Neither requires new infrastructure. OpenBeat has covered explicit prompt caching for open-weight models on Bedrock separately.
Phase 2: architecture
Besides native Bedrock routing, Accenture describes two patterns that reduce input tokens, especially in multi-agent chains. Context scoping gives each agent only the document slice relevant to its step. Context compression summarizes or filters documents before they reach the model. The post tells readers to check Bedrock pricing for any routing charges.
Phase 3: infrastructure and training
The semantic caching benchmark covered 63,796 real chatbot queries and reported 92.3 percent accuracy and 72.5 percent cost savings alongside the hit rate. The post advises keeping Amazon Bedrock Guardrails on every route and setting cache expiry so users don't get outdated answers. Distillation is suited to stable, high-volume workloads where volume justifies the training investment.
What Accenture reports from clients
The four engagements span telecom, insurance, mining and a large enterprise deployment. Accenture withheld client names, and the post notes that results depend on each client's workload. In two engagements several changes were applied together, so those results reflect the combined effect.
- Large enterprise deployment: every request previously went to a frontier model, with no caching, tiering or batching. Accenture combined model tiering, prompt caching and batch processing and cut monthly token cost by 85 percent, without separating the effect of each change. About 15 percent of tasks still needed the top model.
- Telecom operator with $12 million in annual token spend: unbounded agent fan-out and frontier-model use for every request drove the bill. Triage routing, semantic caching and context compression together cut input tokens by 70 percent and annual spend from $12 million to $3.8 million.
Why it matters
The practical ordering is useful: tags and logging first, so you can see whether a change worked; low-effort configuration changes next; heavier investments last. Because the enterprise and telecom results combine several changes, they don't show how much any single lever contributes. Measure per-lever impact on your own traffic, for example with the per-task cost view described above.
Checked claims
| Claim | Status | Evidence |
|---|---|---|
| Where Accenture measured savings in its client engagements, token spend fell by about 40–85 percent, and results vary by workload. | confirmed | Where savings were measured, token spend fell by about 40–85 percent. |
| Amazon Bedrock batch inference processes non-urgent workloads asynchronously at 50 percent lower cost than on-demand pricing for select models. | confirmed | Amazon Bedrock batch inference processes non-urgent workloads asynchronously as a group at 50 percent lower cost than on-demand pricing for select models |
| An AWS benchmark of semantic caching on 63,796 chatbot queries at a 0.90 similarity threshold reported a 74.5 percent hit rate, 92.3 percent accuracy and 72.5 percent cost savings. | confirmed | In an AWS benchmark on 63,796 real chatbot queries, a 0.90 similarity threshold produced a 74.5 percent cache hit rate, 92.3 percent accuracy, and 72.5 percent cost savings |
| In the large enterprise deployment, Accenture's combined changes cut monthly token cost by 85 percent without separating the effect of each change. | confirmed | Together they cut monthly token cost by 85 percent, and the saving was not separated by change. |
| For the telecom operator, input tokens fell by 70 percent and annual token spend dropped from $12 million to $3.8 million. | confirmed | Input tokens fell by 70 percent, and annual token spend fell from $12 million to $3.8 million |
Found a mistake? Use Report above; an editor reads every one. Editorial policy
Question for readersWhich of these levers would you try first on your own Bedrock workloads, and how would you measure whether it worked?
Discussion · 0
No comments yet. Start the discussion.
Sign in to join the discussion.