人工智能
agents的土拨鼠日：在Amazon Bedrock上使用OpenAI模型的快速缓存模式
每次你离开你的agent，它必须重新读取它的整个生活故事，然后进行一个动作。 如何让它记住
在电影"地道浩克日"中，比尔·默里的角色每天在同名的"2月2日"重生。 他醒来后在收音机，同名客栈，同名小镇播放了同名歌曲，因为一些电影迷估计这时已经34年了。
他前天没有完成任何任务除非他从头再来一次 AI agents直播"地道浩克日"版本。 agent每走一步(叫出一个工具，读出结果，决定下一步要做什么)，它就必须发送模型迄今为止的全部故事来保持连续性：工具定义，指令，每一个前作的工具呼叫，以及每个结果。 模型只根据您提供的输入来预测下个符号。 它在请求之间没有内部记忆，所以它必须再读一遍所有的内容。 到了编码会话的第20回，模型的重读会变为1–19
快速缓存打破了循环。 服务会存储您的输入快取(指令，工具和内容)，并在下次请求时以输入价格的一小部分重新使用。
透过agent的生活，一路走来，我告诉你如何快速的缓存降低成本保持agents的响应性和一致性。
agent发送到它的LLM
在谈论缓存之前，它有助于研究典型的agent第一次引用大语言模型(LLM)包含什么：
- 工具列表 - agent每个函数的定义都可以调用。 运送新版本时会改变
- 指导 - 开发者提示：规则，人物，政策，实例。 每个发行版也会有变化。
— 历史 — — 到目前为止这段时期所发生的一切：任务、模型的工具召唤、工具结果(每转一圈都增长).
- 新转弯最新的工具结果或用户消息(每次可能不同).
agent寄给模型
车盖下如何快速地行走
当一个模型读取了提示时，它首先会做一个叫做预填的写作。 在预填时，它会处理每个输入符并构建其键和值向量(也叫"键值"，或"KV")，这构成了模型的工作状态。 这发生在模特儿写出第一个符号之前。
即时缓存可以保存该状态，包含系统即时，工具，内容，历史等。 当后期请求以相同的输入符(即同文本)开始时，即时缓存服务会加载所保存的状态而不是再次计算。 然后它只处理模型下一个有效载荷中的新内容。 快速缓存可以是隐含的，Amazon Bedrock 选择可重复使用的前缀结束的地方，或清晰的，您的代码标记了使用 poil cache breakpoint 的终点。 你们可以一起使用这两种方法，我们建议这样做。
转一与转二
隐含或明确，为什么我们需要两者？
对于代理工作量，隐含的缓存让贝德罗克选择随着对话的增长而缓存什么。 通过明确的缓存，您可以标记共享内容，如工具和指令，用于跨会话的再利用，并在旋转帮助器agents时使用。 你并不总需要两种形式的缓存
新的会话也可以重复使用共享的起步点，而每个对话则建立在自己的缓存历史上。 在本文中，我们从第一个转弯开始，然后展示如何通过后转弯，新的工具，以及跨子剂携带被缓存的内容。
你好缓存世界
第一个胜利是缓存提示的静态部分：在请求中保持不变的工具和指令。 这些示例使用您现有的 Amazon Bedrok 客户端 TOOLS 和任务。 以您的真实政策和实例替换缩短的规范字符串。 可重复使用的前缀必须达到1,024个令牌才能合格。
当缓存静态内容时，使用显式模式：选择所保存的前缀结束位置。
prote cache key 告诉要共享缓存的事务。 当我们在新的会话中读取缓存时，我们回到如何使用它。 在一次使用显式缓存的测试中，大约4000个托起的工具和指令前缀在第一次请求中写了4,120个符，并在后来的请求中读取了这4,120个符。 任务( 或用户请求) 位于缓存边界后，因此不会触发缓存的写入。
记住，只有在你通过完全相同的文本时，才能迅速缓存。 确保agent的指令和工具列表每次都以同样的方式被序列化。 模式上下文协议(MCP)服务器和其他形式的工具提供需要索引，以确保它们保持一致。 在缓存指令之后，保留任何不同内容，如日期、 用户名称或请求ID。
随着内容的增长和变化(不同的工具和内容)，使用隐含模式让服务跟随对话。 这里，第一个请求没有明确的标记。 其最新的合格消息是用户的任务，所以自动断点是在任务完成之后，并包括工具、指示和任务。 该模式在以下例子中明确显示，以明确，即使隐含是默认。 设置商店 = 虚假在您的应用程序中留下对话历史。 它不会禁用缓存。
在下一个回合继续执行同样的任务时，这是有用的。 不同的任务与旧任务前缀不相匹配，当您需要缩短共享前缀时，请使用一个清晰的标记。
当agent需要共享起点和不断增长的历史时同时使用。 从下面的代码示例中标记的开发者块开始，并调用函数 ask with implicity cache. 请求级别模式保持不变。 块级标记新增是指令后的一个明确保存点。 该服务也可以通过最新的合格消息保存。 这使得共享前缀在对话增长时仍然可用。
缓存对话历史和工具结果
该模型按名称要求工具，并提供了它们的论点。 您的应用程序必须执行。 python 函数并返回其结果。 以下循环使用此帮助运行每个请求的工具。 它将模型的输出和所有工具结果附加到历史列表中，然后再次发送该列表，直到模型返回最后的答案。 在隐含模式下，Amazon Bedrock可以通过上一个工具结果再用不变的前缀并缓存所添加的内容。 保持早期消息和初始TOOLs不变。
对于后续，请附加新的用户消息并再次给帮助者打电话。 保持较早的消息不变，使其缓存的端点仍然可用。
滚出断点：每个转弯都会读出它面前的一切
对于一个单独的明文唯一运行，从合并的Hello缓存世界例子中再现出双信息历史。 这个环绕工具在输入文本块中产生结果，并在每批中标出最终结果。 早期的标记留在历史。 参见当前对长对话的查询规则。
GPT-6 Astra 绑定了最好的隐含结果导致这六轮工作：大约62%的输入成本低于运行时没有缓冲。 明确的滚动断点在同一模型上节省了约61%. 两人随着对话的增长而重新利用了更多的历史。 这些是工作量的结果。 测量您自己的 agent 储蓄。
当心：
- 仅以明确模式出现的静态断点工具和指令被缓存，但不断增长的历史在每个回合(0.61)再次被处理。 这个限制不适用于综合例子
- 移动单个断点可以防止缓存再利用，并导致以明文唯一模式重复写入。 保留前一个回合的断点。
– 只有通过变化中的任务来隐含写作 – 新的任务可能共享工具和指令，而不是之前保存的终点。 如果不同的任务需要再利用这个较短的前缀，在共享指令后添加一个明确的标记。
当工具改变时
有时agent的工具列表会被更改，或者会被添加出新的工具。 保持初始列表的稳定：添加，删除，或重排定义可以使缓存前缀无效。 选择适合工具如何可用的方法。
当已知工具在 agent 流动的不同步骤被引用时：发送同样排序的 TOOLS 并使用允许的 tools 来限制模型的称谓。 在允许的工具上，"mode":"auto"让模型决定是否调用允许的工具，而"mode":"required"需要至少一个允许的工具调用。 通过此工具选择。 限制不会改变缓存目录：
会议期间的新工具。 Amazon Bedrock runtime和Amazon Bedrock Mantle 端点都接受一种名为附加"工具"的输入项目类型，用于GPT-6 Astra,GPT-6.1 Sol，和GPT-5.6 Luna. 关于由AWS区域提供的模型可用性，参见由AWS区域在Amazon Bedrock中支持的模型。 在助手函数中注册新工具 get dead date execution 函数，然后从 agent 循环将其计划附加到已完成的历史中。 保持初始 ToOLs 不变：
新工具在会中到达
继续工具执行循环与新响应。 循环新增模型的输出到历史，运行任何请求的工具，并附加其结果。 在以后的请求中保留附加的 tools项，以便保留新工具。 在隐含模式下，Amazon Bedrock可以通过新用户消息缓存，包括之前的工具定义。 在明文唯一模式中，保留更早的断点，并在该用户信件的输入文本块中添加一个。 附加工具项目本身不能有一个断点。
不同的角色，不同的目录。 当不同的agent角色需要不同的工具时，在每个会话开始时选择一个稳定的工具配置，如支持或计费。 每个配置文件都获得自己的缓存前缀。 标出共享的指令，这样后期的任务与该配置文件可以重用它们，然后随着对话的增长使用任意一种模式。
缓存工具配置
副代理和副电话
授权不应该让每个助手重读相同的背景。 共享前缀，然后在末尾添加每个助手的任务。 要注意，在用AI agents进行缓存时，工具和系统提示可能会被泄露或被利用。 建立负责任的人工智能做法，包括迅速实施，作为治理的一部分。
有一次暖和，然后扇出。 先发送普通请求并等待在发布帮助者之前完成。 在每个请求中明确标记共享上下文，即使模式是隐含的：
帮助者使用与热身相同的TOOLs，型号，和关键。 以输出封顶结束的热身仍能充充缓存。 用法检查在帮助者开始前确认缓存活动。
子代理：温暖共享前缀一次，然后扇出
给一方打电话相同的开始。 agent循环通过附加每个resp.output及其匹配的工具结果来建立历史，包括加密推理。 在父母完成后，复制该完整清单供摘要或法官使用。 同一帮助者使工具、模型和密钥保持不变：
帮助者可以重用历史中被缓存的部分，但附加的开发者消息不会自动扩展缓存以包含摘要指令。 如果以后要求再利用，请输入相同的历史和指示
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Artificial Intelligence
Groundhog Day for AI agents: Prompt caching patterns for using OpenAI models on Amazon Bedrock
Every time you spin off your agent, it has to re-read its whole life story before performing an action. Here’s how to make it remember without having to feed it the same context again.
In the movie Groundhog Day, Bill Murray’s character re-lives the same February 2 every day. He wakes up to the same song playing on the radio, in the same inn, and the same small town, again and again for what some movie fans estimate to be 34 years.
Nothing he accomplished the day before carries over unless he works it out again from scratch. AI agents live in a version of Groundhog Day. Every time an agent takes a step (calls a tool, reads the result, decides what to do next), it has to send the model its entire story so far to maintain continuity: the tool definitions, the instructions, every previous tool call, and every result. The model only predicts the next token based on the input you provide. It has no internal memory between requests, so it must read all of it again. By turn 20 of a coding session, the model re-reads turns 1–19, and it’s you who pays for every one of those tokens, again and again.
Prompt caching breaks the cycle. The service stores your input prompt (instructions, tools, and content) and reuses it on the next request, at a fraction of the input price.
In this post, I walk you through an agent’s life, from its first turn to when it gets new tools and spawns helpers. Along the way, I show you how prompt caching keeps costs down and keeps agents responsive and consistent.
What an agent sends to its LLM
Before talking about caching, it helps to look at what a typical agent’s first invocation to the large language model (LLM) contains:
- Tool list – The definitions of every function the agent can call. Changes when you ship a new release.
- Instructions – The developer prompt: rules, persona, policies, examples. Also changes per release.
- History – Everything that happened in this session so far: the task, the model’s tool calls, and the tool results (which grows every turn).
- New turn – The latest tool result or user message (which might be different every time).
What an agent sends to the model, turn after turn if you don’t set up caching:
How prompt caching works under the hood
When a model reads a prompt, it first does a write called prefill. During prefill, it processes every input token and builds its key and value vectors (also known as key-value, or KV), which form the model’s working state. This happens before the model writes its first token.
Prompt caching saves that state, which contains the system prompt, tools, content, history, and so on. When a later request starts with the same input tokens (that is, the same text), the prompt caching service loads the saved state instead of computing it again. It then processes only what is new in the next payload to the model. Prompt caching can be implicit, where Amazon Bedrock selects where the reusable prefix ends, or explicit, where your code marks that endpoint using prompt_cache_breakpoint. You can use both approaches together, and we recommend doing so.
Turn 1 compared to turn 2
Implicit or explicit, and why do we need both?
For agentic workloads, implicit caching lets Bedrock choose what to cache as the conversation grows. With explicit caching, you can mark shared content, such as tools and instructions, for reuse across sessions and when spinning up helper agents. You don’t always need both forms of caching, but combining them gives you a reusable starting point while Bedrock manages the growing conversation automatically.
New sessions can also reuse that shared starting point while each conversation builds on its own cached history. In this post, we start with the first turn, then show how to carry cached content through later turns, new tools, and across subagents.
Hello cached world
The first win is to cache the static part of the prompt: the tools and instructions that stay the same across requests. These examples use your existing Amazon Bedrock client, TOOLS, and task. Replace the shortened INSTRUCTIONS string with your real policies and examples. The reusable prefix must reach 1,024 tokens to qualify.
When caching static content, use explicit mode: choose where the saved prefix ends.
The prompt_cache_key tells the service which requests are meant to share a cache. We come back to how to use it when we get to reading the cache in new sessions. In one of our tests using explicit caching, the roughly 4,000-token tools-and-instructions prefix wrote 4,120 tokens on the first request and read those 4,120 tokens on later requests. The task (or user request) comes after the cache boundary, so this doesn’t trigger a write to the cache.
Keep in mind that prompt caching works only when you pass exactly the same text. Make sure that the agent’s instructions and tool lists are serialized the same way every time. Model Context Protocol (MCP) servers and other forms of tool delivery need to be indexed to make sure they stay consistent. Keep anything that varies, such as the date, the user’s name, or a request ID, after the cached instructions.
As the content grows and varies (different tools and content), use implicit mode to let the service follow the conversation. Here, the first request has no explicit marker. Its latest eligible message is the user’s task, so the automatic breakpoint comes after that task and covers the tools, instructions, and task together. The mode is shown explicitly in the following example for clarity, even though implicit is the default. Setting store=False leaves conversation history in your application. It doesn’t disable caching.
This is useful when the next turn continues the same task. A different task doesn’t match the prefix through the old task, and the implicit write doesn’t necessarily create a separate entry for only the tools and instructions. Use an explicit marker there when you need that shorter shared prefix to be reusable across tasks.
Use both when the agent needs a shared starting point and a growing history. Start with the marked developer block in the following code example and call the function ask_with_implicit_cache. The request-level mode stays implicit. The block-level marker adds an explicit save point after the instructions. The service can also save through the latest eligible message. This keeps the shared prefix available while the conversation grows.
Caching conversation history and tool results
The model requests tools by name and supplies their arguments. Your application must execute them. execute_function is a helper you build to route those requests to the matching Python functions and return their results. The following loop uses this helper to run each requested tool. It appends the model’s output and all tool results to the history list, then sends that list again until the model returns a final answer. In implicit mode, Amazon Bedrock can reuse the unchanged prefix and cache the added content through the last tool result. Keep earlier messages and the initial TOOLS unchanged.
For follow-ups, append a new user message and call the helper again. Keep earlier messages unchanged so their cached endpoints remain available.
Rolling breakpoints: Each turn reads everything before it
For a separate explicit-only run, recreate the two-message history from the combined Hello cached world example. This loop wraps tool results in input_text blocks and marks the final result in each batch. Earlier markers stay in history. See the current lookup rules for long conversations.
GPT-6 Astra tied for the best implicit result in this six-turn workload: about 62 percent lower input cost than running without caching. Explicit rolling breakpoints saved about 61 percent on the same model. Both reused more history as the conversation grew. These are workload results. Measure your own agent’s savings.
Watch out for:
- Only a static breakpoint in explicit-only mode – The tools and instructions are cached, but the growing history is processed again on every turn (0.61). This limitation doesn’t apply to the combined example, which also saves through eligible conversation endings.
- Moving a single breakpoint can prevent cache reuse and cause repeated writes in explicit-only mode. Keep the previous turn’s breakpoint too.
- Only an implicit write through a changing task – A new task might share the tools and instructions but not the previously saved endpoint. Add an explicit marker after the shared instructions if different tasks need to reuse that shorter prefix.
When tools change
Sometimes an agent’s tool list changes, or new tools get added. Keep the initial list stable: adding, removing, or reordering definitions can invalidate the cached prefix. Choose the approach that fits how the tool becomes available.
When known tools are invoked in different steps of the agent’s flow: Send the same sorted TOOLS and use allowed_tools to limit what the model can call. With allowed_tools, "mode": "auto" lets the model decide whether to call an allowed tool, and "mode": "required" requires at least one allowed tool call. Pass this as tool_choice. The restriction doesn’t change the cached catalog:
New tools during a session. Both Amazon Bedrock Runtime and Amazon Bedrock Mantle endpoints accept an input item type called additional_tools for GPT-6 Astra, GPT-6.1 Sol, and GPT-5.6 Luna. For model availability by AWS Region, see Supported models by AWS Region in Amazon Bedrock. Register the new tool get_due_date in the helper function execute_function, then append its schema to the completed history from the agent loop. Keep the initial TOOLS unchanged:
A new tool arrives mid-session
Continue the tool-execution loop with the new response. The loop adds the model’s output to history, runs any requested tools, and appends their results. Keep the additional_tools item in later requests so the new tool remains available. In implicit mode, Amazon Bedrock can cache through the new user message, including the tool definitions that precede it. In explicit-only mode, retain earlier breakpoints and add one to an input_text block in that user message. The additional_tools item itself can’t hold a breakpoint.
Different roles, different catalogs. When different agent roles need different tools, choose a stable tool profile, such as support or billing, at the start of each session. Each profile gets its own cached prefix. Mark the shared instructions so later tasks with that profile can reuse them, then use either mode as the conversation grows.
Cached tool profiles
Subagents and side calls
Delegating shouldn’t make every helper reread the same background. Share the prefix, then add each helper’s task at the end. Be aware that when you use caching with AI agents, tools and system prompts can be leaked or exploited. Establish responsible AI practices that include your prompt caching implementation as part of that governance.
Warm once, then fan out. Send an ordinary request first and wait for it to finish before launching helpers. Mark the shared context explicitly in every request, even when the mode is implicit:
The helpers use the same TOOLS, model, and KEY as the warm-up. A warm-up that ends at the output cap can still populate the cache. The usage check confirms cache activity before the helpers start.
Subagents: Warm the shared prefix once, then fan out
Give side calls the same beginning. The agent loop builds history by appending every resp.output and its matching tool results, including encrypted reasoning. After the parent finishes, copy that complete list for a summarizer or judge. The same helper keeps the tools, model, and key unchanged:
The helper can reuse the cached portion of history, but the appended developer message doesn’t automatically extend the cache to include the summary instruction. If later requests reuse the same history and instruction, place the instruction
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.