人工智能
在Amazon SageMaker AI上以知识分馏方式建立高效的agents
你在边疆模型上造了个购物助理，它成功了 agent搜索目录，检查候选人，检查限制条件，并承诺推荐。 但是这种互动实际上需要多少？代理助手不是一个模型调用，而是循环：搜索，读取结果，精炼，验证，决定。 一个完成的任务可以是多次呼叫那个前沿模型，你支付每一个，每个shopper。 在生产规模上，这种每任务成本的化合物会侵蚀项目的ROI，并阻止项目的航运。
提高代理应用效率的一个方法是使用更小的模型，这个模型反应更快，主机成本效益更高。 捕获量是质量的：一个更小的模型通常没有完成任务，这就是为什么更大的模型首先被选中的原因。 知识分解通过将更大的“教师”模式的行为转化为更小的“学生 ” 来弥补这一差距，培训学生复制教师的产出，从而以更小的模式的速度和成本保持大部分质量。
在本文中，您将学习如何使用知识分馏，特别是序列级知识分馏(SeqKD)，到微调 a shopping agent on Amazon SageMaker AI. 我们介绍如何与边疆教师一起产生高质量的多步轨迹，并保留解决任务的轨迹，如何为受监督的微调编码，如何进行无服务器基础设施的培训，以及如何对学生进行针对被搁置的基准测试的评价。
什么是序列级知识分馏(SeqKD)?
最简单的适用蒸馏形式是序列级知识蒸馏(SeqKD). 它使用教师模式生成高质量的响应，然后用监督的微调(SFT)在这些响应上培训学生模式。 SeqKD只需要老师的文字，没有老师的日志，也没有强化学习的基础设施，所以即使有一位封闭的或只使用API的老师也能工作。 与基于登录或政策的其他方法相比，它处于蒸馏光谱中最具成本效益和可移植的一端。 老师写出答案，学生对答案微调。 因为这些答案作为数据集持续存在，你可以重新使用它们来训练或测试不同的学生，而不重新生成数据。
原则上，您希望学生能与老师的分布相匹配。 这里的警告是，你不能。 可能测序的空间在天文上很大，SeqKD绕道，通过将它与老师实际生产的样本相近。
对agent来说，老师所制作的序列不是道行道的块而是一个道行道，是命令的推理跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跳跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑跑对学生进行关于这整个序列的培训，教他们如何推理导致每个动作和多步骤行动方针本身，而不是回答文本的一段落。 这使得一个代理任务成为序列级KD最干净的插图之一，因为被复制的序列是行为本身。
当 SeqKD 是正确的选择
SeqKD 符合一组特定的情况：
您有输入，但没有高质量的目标产出
这是SeqKD的核心案例，它基本上是数据生成。 您提供输入，老师会写目标，这样您就可以在一个任务或域上使用微调，没有现有的高质量标签。 目标可以像单一答案一样简单，也可以像多回合工具使用轨迹一样参与。 这篇文章占据了那段频谱，特质轨迹的苛刻一端，这些轨迹尤其难以取出。 单任务分支进入许多部分正确的跑道，告诉从失败者中解出跑道需要可靠的分级器。 一个基准测试，有一个程序计分器提供精确的，这样你就可以让老师生成出轨迹，并且只保留那些解决任务的。
你的老师是个前沿模特你只能通过API才能找到
这是边境模型的正常案例。 您无法亲自访问重心，无法读取其日志，这排除了基于登录的蒸馏和基于政策的方法，它们都需要内部的准入。 SeqKD只需要老师的文本响应，所以它是唯一仍然可行的方法，它让你在学生接管后不按要求支付边境价格。
你想要一个强大，成本效益高的第一步
一些最强小推理模型是和SeqKD(在教师推理痕迹上监督了微调)一起在相当大程度上被构建而来，因此它是在设置正确时推理转移的有效起点。 DeepSeek-R1蒸馏模型是一出有名的例子：一位大老师的推理被微调在老师生成的推理痕迹上分馏为较小的Qwen和Llama学生。
先决条件
要运行这个例子，你需要：
- AWS账户，由SageMaker AI担任AWS身份和访问管理(IAM)角色，该用户已读取和写入Amazon简易存储服务(Amazon S3)桶并允许引用您的老师模型。
- 一个边疆教师模型，以产生出轨迹(这个穿行方式使用Amazon Bedrock上的一款).
- 在支持的AWS区访问SageMaker AI模型定制。
- Amazon SageMaker Studio(或一个配置有AWS CLI的本地IDE)，运行一个Python 3.12或更高内核，由SageMaker Python SDK v3和boto3(pin your version). 对笔记本环境至少使用ml.m5.4x大例，因为数据编制和本地评分步骤运行在它的CPU上。
- Python的工作知识，监督微调，工具使用agents。
- 配有程序计分器的配套代码寄存器和任务(这一走行部使用Schopping Bench测试集)。 代替你自己的任务和分级员。
解决办法概述
本节从一端行走管道。 Amazon Bedrock上的边疆教师GLM-5.3扮演了"购物奔驰"agent来生成出轨。 基准测试自己的计分员只保留解决任务的计分员，那些解开的轨迹微调 Qwen3.5-4B学生在SageMaker AI模型定制上拥有无服务器监督的微调。 蒸馏后的学生在被搁置的"购物奔驰"分出后被评价，并服务于运行工具循环的"SageMaker AI推论"实时端点后。
正在准备数据集
测试值得模仿的轨迹是困难的，而购物奔驰计分器是将成功者与失败者分开的。 购物奔驰出版750个综合购物问题，每件250个用于产品，商店，和券意向，每件都有隐藏的黄金答案和决定成功的各种限制。 我们曾把它们分成426个培训问题和324个搁置的评价问题(产品103、商店105、凭证116)
对于每个训练问题，一个边疆教师GLM-5.3在Amazon Bedrock上进行推理和温度为1.0，扮演购物奔驰agent. 它搜索约270万个产品的目录，检查候选人，推荐符合限制条件的产品。 然后让Schopping Bench自己的计分员只保留那些能解决问题的计分员。 温度1.0是故意的。 采样而不是贪婪地解码使得两次尝试探索不同的询问和推理路径，这给更多的问题至少一个已经解决的轨迹，以及更多样化的一组来学习。 这个拒绝-抽样步骤将一个有能力但又不完善的老师变成了一个干净的训练场景，因为老师只需要经常做到正确，解决的跑步就值得模仿。
每个解开的轨迹都是完整的一集：shopper请求，老师在每个步骤(搜索，检查，推荐，终止)的推理和工具呼叫，以及观察环境返回。 我们把推理和行动放在一起，“结果”显示，是学生去其老师身边的主要途径。
培训的编码轨迹
序列级分馏是在已经存在的轨迹上监督学习，因此在训练期间没有任何东西称为工具，应用程序只看到文本。 因为这些轨迹是多步工具使用的对话，我们把每个轨迹编码为聊天转折的信息列表，让学生自己的聊天模板制作出来，因此训练员看到的符号与agent推想产生的符号相匹配。
每个轨迹成为每步一行训练。 用户转身携带系统即时和对话至今。 助手转过身来提出这个步骤的推理，损失落在助手身上只转弯，所以模型学会产生下个推理与动作，而不是预测环境返回的观测。 一个有数个步骤的轨迹扩展进入多行。 在此解开的轨迹被扩展至过滤后约2600个培训和130个验证行。
单行将到目前为止的对话(用户)与下一步学习(助手)对齐：
每后一步会成为自己的一行，新的观察会附在用户内容上，接下来的推理-和工具-调用作为助理目标，通到最终推荐产品并终止调用。 使用您的模型精确的聊天模板逐行渲染。
数据按问题划分，而不是按行划分。 老师每个问题被抽样两次，每个轨迹扩展分入几个步骤，因此许多行都有一个问题，一行平分就会在训练和验证过程中泄露出近同上下文。 持有整个问题(大约5%) 保持验证的诚实。
每步编码的成本是重复的上下文，因为每行会重述不断增长的对话，而没有服务器的微调会截断任何过去dataset max len，所以一行跨越极限会失去它需要的部分。 与其让限制决定什么被削减，不如让排行在预算范围内。 最有效的修补是压缩工具观察，它支配了符号计数。 对于产品搜索结果，只保留下个决定取决于的字段(id，价格，评分，以及正在检查的限制)，放下原始上市文本，并限制每次搜索保留多少结果。 如果轨迹仍然超出预算，则每个轨迹的步数上限。 这里的dataset max len是16,384个令牌，这在观测被修剪后符合大多数的轨迹。
在JSONL为每分块在S3后，将每分块登记为DataSet的自定义数据集资产。 从 SageMaker Python SDK v3 创建，它返回 train dataset 和 val dataset 所消耗的训练任务。
微调学生
训练使用SageMaker AI模型定制，无服务器的SFT. 您从模型中心选择一个基础模型，将任务指向被编码的数据集，而管理的服务提供则计算，运行任务，并返回定制模型，在Amazon SageMaker中用ML流跟踪测量。 由于数据已经以被支持的格式存在，因此没有训练脚本，容器图像，也没有实例可以管理。
任务将其输出写入一个模型包组，该组将它所制作的每个模型进行版本，因此先用模型包组。 Create(如果名称已经存在的话再使用它)创建该组，并传给训练员。 然后配置定制任务。 选择基本模型和训练类型，将其指向列车和验证数据集外加模型包组，然后命名为 MLflow e
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Artificial Intelligence
Build efficient agents with knowledge distillation on Amazon SageMaker AI
You built a shopping assistant on a frontier model, and it works. A shopper asks for “a black waist bag under twenty dollars from a shop with at least 4.5 stars,” and the agent searches the catalog, inspects candidates, checks the constraints, and commits to a recommendation. But what does that interaction actually cost? An agentic assistant is not one model call, it is a loop: search, read results, refine, verify, decide. A single completed task can be multiple calls to that frontier model, and you pay for every one, on every shopper. At production scale, that per-task cost compounds, erodes the project’s ROI, and stops the project from shipping.
One way to make an agentic application more efficient is to use a smaller model, which is faster to respond and more cost-effective to host. The catch is quality: a smaller model usually falls short on the task, which is why the larger one was chosen in the first place. Knowledge distillation closes that gap by transferring the behavior of the larger “teacher” model into a smaller “student”, training the student to reproduce the teacher’s outputs, so it keeps much of the quality at the smaller model’s speed and cost.
In this post you will learn how to use knowledge distillation, and specifically sequence-level knowledge distillation (SeqKD), to fine-tune a shopping agent on Amazon SageMaker AI. We cover how to generate high-quality multi-step trajectories with a frontier teacher and keep the ones that solve the task, how to encode them for supervised fine-tuning, how to run the training on serverless infrastructure, and how to evaluate the student against a held-out benchmark.
What is sequence-level knowledge distillation (SeqKD)?
The simplest applicable form of distillation is sequence-level knowledge distillation (SeqKD). It uses a teacher model to generate high-quality responses, then trains a student model on those responses with supervised fine-tuning (SFT). SeqKD needs only the teacher’s text, with no access to teacher logits and no reinforcement-learning infrastructure, so it works even with a closed or API-only teacher. Compared to other approaches such as logit-based or on-policy, it sits at the most cost-effective and portable end of the distillation spectrum. The teacher writes the answers, and the student is fine-tuned on them. Because those answers persist as a dataset, you can reuse them to train or test different students without regenerating the data.
In principle, you would want the student to match the teacher’s full distribution over possible outputs. The caveat here is that you can’t. The space of possible sequences is astronomically large, so matching it exactly isn’t feasible. SeqKD sidesteps that by approximating it with samples the teacher actually produces.
For an agent, the sequence the teacher produces is not a block of prose but a trajectory, an ordered run of reasoning, tool calls, and the observations they return, where each step is conditioned on everything before it. Training the student on that whole sequence teaches it the reasoning that leads to each action and the multi-step course of action itself, not a single paragraph of answer text. That makes an agentic task one of the cleanest illustrations of sequence-level KD, because the sequence being copied is the behavior itself.
When SeqKD is the right choice
SeqKD fits a specific set of situations:
You have the inputs but no high-quality target outputs
This is the core case for SeqKD, which is fundamentally data generation. You supply the inputs, and the teacher writes the targets, so you can fine-tune on a task or domain that has no existing high-quality labels. The targets can be as simple as single answers or as involved as multi-turn tool-use trajectories. This post takes the demanding end of that spectrum, agentic trajectories, which are especially hard to source. A single task branches into many partially correct runs, and telling a solved run from a failed one takes a reliable grader. A benchmark with a programmatic scorer supplies exactly that, so you can let the teacher generate trajectories and keep only the ones that solve the task.
Your teacher is a frontier model you can only reach through an API
This is the normal case for frontier models. You can’t access the weights to host it yourself, you can’t read its logits, and you can’t fine-tune it. That rules out logit-based distillation and on-policy methods, which all need that internal access. SeqKD needs only the teacher’s text responses, so it is the one approach that still works, and it frees you from paying frontier prices on every request after the student takes over.
You want a strong, cost-effective first step
Some of the strongest small reasoning models were built in significant part with SeqKD (supervised fine-tuning on teacher reasoning traces), so it is an effective starting point for reasoning transfer when the setup is right. The DeepSeek-R1 distilled models are a well-known example: the reasoning of a large teacher was distilled into smaller Qwen and Llama students by fine-tuning them on the teacher’s generated reasoning traces.
Prerequisites
To run this example, you need:
- An AWS account with an AWS Identity and Access Management (IAM) role for SageMaker AI that has read and write access to an Amazon Simple Storage Service (Amazon S3) bucket and permission to invoke your teacher model.
- A frontier teacher model to generate the trajectories (this walkthrough uses one on Amazon Bedrock).
- Access to SageMaker AI model customization in a supported AWS Region.
- Amazon SageMaker Studio (or a local IDE with the AWS CLI configured), running a Python 3.12 or higher kernel, with the SageMaker Python SDK v3 and boto3 (pin your versions). Use an ml.m5.4xlarge instance at minimum for the notebook environment, since the data preparation and local scoring steps run on its CPU.
- Working knowledge of Python, supervised fine-tuning, and tool-using agents.
- The accompanying code repository and a task with a programmatic scorer (this walkthrough uses the ShoppingBench test set. Substitute your own task and grader).
Solution overview
This section walks the pipeline from end to end. A frontier teacher, GLM-5.3 on Amazon Bedrock, acts as the ShoppingBench agent to generate trajectories. The benchmark’s own scorer keeps only the ones that solve the task, and those solved trajectories fine-tune a Qwen3.5-4B student with serverless supervised fine-tuning on SageMaker AI model customization. The distilled student is then evaluated on a held-out ShoppingBench split and served behind a SageMaker AI Inference real-time endpoint that runs the tool loop.
Preparing the dataset
Sourcing trajectories worth imitating is the hard part, and the ShoppingBench scorer is what separates the successful runs from the failed ones. ShoppingBench publishes 750 synthesized shopping problems, 250 each for the product, shop, and voucher intents, each with a hidden gold answer and the constraints that define success. We split them once into 426 training problems and 324 held-out evaluation problems (product 103, shop 105, voucher 116), as fixed indices so the two sets don’t overlap.
For each training problem, a frontier teacher, GLM-5.3 on Amazon Bedrock with reasoning on and temperature 1.0, acts as the ShoppingBench agent. It searches the catalog of about 2.7 million products, inspects candidates, and recommends the ones that meet the constraints. We sample the teacher twice per problem, which gives 850 trajectories, then let ShoppingBench’s own scorer keep only the ones that solve the task. Temperature 1.0 is deliberate. Sampling rather than greedy decoding makes the two attempts explore different queries and reasoning paths, which gives more problems at least one solved trajectory and a more diverse set to learn from. This rejection-sampling step turns a capable-but-imperfect teacher into a clean training set, since the teacher only has to be right often enough that the solved runs are worth imitating.
Each solved trajectory is a complete episode: the shopper request, the teacher’s reasoning and tool calls at each step (search, inspect, recommend, terminate), and the observation the environment returned. We keep the reasoning alongside the actions, which the Results show is what carries the student most of the way to its teacher.
Encoding trajectories for training
Sequence-level distillation is supervised learning over trajectories that already exist, so nothing calls a tool during training, and the application only ever sees text. Because these trajectories are multi-step tool-use conversations, we encode each as a messages list of chat turns and let the student’s own chat template render it, so the tokens the trainer sees match the tokens the agent produces at inference.
Each trajectory becomes one training row per step. The user turn carries the system prompt and the conversation so far. The assistant turn carries that step’s reasoning and the tool call it leads to, a … block followed by a [{…}] in the harness’s native form. Loss falls on the assistant turn only, so the model learns to produce the next reasoning-and-action, not to predict the observations the environment returned. A trajectory with several steps expands into several rows. Here the solved trajectories expand to about 2,600 training and 130 validation rows after filtering.
A single row pairs the conversation so far (user) with the one next step to learn (assistant):
Each later step becomes its own row, with the new observation appended to the user content and the next reasoning-and-tool-call as the assistant target, through to the final recommend_product and terminate calls. Render every row with your model’s exact chat template.
Split the data by problem, not by row. The teacher is sampled twice per problem, and each trajectory expands into several steps, so many rows share one problem, and a row-level split would leak near-identical context across training and validation. Holding out whole problems (about five percent) keeps the validation set honest.
The cost of per-step encoding is repeated context, because each row restates the growing conversation, and serverless fine-tuning truncates anything past dataset_max_len, so a row that crosses the limit loses part of what it needs. Rather than let the limit decide what gets cut, keep rows within budget. The most effective fix is to compress tool observations, which dominate the token count. For product search results, keep only the fields the next decision depends on (id, price, rating, and the constraints being checked), drop the raw listing text, and cap how many results you retain per search. If trajectories are still over budget, cap the steps per trajectory. Here dataset_max_len is 16,384 tokens, which fits most trajectories after observations are trimmed.
After the JSONL for each split is in S3, register each as a customization dataset asset with DataSet.create from the SageMaker Python SDK v3, which returns the train_dataset and val_dataset the training job consumes.
Fine-tuning the student
Training uses SageMaker AI model customization with serverless SFT. You select a base model from the model hub, point the job at the encoded datasets, and the managed service provisions the compute, runs the job, and returns the customized model, with metrics tracked in Amazon SageMaker with MLflow. Because the data is already in a supported format, there is no training script, container image, or instance to manage.
The job writes its output into a model package group that versions every model it produces, so create the group first with ModelPackageGroup.create (reusing it if the name already exists) and pass it to the trainer. Then configure the customization job. Choose the base model and training type, point it at the train and validation datasets plus the model package group, then name an MLflow e
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.