人工智能
为更快的 SageMaker AI 任务发布 SOCI 索引图像
你的深层学习容器图像每分每分每分每分每分每分每分每分每分每分每分每分每分每分每分每分每分每分每分每分每分每分集装箱图像下载是纯高通：可计费的工作时间不会产生模型进度。 对于现代的深层学习图像来说，管理费是每个工作的几分钟。 SageMaker AI训练及Amazon SageMaker AI处理工作
通过SOCI索引，SageMaker AI懒惰地装入了你的容器图像。 它只获取启动容器所需的文件，并且在您代码运行时会流出背景中的其余部分，因此您的工作以秒而不是分钟开始。 您通过指代一个预索引的 AWS 深层学习容器( DLC) 或者在您自己的图像中添加一个索引来通过它。 两种方法都要求您修改工作定义、切入点或数据通道。 其结果是更快地开始工作，降低工作成本，花费更多的时间来建造模型，而不是等待基础设施。
SOCI-index化的图像在几秒钟内开始Amazon SageMaker AI训练容器，对训练性能没有影响。 正如我们所测量的部分所显示的，这把GPU在图像拉动时的闲置时间减少了90% 在我们的基准测试。
这篇文章显示如何通过 SOCI 索引：是什么，我们测量的结果，以及如何为自己的工作打开它。
挑战
SageMaker AI的培训和处理工作在Amazon弹性集装箱登记册(Amazon ECR)所储存的集装箱图像内进行。 在任务执行您的代码之前，服务会将整个图像下载到实例中。 它会解压，并将每一层写入磁盘。
对于小图像，这需要几秒钟。 但深层学习图像通常会超过10GB或以上。 下载新增分钟给每个工作，你支付它作为工作时间。 影响是最大的，具体而言，在迭代速度最紧要的地方：超参数调试扫荡上百个工作，快速实验，以及短处理工作，其中图像下载可以支配总运行时间。
SOCI如何编制索引
可搜索的OCI(SOCI)是一种开源技术，可以懒散地装载容器图像。 运行时间不是在开始前下载整个图像，而是使用一个SOCI索引，只获取容器发布所需的文件。 然后随着你的代码访问，它会流出背景中的其余部分。
一个 SOCI 索引是元数据，用于映射您图像中的每个文件。 你把它存放在Amazon ECR中，作为符合OCI要求的与你原始图像相联的文物。 由于索引是单独的文物，你的图像摘要和任何现存的签名仍然有效。
SageMaker AI 自动检查索引
当您创建培训或处理工作时，SageMaker AI 检查您指定的图像是否有 SOCI 索引，并相应行事：
因为工作总是在需要的时候会回到标准的下载，所以添加一个SOCI索引永远不会妨碍你的工作运行。 你只得到更快的启动。 你从不冒险打破现有的工作流程
我们衡量的
我们在真正的 SageMaker AI 训练工作上绘制了相同的容器图像：标准下载和SOCI索引的懒惰加载。 然后，我们从两个方面开展全面培训工作，以衡量对培训速度的影响。
设置
- 8个图像，1.8–35.4 GB压缩。
- GPU和CPU实例：ml.g4dn、ml.g5、ml.g6、ml.g6e、ml.g7、ml.g7e、ml.p4d、ml.p5和ml.m7i，从x大到48大，分布在美国西部(奥来贡)地区(西-2)和美国东部(北弗吉尼亚)地区(东-1)。
- 从图像拉动开始直到容器准备运行为止的下载时间。 所有数字均为重复运行的中位数。
结果 1：在每个图像大小下下载时间从分到秒
以下图表比较了容器图像下载时间，用于标准Docker拉出反SOCI懒惰地加载从1.8GB到35.4GB的8个图像尺寸。
每个图像大小为显示二条。 Docker下载时间栏(蓝色)随图像大小而稳步增长，从1.8GB的48秒到35.4GB的7分17秒。
每对显示上方的快取标签是Docker时间与SOCI时间之比。 速度从最小图像大约10x到最大图像超过35x不等。
标准的下载时间随着图像大小而增加，因为每个文件都被解压缩并被写入。 懒散的装载不管大小都停留在3–11秒，是一款8x–46x的改进。 图像越大，你节省的时间就越多。
成果2：每个实例类型和大小的节省都存在
在13个实例类型/大小组合中运行13.2 GB图像，标准下载范围从3分16秒到4分20秒不等，而懒惰的加载则维持在6-8秒。 每个配置都快34分您不需要更改实例类型或大小才能受益。
成果3：培训工作未受影响，超过头一个时期的惩罚
我们从Docker和SOCI指数化的图像中完成了两种完整的训练工作并且比较了每一个时代：
- 关于图像网的ResNet-50的图像分类(9 469个图像)
- 集装箱图像：PyTorch (13.2 GB DLC)和TensorFlow (9.9 GB DLC)
- 实例：ml.p5.48x大，ml.p4d.24x大，ml.g7e.48x大，ml.g7.48x大，ml.g6e.12x大，ml.g6e.24x大，ml.g6e.48x大。
- 大型语言模型(LLM) 微调：全参数导出微调 (SFT) Qwen2.5-0.5B 在平原上清理(51,760个实例) 有Hugging Face训练员。
- 集装箱图像：9.4 GB Hugging Face PyTorch DLC.
- 实例：ml.p5.48x大，ml.p4d.24x大，ml.g6.48x大。
以下图表将任务启动分解为图像下载时间和第一次培训时段，将标准Docker拉动与跨GPU和CPU配置的SOCI索引图像进行对比，以完成这两个工作量。
- 从第二个时期开始，每个时期的时间(在运行中的变化范围内)在每一情况下都是一样的，对两种工作量都是如此。
– 第一个时代需要从懒惰的图像中取出3–16 s 更长的时间，而文件框架第一次被取出。 头一个时期的成本在每一个实例大小上都是一样的。
- 网：每份工作提前1-6分钟完成。 即下载节省减去一次性的第一个时期成本，每个工作固定节省。
测试你自己的工作量
这些数字来自前面所述的两个工作量。 第一个时代的成本取决于您的图像和您的代码在启动时会碰到什么：
- 懒人加载在第一训练步骤只需要图像的一分之一时获胜，其余的溪流在训练时获胜。
- 如果您的入门点在训练开始前读取了大部分图像(例如扫描、检查或预装其内容)，第一个时代将等待大约相同的字节，一个完整的拉出。 净改善不大。
- 我们的测试中仍然有前置开始：
——"火炬". 编译和正时(JIT)所建造的CUDA扩展器在第一个时代增加了7-15 s (净节省了3分25 s-3分56 s)
- 6GB型号在加载时间被烘烤入图像后将重量流出，增加了2-3.5分，但总体工作仍能更早地完成索引成像上的2-2.5分。
开始
您可以采用 SOCI 索引两种方式：使用预索引的 AWS 深层学习容器或为自己的自定义图像创建索引。
备选案文1：使用预索引的AWS深层学习容器
AWS发布多款受欢迎的深层学习容器的SOCI索引变体
预索引变体可用于PyTorch训练，TensorFlow训练，vLLM和SGLang，包括一些框架的arm64变体。 要采用懒惰的加载，请将工作定义中的图像标记更新到 -soci 变体。 浏览 AWS 深层学习容器图片库中的完整列表。
注意：如果您从 DLC 碱基图像构建自己的图像，您必须为您的最终图像创建 SOCI 索引。 来自基图像的索引不会延续到所衍生的图像。
选项2：为您自己的图像创建 SOCI 索引
要索引自定义图像，您需要Linux主机(如Amazon弹性计算云(Amazon EC2))实例运行 Amazon Linux 2023，并安装了容器，SOCI快照器，并安装了芬奇或书呆克ctl.
- 设定环境变量。
- 安装所需的工具。
- 认证到Amazon ECR。
- 拉出你最初的形象
- 创建SOCI索引图像。
- 推指数图像。
这在您的寄存器中创建了两种文物：SOCI索引和符合OCI的图像索引表，将您的图像与索引联系起来。 您的原始形象及其文摘保持不变。
CI/CD 自动创建索引
为了在每次推力上自动生成索引，部署CFN ECR SOCI索引构建器。 它使用Amazon EventBridge规则来检测Amazon ECR图像推力事件并构建索引。 您的建设管道不需要更改。
在 SageMaker AI 工作中使用索引图像
引用您 SOCI 索引的图像与您其他图像完全一样。 不需要修改 API。 只有标记不同。
培训实例( SageMaker Python SDK)
校验您的图像是否包含 SOCI 索引
在发布之前，请确认您的标记解析到索引图像：
寻找儿童名单，其文物Type从应用/vnd.amazon.soci.index开始。 与您的平台显示。 如果它不存在，标记不会指向有索引的图像。
要记住的事
- Amazon ECR 仅限私人存储器。 其他登记册中的图像，或者通过私人的Docker登记册取出(Training Repository Access Mode set to Vpc)，不具备懒惰加载的资格。
- 有图像大小的改进尺度。 图像小于约4GB，可见相对较小的改进。
- 检查Docker的具体假设。 SOCI指数化的图像运行在一个基于容器的运行时间上。 审查密码的切入点和设置脚本，这些代码称Docker CLI，读作/var/run/docker.sock，或从cgroup路径得到标识符。 SageMaker AI 为后向相容性挂载了 /.dockerenv 标记，但避免添加新的依赖性。
- 所有AWS区都有SageMaker AI工作的SOCI索引。
结论
通过 SOCI 索引，您可以从您的 SageMaker AI 培训和处理任务中将容器图像下载时间最小化。 在您的图像中添加 SOCI 索引，或者切换到预索引的 AWS 深层学习容器。 无论哪种方式，你都将启动时间从分到秒，降低总的工作成本，并加快机器学习(ML)迭代周期。 而您现有的工作定义，API呼叫，和数据管道保持完全一致。
为了更多地了解Amazon SageMaker AI，并开始使用SOCI索引，探索以下资源：
- 选择一个SOCI指数化的深层学习容器。 浏览您图像标签中的“ 深层学习容器” 图片库和附件 - soci。 关于具体框架的指导，见AWS深层学习容器的SCI指数。
- 索引你自己的自定义图像。 遵循"SageMaker AI开发者指南"中的分步走过，或参考"SOCI快照"为高级选项启动的指南。
- 在你的CI/CD管道自动索引。 部署 CFN ECR SOCI 索引构建器，在每次图像推力上自动生成索引。 无需改变管道。
- 和芬奇进行局部测试利用芬奇懒散的穿行来验证你的形象在一个基于容器的运行时间在你提交工作之前。
- 潜入科技深处使用 DLAMI 上的 SOCI 索引和 AWS 机器学习 Blog 上的 DLC 来读取容器冷启动时间，或在引擎盖下探索：在 AWS 容器 Blog 上用可搜索的 OCI 和 AWS Fargate 来装入懒惰的容器图像，以详细了解懒惰装入机制。
更快的容器启动只差一个图像标记。 试试下次SageMaker AI的工作，看看区别
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Artificial Intelligence
Announcing SOCI-indexed images for faster SageMaker AI jobs
Every minute your deep learning container image spends downloading is a minute you pay for and a minute your team waits. Container image download is pure overhead: billable job time that produces no model progress. For modern deep learning images, that overhead is several minutes on every single job. Today, we’re excited to announce Seekable OCI (SOCI) indexing for Amazon SageMaker AI Training and Amazon SageMaker AI processing jobs, which takes full container image download off the critical path of every job.
With SOCI indexing, SageMaker AI lazy-loads your container image. It fetches only the files needed to start the container and streams the rest in the background as your code runs, so your job begins in seconds instead of minutes. You adopt it by pointing to a pre-indexed AWS Deep Learning Container (DLC) or adding an index to your own image. Neither approach requires changes to your job definition, entry point, or data channels. The result is faster job starts, lower job cost, and more time spent building models rather than waiting on infrastructure.
SOCI-indexed images start Amazon SageMaker AI training containers in seconds, with no impact on training performance. As we show in the What we measured section, this reduced GPU idle time during image pulls by up to 90 percent in our benchmarks.
This post shows how to adopt SOCI indexing: what it is, the results we measured, and how to turn it on for your own jobs.
The challenge
SageMaker AI training and processing jobs run inside container images stored in Amazon Elastic Container Registry (Amazon ECR). Before a job can execute your code, the service downloads the entire image to the instance. It decompresses and writes every layer to disk.
For small images this takes seconds. But deep learning images routinely exceed 10 GB or more. That download adds minutes to every job, and you pay for it as job time. The impact is largest exactly where iteration speed matters most: hyperparameter tuning sweeps for hundreds of jobs, quick experiments, and short processing jobs, where image download can dominate total runtime.
How SOCI indexing works
Seekable OCI (SOCI) is an open source technology that lazy loads container images. Instead of downloading the whole image before starting, the runtime uses a SOCI index to fetch only the files needed to launch the container immediately. It then streams the rest in the background as your code accesses it.
A SOCI index is metadata that maps every file inside your image. You store it in Amazon ECR as an OCI-compliant artifact linked to your original image. Because the index is a separate artifact, your image digest and any existing signatures remain valid.
SageMaker AI checks for an index automatically
When you create a training or processing job, SageMaker AI checks whether the image you specified has a SOCI index and behaves accordingly:
Because jobs always fall back to a standard download when needed, adding a SOCI index never prevents your job from running. You only gain faster startup. You never risk breaking an existing workflow.
What we measured
We pulled the same container images two ways on real SageMaker AI training jobs: standard download and SOCI-indexed lazy loading. We then ran full training jobs from both to measure the effect on training speed.
Setup
- Eight images, 1.8–35.4 GB compressed.
- GPU and CPU instances ml.g4dn, ml.g5, ml.g6, ml.g6e, ml.g7, ml.g7e, ml.p4d, ml.p5, and ml.m7i, from xlarge to 48xlarge, in the US West (Oregon) Region (us-west-2) and the US East (N. Virginia) Region (us-east-1).
- Download time measured from the start of the image pull until the container is ready to run. All figures are medians of repeated runs.
Result 1: Download time drops from minutes to seconds at every image size
The following chart compares container image download time for a standard Docker pull against SOCI lazy loading across eight image sizes, from 1.8 GB to 35.4 GB.
Each image size shows two bars. The Docker download time bar (blue) grows steadily with image size, from 48 seconds at 1.8 GB to 7 minutes 17 seconds at 35.4 GB.
A speedup label above each pair shows the ratio of Docker time to SOCI time. Speedups range from approximately 10x for the smallest image to over 35x for the largest.
Standard download time grows with image size because every file is decompressed and written. Lazy loading stayed at 3–11 seconds regardless of size, an 8x–46x improvement. The bigger the image, the more minutes you save.
Result 2: The saving holds on every instance type and size
Running the 13.2 GB image across 13 instance type/size combinations, standard download ranged from 3 min 16 s to 4 min 20 s, while lazy loading held at 6-8 s. That is about 34x faster on every configuration. You don’t need to change instance type or size to benefit.
Result 3: Training performance is unaffected beyond a first-epoch penalty
We ran two kinds of full training jobs from the Docker-pulled and from the SOCI-indexed image and compared every epoch:
- Image classification on ResNet-50 on Imagenette (9,469 images)
- Container images: PyTorch (13.2 GB DLC) and TensorFlow (9.9 GB DLC)
- Instances: ml.p5.48xlarge, ml.p4d.24xlarge, ml.g7e.48xlarge, ml.g7.48xlarge, ml.g6e.12xlarge, ml.g6e.24xlarge, and ml.g6e.48xlarge.
- Large language model (LLM) fine-tuning: full-parameter supervised fine-tuning (SFT) of Qwen2.5-0.5B on alpaca-cleaned (51,760 examples) with the Hugging Face Trainer.
- Container image: 9.4 GB Hugging Face PyTorch DLC.
- Instances: ml.p5.48xlarge, ml.p4d.24xlarge, ml.g6.48xlarge.
The following chart breaks job startup into image download time and first training epoch time, comparing a standard Docker pull with a SOCI-indexed image across GPU and CPU configurations for both workloads.
- From the second epoch on, time per epoch is the same (within run-to-run variation) on every instance, for both workloads.
- The first epoch takes 3–16 s longer from the lazy-loaded image, while files the framework touches for the first time are fetched. This first-epoch cost is the same on every instance size.
- Net: each job finished 1-6 min sooner. That is the download saving minus the one-time first-epoch cost, a fixed saving per job.
Test this on your own workload
These numbers are from the two workloads described earlier. The first-epoch cost depends on your image and what your code touches at startup:
- Lazy loading wins when the first training step needs only a fraction of the image, the rest streams in while you train.
- If your entry point reads most of the image before training starts (for example, scanning, checksumming, or preloading its contents), the first epoch waits for roughly the same bytes a full pull would have fetched. The net improvement would be small.
- Front-loaded starts still came out ahead in our tests:
- torch.compile and a just-in-time (JIT) built CUDA extension added 7-15 s to the first epoch (net saving still 3 min 25 s–3 min 56 s)
- A 6 GB model baked into the image streamed its weights at load time, adding 2-3.5 min, but the overall job still finished 2-2.5 mins sooner on the indexed image.
Getting started
You can adopt SOCI indexing two ways: use a pre-indexed AWS Deep Learning Container or create an index for your own custom image.
Option 1: Use a pre-indexed AWS Deep Learning Container
AWS publishes SOCI-indexed variants of several popular Deep Learning Containers, identified by the -soci suffix on the image tag:
Pre-indexed variants are available for PyTorch Training, TensorFlow Training, vLLM, and SGLang, including arm64 variants for some frameworks. To adopt lazy loading, update the image tag in your job definition to the -soci variant. Browse the full list in the AWS Deep Learning Containers Gallery.
Note: If you build your own image FROM a DLC base image, you must create a SOCI index for your final image. The index from the base image does not carry forward to derived images.
Option 2: Create a SOCI index for your own image
To index a custom image, you need a Linux host (such as an Amazon Elastic Compute Cloud (Amazon EC2) instance running Amazon Linux 2023) with containerd, the SOCI snapshotter, and either Finch or nerdctl installed.
- Set your environment variables.
- Install the required tools.
- Authenticate to Amazon ECR.
- Pull your original image.
- Create the SOCI-indexed image.
- Push the indexed image.
This creates two artifacts in your repository: the SOCI index and an OCI-compliant image index manifest linking your image to the index. Your original image and its digest remain unchanged.
Automate index creation in CI/CD
To generate indexes automatically on every push, deploy the CFN ECR SOCI Index Builder. It uses an Amazon EventBridge rule to detect Amazon ECR image push events and build the index. No changes to your build pipeline are required.
Use the indexed image in your SageMaker AI job
Reference your SOCI-indexed image exactly as you would any other image. No API changes are needed. Only the tag differs.
Training example (SageMaker Python SDK)
Verify that your image includes a SOCI index
Before launching, confirm your tag resolves to an indexed image:
Look for a child manifest whose artifactType begins with application/vnd.amazon.soci.index. alongside your platform manifest. If it isn’t there, the tag does not point to an indexed image.
Things to keep in mind
- Amazon ECR private repositories only. Images in other registries, or pulled through a private Docker registry (TrainingRepositoryAccessMode set to Vpc), are not eligible for lazy loading.
- Improvement scales with image size. Images smaller than roughly 4 GB see relatively small improvements.
- Check for Docker-specific assumptions. SOCI-indexed images run on a containerd-based runtime. Review entry points and setup scripts for code that calls the Docker CLI, reads /var/run/docker.sock, or derives identifiers from cgroup paths. SageMaker AI mounts a /.dockerenv marker for backward compatibility, but avoid adding new dependencies on it.
- SOCI indexing for SageMaker AI jobs is available across all AWS Regions.
Conclusion
With SOCI indexing, you can minimize container image download time from your SageMaker AI training and processing jobs. Add a SOCI index to your images, or switch to a pre-indexed AWS Deep Learning Container. Either way, you cut startup time from minutes to seconds, reduce total job cost, and accelerate your machine learning (ML) iteration cycle. And your existing job definitions, API calls, and data pipelines stay exactly the same.
To learn more about Amazon SageMaker AI and start using SOCI indexing, explore the following resources:
- Pick a SOCI-indexed Deep Learning Container. Browse the Deep Learning Containers Gallery and append -soci to your image tag. For framework-specific guidance, see SOCI index for AWS Deep Learning Containers.
- Index your own custom image. Follow the step-by-step walkthrough in the SageMaker AI Developer Guide, or refer to the SOCI snapshotter getting-started guide for advanced options.
- Automate indexing in your CI/CD pipeline. Deploy the CFN ECR SOCI Index Builder to generate indexes automatically on every image push. No pipeline changes are required.
- Test locally with Finch. Use the Finch lazy-loading walkthrough to validate your image on a containerd-based runtime before you submit a job.
- Dive deeper into the technology. Read Reducing container cold start times using SOCI index on DLAMI and DLC on the AWS Machine Learning Blog for benchmarks, or explore Under the hood: Lazy loading container images with Seekable OCI and AWS Fargate on the AWS Containers Blog to understand the lazy-loading mechanism in detail.
Faster container startup is only one image tag away. Try it on your next SageMaker AI job and see the difference.
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.