Amazon SageMaker AI introduces SOCI indexing for faster job startup
Amazon SageMaker AI now supports SOCI indexing for training and data processing. The container starts before the entire image has been downloaded. AWS reports that internal measurements showed a reduction in GPU idle time during downloads of up to 90 %.
AWS announced support for SOCI indexing for training and data processing jobs in Amazon SageMaker AI. The service first downloads only the files needed to start the container, then loads the remaining image contents in the background while the code runs. Seekable OCI (SOCI) is open source.
According to internal measurements by AWS, GPU idle time during image downloads fell by up to 90 %. For a compressed image of 35.4 GB, the company measured 7 minutes and 17 seconds for a standard download, while startup using SOCI took 3–11 seconds. The company says subsequent training performance was unaffected; these results have not been independently verified.
Users can choose pre-indexed AWS Deep Learning Containers images with the “-soci” tag suffix or create an index for their own image. The available variants include PyTorch Training, TensorFlow Training, vLLM and SGLang. Changing the image tag requires no changes to the entry point or data channels. The index is stored in Amazon ECR separately from the image, so the image digest and existing signatures remain valid. If the index is unreadable or loading fails, the service falls back to a standard download. Details of the procedure for custom images are available in the source article.
Why it matters
According to the source, image downloads count toward billed job time, even though training has not yet begun during this phase. Shortening this phase may therefore reduce costs, particularly for short jobs and hyperparameter tuning involving hundreds of repeated runs. However, the reported improvement in startup speed should not be taken to mean an equivalent speedup of the entire job.
Two audiences, two different impacts
What this means
For individuals
A developer running short experiments may reduce the wait for them to start by using an indexed image, without modifying their own training code.
For a business
Teams running large numbers of training or processing jobs may reduce the paid time spent downloading images. Actual savings depend on how much of the total runtime this phase accounts for.
ProcessesCheck the original
Event sources
clearly official source · 1 publisher, 0 independent. We count feeds from the same owner only once.