Skip to content
AI Radar by Wagner Solution
CS
Menu
worth noting Coding

AWS outlines fine-tuning of Qwen2.5-1.5B on eight GPUs on Amazon EKS

only one source so far

A guide from AWS demonstrates distributed fine-tuning of Qwen2.5-1.5B using Ray Train DLC, KubeRay and EFA on Amazon EKS. The example uses eight NVIDIA L4 GPUs and, according to AWS, does not require building a custom container image.

AWS has published a guide to distributed fine-tuning of Qwen2.5-1.5B using PyTorch Fully Sharded Data Parallel (FSDP). The example runs on eight NVIDIA L4 GPUs across two g6.12xlarge instances on Amazon EKS. KubeRay creates and maintains the head and worker pods according to the RayCluster resource declaration.

The setup is based on the prebuilt Ray Train DLC container image tagged train-ml-cuda-v1.1. It includes Ray 2.58.0, PyTorch 2.13.0, CUDA 13.0.2 and NCCL 2.29.7, as well as libraries for communication over EFA. According to AWS, these components are built and tested together, and the example does not require building a custom image.

The procedure separates the training function, the RayCluster manifest and the file containing cluster configuration variables. Each worker pod requests four GPUs and one EFA device; network communication is configured through environment variables. The same image is used for both the head and worker pods, and any additional libraries can be added when submitting the job through runtime_env. Details are available in the source article.

What changed

Why it matters

The guide provides a concrete starting point for teams looking to distribute fine-tuning across multiple machines. It also addresses network configuration: according to AWS, mismatched communication libraries can cause a fallback to TCP and a slowdown, even though the job completes without an error.

Two audiences, two different impacts

What this means

01

For individuals

A developer can prepare an experiment using a single training function and a cluster manifest; the model, number of steps, sequence length and batch size are set through arguments to the training script.

What to do When trying the example, check that the manifest allocates four GPUs and one EFA device to each worker pod.
More practical updates →
02

For a business

For a team managing training infrastructure, the prebuilt image offers a way to reduce the work involved in building images and aligning dependencies. AWS says it continually patches the image and validates its releases with multi-node training tests.

Development
What to decide For reproducible deployments, pin the container image version using a specific tag or digest.
More business impacts →
Amazon EKS EFA KubeRay PyTorch FSDP Qwen2.5-1.5B Ray Train DLC

Check the original

Event sources

only one source so far · 1 publisher, 0 independent. We count feeds from the same owner only once.

1
AWS Machine Learning Blog primary source · first detected Distributed training with Ray Train DLCs and EFA on EKS