AWS expands Amazon SageMaker Inference with Inference Gateway, managed NVIDIA Dynamo, and cross-region routing
The company AWS has released new features for Amazon SageMaker Inference: Inference Gateway with managed NVIDIA Dynamo, cross-region and cross-account routing of endpoints, and expanded configuration recommendations including support for LoRA adapters.
Companies running inference on SageMaker AI gain tools for more accurate GPU capacity planning (number of LoRA adapters, maximum concurrency under SLA) and for cross-region/cross-account routing, which reduces the risk of downtime and helps keep GPU costs under control as traffic grows.