AWS describes a process for deploying and comparing models in Amazon SageMaker JumpStart
AWS published a process for deploying a model in Amazon SageMaker JumpStart, running repeatable tests using SageMaker Python SDK and then deleting the endpoint. The comparison tracks latency, throughput and output quality.
AWS described a process on its blog that uses a reusable Python script: deploy an available model from the Amazon SageMaker JumpStart catalog, run a test using SageMaker Python SDK and delete the endpoint when finished. The stated requirement for the sagemaker package is a version of at least 3 and below 4.
Model comparisons should use the same representative test inputs and track latency, throughput and output quality. According to AWS, this process can be used for a range of compatible models, but input formats, response processing, the required instance type and EULA licensing terms may require individual adjustments. Deployment and its duration also depend on quotas, available capacity and the AWS region. Details are available in the source article.
Why it matters
The choice of a model for an application can be based on its behavior on representative tasks. Shared test inputs make it possible to assess the trade-off between speed and response quality; however, differences in request and response formats require adjustments to the test script.
Two audiences, two different impacts
What this means
For individuals
Developers get a concrete process for repeatedly comparing candidate models when choosing one for their own application.
For a business
Testing models in a business setting requires planning for compute instance availability, quotas and the deployment region, and checking any applicable EULA licensing terms. The process includes deleting the endpoint after testing.
DevelopmentCheck the original
Event sources
only one source so far · 1 publisher, 0 independent. We count feeds from the same owner only once.