AWS introduces usage-based billing for streaming AI inference in Amazon SageMaker AI
AWS has launched usage-based metering for bidirectional streaming inference in Amazon SageMaker AI – model providers in AWS Marketplace can now charge for actual consumption during a live stream, not just a flat fee for availability. The launch partner is Deepgram with STT/TTS billing.
AWS has announced usage-based metering for bidirectional streaming inference on Amazon SageMaker AI. Model providers selling through AWS Marketplace can now charge a software fee based on actual consumption during a live stream, rather than just a flat fee for model availability. The launch partner is Deepgram, a company focused on real-time speech AI, which is thereby bringing usage-based billing for Speech-to-Text and Text-to-Speech services to AWS Marketplace.
The solution addresses the specifics of bidirectional streaming: the client streams audio and receives the transcription back over a WebSocket, a session can last from tens of seconds to tens of minutes, and the total consumption volume is only known after it ends. According to AWS, the standard mechanism, where the number of units is reported in the HTTP response header, only works for request-response APIs, where the volume is known at the moment of the response.
The technical solution consists of a second, concurrent WebSocket connection intended solely for consumption metadata. The model's server signals support for this feature with a special header and then sends JSON records with consumed units (the inference.count dimension, the ConsumedUnits field), recommended to be sent incrementally over the course of the session. The connection has a limit of one message per second and a maximum message size of 512 bytes, with the ClientToken field used for deduplicating records. SageMaker AI endpoint infrastructure fees remain separate and continue to be paid regardless of this new mechanism.
According to AWS, the new model brings consolidated billing through AWS Marketplace, including the option of negotiated pricing within private offers, without the need for a separate billing relationship with the model provider. The target audience is companies deploying models for transcription, contact centers, voice agents, or captioning.
Why it matters
For companies deploying streaming voice AI models from AWS Marketplace on SageMaker AI, the billing method for the software license is changing from a flat fee for availability to payment based on actual consumption – this can reduce costs for uneven usage, but can conversely increase them for high-load operations. The endpoint infrastructure fee, however, is not changing and continues to be calculated separately, so the total savings or cost increase needs to be assessed by adding both components together.
Relevant practical impact
What this means
For a business
Companies deploying streaming voice AI models (STT/TTS, transcription, voice agents, captioning) from AWS Marketplace on SageMaker AI will need to account for a new, usage-based billing model for the software license instead of a flat fee, alongside unchanged endpoint infrastructure charges.
Risks and complianceCheck the original
Event sources
clearly official source · 1 publisher, 0 independent. We count feeds from the same owner only once.