Skip to content
worth noting Productivity

ArisGlobal cut Amazon Bedrock inference costs by rearranging prompt caching

only one source so far

The company ArisGlobal changed the order of content in the prompts of its NavaX system on Amazon Bedrock using Claude models: instead of the document, it now caches the shared prompt prefix, which curbed the growth of inference costs as clients were added.

The company ArisGlobal runs NavaX, an AI tool for processing adverse drug reaction reports (pharmacovigilance), on the Amazon Bedrock platform using Claude models from Anthropic. The system has been in production since March 2025 and, according to the company, has processed over 1 million real transactions in a regulated pharmaceutical environment across multiple clients (tenants). From multi-page PDF reports, the assistant extracts roughly 130 fields – patient data, medication information, the timeline of the event, and regulatory classification.

As ArisGlobal added more pharmaceutical companies as clients, each required its own specific rules and formatting, so the shared opening block of the prompt (system rules, extraction schema, examples) grew over 6 months from approximately 72 000 to 180 000 tokens, i.e. by 150%. Meanwhile, inference costs per case rose by more than 200%. The cause was the original setup, in which the cache mainly stored the case's source document (because it was the largest in volume), while the growing shared prompt prefix remained outside the cache and was processed at full rate with every request.

ArisGlobal reversed the order of content in the request: the shared prompt prefix is now sent first, followed by a cache checkpoint, and only after that comes the case-specific document. As a result, the cache now retains precisely the part that repeats across requests, cases, and clients, whereas previously the content unique to each case was being cached instead. This reduced the effective cost of input tokens, and the company says it was able to support additional clients without a proportional increase in costs. The solution also uses Amazon Bedrock Guardrails to protect sensitive health data (PHI), and routing through the EU region remained unchanged.

What changed

Why it matters

This case demonstrates a concrete technique for models with prompt caching (here, Claude on Amazon Bedrock) to prevent inference costs from growing faster than the number of clients or the volume of data. It is relevant for teams running multi-tenant AI services with a shared but gradually growing system prompt – the wrong order of content in a request can completely negate the cache, while the right order enables scaling with sublinear cost growth.

Two audiences, two different impacts

What this means

01

For individuals

For developers designing LLM applications with prompt caching, it's instructive that the order of content in a prompt determines what can be reused from the cache – what should be cached is what repeats across requests, not what changes with each case.

What to do When designing prompts with caching, place shared, repeating content (system instructions, schema) before request-specific content, not the other way around.
More practical updates →
02

For a business

Companies running multi-tenant AI services with a growing shared prompt (e.g., due to specific requirements of individual clients) can, by rearranging prompt caching, stop the proportional growth of inference costs as clients are added.

Development
What to decide Verify the ordering of cached content in prompts for custom multi-tenant deployments on Amazon Bedrock or a similar platform.
More business impacts →
Amazon Bedrock ArisGlobal Claude Pharmacy NavaX Prompt caching

Check the original

Event sources

only one source so far · 1 publisher, 0 independent. We count feeds from the same owner only once.

1
AWS Machine Learning Blog primary source · first detected How ArisGlobal scaled generative AI inference on Amazon Bedrock