AWS releases templates for a centralized data lake over Amazon Bedrock invocation logs
Amazon has published open-source templates and code for a centralized data lake that aggregates Amazon Bedrock invocation logs across AWS accounts and makes it possible to track prompts, responses, token counts, and costs at the level of individual invocations.
Amazon published a solution called AWS Sample Bedrock Invocation Data Lake on the AWS Machine Learning Blog, which addresses centralized visibility into Amazon Bedrock model invocations across multiple AWS accounts. According to the company, the solution emerged as a response to a situation where platform and ML engineers see only an aggregated Bedrock bill but have no insight into what happened inside individual invocations – what prompt was sent, what the model returned, how many tokens it consumed, which model served the invocation, and what the quality of the response was.
The architecture transfers logs from individual linked AWS accounts via Amazon CloudWatch using subscription filters to a central account, where Amazon Data Firehose processes them (converting from JSON to Parquet format, storing in Amazon S3 by date) and then exposes them via AWS Glue and Amazon Athena as a single queryable table. The entire deployment is handled as infrastructure-as-code using two AWS CloudFormation templates – one for the central account and one for the linked source accounts, with the central stack needing to be deployed first.
Once deployed, the solution provides a per-invocation overview covering prompts, responses, token counts, and derived costs, and according to the company allows moving from a generic "how many tokens did we use" to a detailed overview of what individual model invocations contained and how efficient they were. According to the article, the solution can also be connected to the AWS Cost and Usage Report for tracking expenses by prompt over time. The code, templates, and SQL queries are available as open source in the AWS Sample Bedrock Invocation Data Lake GitHub repository; details on deployment prerequisites and the complete procedure can be found in the source article.
Why it matters
For companies running Amazon Bedrock across multiple AWS accounts (e.g., by team or environment), the solution eliminates the blind spot between the summary bill and the actual content of individual invocations. It makes it possible to trace which teams or prompts generate the highest costs, identify inefficient or unnecessarily long prompts, and allocate expenses between organizational units without having to build custom logging infrastructure from scratch.
Relevant practical impact
What this means
For a business
Companies running Amazon Bedrock across multiple AWS accounts get a ready-made solution for tracking costs, prompts, and responses on a per-invocation basis, which makes it possible to identify waste and allocate costs between teams.
Risks and complianceCheck the original
Event sources
only one source so far · 1 publisher, 0 independent. We count feeds from the same owner only once.