AWS introduced the Agentic Data Operations Platform reference architecture to automate data engineering using agents in Claude Code

AWS introduced Agentic Data Operations Platform (ADOP) — a reference architecture built on Amazon Bedrock that aims to significantly reduce the time needed to onboard a new data source using specialized AI agents. According to AWS, the agents automate the entire data lifecycle from Bronze through Silver to Gold, including the creation of ETL code, data quality checks, semantic models and regulatory checks — tasks that, according to AWS, typically take data engineering teams weeks of manual work.

The architecture runs the so-called Data Onboarding Agent in Claude Code through Amazon Bedrock and uses the Dynamic Workflow feature in Claude Code to launch specialized sub-agents for individual stages of pipeline construction — metadata generation, data ontology inference, data quality checks, ETL transformations and orchestration using Airflow or AWS Step Functions. The architecture includes a so-called Decision Engine intended to encode the standards and design philosophy of a given organization into the construction process, along with a set of guardrails in the form of tool routing rules, Cedar authorization policies and ongoing compliance prompts. The reference implementation targets AWS, but according to AWS, the framework can also be extended to other services through CLI interfaces or Model Context Protocol for hybrid and multicloud environments. In addition to Claude Code, the agents also integrate with Kiro, Cursor and Codex.

A key design decision is the separation of development from operations: agents run in development environments, where they generate code that engineers review, and only deterministic artifacts are deployed to production through CI/CD — PySpark and SQL code, an Airflow DAG and IAM/Cedar policies. By default, the model is not called in production; organizations that also need model inference at runtime can extend the architecture using Amazon Bedrock endpoints, but the generated pipeline code itself remains static and auditable. Agent decisions are recorded using AgentTrace and can be published to Amazon CloudWatch or OpenTelemetry for auditing purposes; when scaling is needed, the workload can be moved to the AgentCore runtime within Amazon Bedrock AgentCore without changing the architecture.

According to AWS, teams that adopted this approach significantly reduced the time needed to onboard additional data sources after the initial architecture setup, but the source provides no specific figures. Details on the next deployment steps can be found in the source article.