A modular repository demonstrating how to build an agentic platform including labs and a deployable sample "agentic platform" to demonstrate how to operatinalize agents from multiple open source frameworks as well as Bedrock agents.
This sample is active as of 4/22/2025.
The sample platform is built on EKS uses a variety of AWS services to deploy 10+ agentic systems demonstrating how everything works together. It's instrumented with telemetry and demonstrates operational considerations when deploying agents. Code is written in an abstracted/modular way to make it easy to switch the underlying infrastructure components to suite your needs.
Each agent runs as a FastAPI server sharing a core package which contains types and client abstrations (like LLM API provider). APIs are authenticated using Cognito through a middleware layer that's simple to swap out for your own IDP.
The agents themselves do not have IAM roles attached to them. Instead they connect to AWS resources through microservices like the LLM Gateway, Memory Gateway, and Retrieval Gateway (which do have IAM roles through IRSA). Those requests are authenticated by passing JWT tokens between each request. Each pod running in EKS is authenticated using oAuth regardless of whether the request was service <> service or user <> service by validating tokens against the IDP's public cert.
Lastly, telemetry information is collected using open telemetry collectors and pushed to X-Ray for traces, Cloudwatch for metrics, and OpenSearch for logs. An agent uses the observability facade (in the common package) which is pre-configured to send to our open telemetry endpoints. The OTEL collectors then push the telemetry data to any endpoint that supports open telemetry protocol (OTLP). This makes it easy to switch vendors or different services. Anything that supports OTLP will work.
You'll need to have terraform installed and the AWS CLI installed in the environment you want to deploy this code from
- Install Terraform using tfenv
- Install AWS CLI & configure
- Install SSM Plugin for the AWS CLI for Port Forwarding
- "Optionally" configure a bucket and dynamoDB table to store terraform state.
- Install uv for local python development.
- Install Docker
- Install kubectl
Run git clone on this repo:
git clone https://github.com/aws-samples/sample-agentic-platform.gitImportant Notice: This project deploys resources in your AWS environment using Terraform. You will incur costs for the AWS resources used. Please be aware of the pricing for services like EKS, Bedrock, OpenSearch, DynamoDB, Elasticache, S3, etc.. in your AWS region.
Required Permissions: You need elevated permissions, to deploy the Terraform stack.
Please refer to the DEPLOYMENT.md file for details on how to deploy.
There are 5 modules in total that increase in complexity going from the basics to operational considerations when running an agent platform at scale. Only module 5 needs the agent platform deployed to fully run the notebooks. To deploy the agent platform skip to the Deploy Sample Agent Platform section.
You can run the labs locally or within the bastion host. In the deployment, we've provided a code server that can be accessed in your browser by port forwarding through SSM.
If running the lab locally, you'll need to have UV installed:
- Install uv for local python development.
When you have your environment setup, go into the sample-agentic-platform directory and run uv sync.
uv sync If you're running the lab locally, run the following command to start the lab
uv run jupyter labMake sure to run checkov, bandit, and gitleaks if making additional changes to the code.
Suppressed Warnings:
There are a number of warnings suppressed in the code base. Review these prior to using any of the code in your environment. For example, we suppressed warnings on automatic secret rotation in AWS Secrets Manager for the redis and postgres passwords.
See CONTRIBUTING for more information.
We are open to contributions and this project is actively being worked on. A couple things on our roadmap include:
- Making deployment easier through a bootstrap terraform module
- Cleaning up deployments with a more structured approach to GitOps
- Adding additional labs on more advanced agent topics.
- Building a test harness & eval suite that runs against the code base
- Adding more agent examples from the labs into the sample platform
- Tanner McRae
- Randy DeFauw
- James Levine
This library is licensed under the MIT-0 License. See the LICENSE file.