feat: add Datadog observability integration for AgentCore Runtime

01-tutorials/06-AgentCore-observability/04-Agentcore-runtime-partner-observability/Datadog/requirements.txt

@@ -0,0 +1,7 @@
+uv
+boto3
+bedrock-agentcore
+bedrock-agentcore-starter-toolkit
+strands-agents[otel]
+strands-agents-tools
+aws-opentelemetry-distro

01-tutorials/06-AgentCore-observability/04-Agentcore-runtime-partner-observability/Datadog/runtime_with_strands_and_datadog.ipynb

@@ -0,0 +1,366 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Strands Agent with Datadog Observability on Amazon Bedrock AgentCore Runtime\n",
+    "\n",
+    "## Overview\n",
+    "\n",
+    "This notebook demonstrates deploying a Strands agent to Amazon Bedrock AgentCore Runtime with **Datadog** observability. Traces are exported to Datadog's OTLP intake via ADOT (AWS Distro for OpenTelemetry) auto-instrumentation — no Datadog Agent sidecar required.\n",
+    "\n",
+    "## Key Components\n",
+    "\n",
+    "- **Strands Agents**: Python framework for building LLM-powered agents with built-in telemetry support\n",
+    "- **Amazon Bedrock AgentCore Runtime**: Managed runtime service for hosting and scaling agents on AWS\n",
+    "- **Datadog**: Full-stack observability platform with APM, tracing, and AI-focused monitoring\n",
+    "- **ADOT**: AWS Distro for OpenTelemetry — auto-instruments your agent and exports traces via OTLP\n",
+    "\n",
+    "## Prerequisites\n",
+    "\n",
+    "- Python 3.10+\n",
+    "- AWS credentials configured with Bedrock and AgentCore permissions\n",
+    "- Datadog account with an API key ([free trial](https://www.datadoghq.com/free-datadog-trial/))\n",
+    "- Docker installed locally (or use CodeBuild for remote builds)\n",
+    "- Access to Amazon Bedrock Claude models in your region"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 1: Install Dependencies\n",
+    "\n",
+    "Install the required packages from the `requirements.txt` file:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install --force-reinstall -U -r requirements.txt --quiet"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 2: Set Datadog Configuration\n",
+    "\n",
+    "Set your Datadog API key and site. These will be passed as OTEL environment variables to the agent runtime at deploy time.\n",
+    "\n",
+    "Replace `<YOUR_DATADOG_API_KEY>` with your actual Datadog API key."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import boto3\n",
+    "from boto3.session import Session\n",
+    "\n",
+    "boto_session = Session()\n",
+    "region = boto_session.region_name\n",
+    "\n",
+    "DD_API_KEY = \"\"  # Replace with your Datadog API key\n",
+    "DD_SITE = \"datadoghq.com\"  # Change if using a different Datadog site (e.g. datadoghq.eu)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 3: Write the Agent Code\n",
+    "\n",
+    "The agent uses Strands with a calculator tool (built-in) and a custom weather tool. Traces are handled automatically by ADOT — no manual OpenTelemetry setup needed in the agent code."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%writefile strands_datadog.py\n",
+    "\"\"\"Strands agent with Datadog observability on Amazon Bedrock AgentCore Runtime.\n",
+    "\n",
+    "Traces are exported to Datadog via ADOT auto-instrumentation.\n",
+    "All OTEL config is passed as environment variables at deploy time.\n",
+    "\"\"\"\n",
+    "\n",
+    "import os\n",
+    "import logging\n",
+    "from bedrock_agentcore.runtime import BedrockAgentCoreApp\n",
+    "from strands import Agent, tool\n",
+    "from strands.models import BedrockModel\n",
+    "from strands_tools import calculator\n",
+    "\n",
+    "logging.basicConfig(level=logging.INFO, format=\"[%(levelname)s] %(message)s\")\n",
+    "logger = logging.getLogger(__name__)\n",
+    "\n",
+    "\n",
+    "@tool\n",
+    "def weather() -> str:\n",
+    "    \"\"\"Get the current weather.\"\"\"\n",
+    "    return \"sunny\"\n",
+    "\n",
+    "\n",
+    "model_id = os.getenv(\"BEDROCK_MODEL_ID\", \"us.anthropic.claude-sonnet-4-20250514-v1:0\")\n",
+    "model = BedrockModel(model_id=model_id)\n",
+    "\n",
+    "agent = Agent(\n",
+    "    model=model,\n",
+    "    tools=[calculator, weather],\n",
+    "    system_prompt=\"You are a helpful assistant. You can do simple math calculations and tell the weather.\",\n",
+    ")\n",
+    "\n",
+    "app = BedrockAgentCoreApp()\n",
+    "\n",
+    "\n",
+    "@app.entrypoint\n",
+    "def handler(payload: dict, context=None) -> str:\n",
+    "    \"\"\"Handle incoming agent invocations.\"\"\"\n",
+    "    user_input = payload.get(\"prompt\", \"Hello!\")\n",
+    "    logger.info(\"Processing prompt: %s\", user_input[:100])\n",
+    "    response = agent(user_input)\n",
+    "    return response.message[\"content\"][0][\"text\"]\n",
+    "\n",
+    "\n",
+    "if __name__ == \"__main__\":\n",
+    "    app.run()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 4: Configure AgentCore Runtime Deployment\n",
+    "\n",
+    "Use the starter toolkit to configure the AgentCore Runtime deployment. ADOT auto-instrumentation is enabled by default — it reads the OTEL environment variables we pass at deploy time to route traces to Datadog."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from bedrock_agentcore_starter_toolkit import Runtime\n",
+    "\n",
+    "agentcore_runtime = Runtime()\n",
+    "agent_name = \"strands_datadog_observability\"\n",
+    "\n",
+    "response = agentcore_runtime.configure(\n",
+    "    entrypoint=\"strands_datadog.py\",\n",
+    "    auto_create_execution_role=True,\n",
+    "    auto_create_ecr=True,\n",
+    "    requirements_file=\"requirements.txt\",\n",
+    "    region=region,\n",
+    "    agent_name=agent_name,\n",
+    ")\n",
+    "response"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 5: Deploy to AgentCore Runtime\n",
+    "\n",
+    "Launch the agent with the OTEL environment variables that route traces to Datadog. ADOT reads these at container startup and auto-instruments all Bedrock model calls, tool invocations, and agent lifecycle spans."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "launch_result = agentcore_runtime.launch(\n",
+    "    auto_update_on_conflict=True,\n",
+    "    env_vars={\n",
+    "        \"AGENT_OBSERVABILITY_ENABLED\": \"true\",\n",
+    "        \"OTEL_EXPORTER_OTLP_PROTOCOL\": \"http/protobuf\",\n",
+    "        \"OTEL_EXPORTER_OTLP_TRACES_ENDPOINT\": f\"https://otlp.{DD_SITE}/v1/traces\",\n",
+    "        \"OTEL_EXPORTER_OTLP_TRACES_HEADERS\": f\"dd-api-key={DD_API_KEY},dd-otlp-source=llmobs\",\n",
+    "        \"OTEL_EXPORTER_OTLP_TRACES_PROTOCOL\": \"http/protobuf\",\n",
+    "        \"OTEL_PYTHON_CONFIGURATOR\": \"aws_configurator\",\n",
+    "        \"OTEL_PYTHON_DISTRO\": \"aws_distro\",\n",
+    "        \"OTEL_SERVICE_NAME\": \"agentcore-datadog-demo\",\n",
+    "    }\n",
+    ")\n",
+    "launch_result"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 6: Wait for Deployment\n",
+    "\n",
+    "Poll the runtime status until the endpoint is ready."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import time\n",
+    "\n",
+    "status_response = agentcore_runtime.status()\n",
+    "status = status_response.endpoint[\"status\"]\n",
+    "end_status = [\"READY\", \"CREATE_FAILED\", \"DELETE_FAILED\", \"UPDATE_FAILED\"]\n",
+    "\n",
+    "while status not in end_status:\n",
+    "    time.sleep(10)\n",
+    "    status_response = agentcore_runtime.status()\n",
+    "    status = status_response.endpoint[\"status\"]\n",
+    "    print(status)\n",
+    "\n",
+    "print(f\"\\nFinal status: {status}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 7: Invoke the Agent\n",
+    "\n",
+    "Send prompts to the deployed agent. Each invocation generates traces that are exported to both CloudWatch (automatic) and Datadog (via ADOT OTLP export)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "invoke_response = agentcore_runtime.invoke({\"prompt\": \"What is the weather now?\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from IPython.display import Markdown, display\n",
+    "display(Markdown(\"\".join(invoke_response[\"response\"])))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "invoke_response = agentcore_runtime.invoke({\"prompt\": \"What is 42 * 17?\"})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "display(Markdown(\"\".join(invoke_response[\"response\"])))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 8: View Traces in Datadog\n",
+    "\n",
+    "Open your Datadog APM dashboard and filter by `service:agentcore-datadog-demo`.\n",
+    "\n",
+    "You should see traces for each agent invocation with spans for:\n",
+    "- Agent invocation lifecycle\n",
+    "- Model calls (Bedrock `InvokeModel`)\n",
+    "- Tool execution (calculator, weather)\n",
+    "- Token usage and latency metrics\n",
+    "\n",
+    "<!-- TODO: Replace with actual screenshot -->\n",
+    "![Datadog APM Dashboard](images/datadog_dashboard.png)\n",
+    "*Datadog APM showing AgentCore agent traces with LLM and tool spans.*"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Cleanup (Optional)\n",
+    "\n",
+    "Delete the AgentCore Runtime and ECR repository."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Delete the AgentCore Runtime and ECR repository\n",
+    "agentcore_control_client = boto3.client(\"bedrock-agentcore-control\", region_name=region)\n",
+    "ecr_client = boto3.client(\"ecr\", region_name=region)\n",
+    "\n",
+    "# Delete the runtime\n",
+    "agentcore_control_client.delete_agent_runtime(\n",
+    "    agentRuntimeId=launch_result.agent_id,\n",
+    ")\n",
+    "\n",
+    "# Delete the ECR repository\n",
+    "ecr_client.delete_repository(\n",
+    "    repositoryName=launch_result.ecr_uri.split(\"/\")[1],\n",
+    "    force=True,\n",
+    ")\n",
+    "\n",
+    "print(\"Cleanup completed\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Summary\n",
+    "\n",
+    "You have successfully deployed a Strands agent to Amazon Bedrock AgentCore Runtime with Datadog observability. The implementation demonstrates:\n",
+    "\n",
+    "- ADOT auto-instrumentation routing traces to Datadog via OTLP env vars\n",
+    "- Direct OTLP export to Datadog (no sidecar agent needed)\n",
+    "- Dual observability: CloudWatch (automatic from AgentCore) + Datadog (via ADOT)\n",
+    "- Deployment using the AgentCore starter toolkit\n",
+    "\n",
+    "The agent is now running in a managed, scalable environment with full observability through Datadog APM."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "venv",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.14.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}