diff --git a/databricks-skills/databricks-genie-bedrock-agentcore/1-architecture-and-auth.md b/databricks-skills/databricks-genie-bedrock-agentcore/1-architecture-and-auth.md new file mode 100644 index 00000000..d8e7af7d --- /dev/null +++ b/databricks-skills/databricks-genie-bedrock-agentcore/1-architecture-and-auth.md @@ -0,0 +1,100 @@ +# Architecture and Authentication + +## Component map + +| Component | Provided by | Role | +| --- | --- | --- | +| Bedrock Agent (Claude / Nova) | AWS | Conversational LLM, calls `query_genie` as a tool | +| AgentCore Gateway | AWS | Hosts the Genie MCP target, brokers tool calls | +| AgentCore Identity | AWS | OAuth credential provider — mints Databricks tokens for the Gateway | +| CloudWatch | AWS | Audit and tracing | +| Managed MCP Server | Databricks | `{DATABRICKS_HOST}/api/2.0/mcp/genie/{space_id}` | +| Genie Space + Trusted Assets | Databricks | NL → SQL with metric definitions | +| Unity Catalog | Databricks | Governance, lineage, audit attribution | +| Delta Lake | Databricks | Governed tables backing the Genie space | + +## End-to-end flow (OBO mode) + +1. End user sends an NL question to the Bedrock agent +2. The agent decides to call `query_genie` (an MCP tool registered via AgentCore Gateway) +3. AgentCore Gateway looks up the credential provider configured on the target +4. AgentCore Identity exchanges the user's session for a Databricks bearer token via the OAuth `authorization_code` flow against the Databricks workspace OAuth app +5. The Gateway calls `POST {DATABRICKS_HOST}/api/2.0/mcp/genie/{space_id}/tools/query_genie` with that bearer token +6. Databricks validates the token, executes the Genie conversation, runs the resulting SQL through Unity Catalog with the end-user's grants enforced +7. Result (SQL, narrative, rows) flows back to the agent +8. UC audit log attributes the SQL to the human end user + +## Two auth modes + +### `obo` (recommended for production) + +End-user identity propagates via OAuth U2M. + +- `grant_type=authorization_code` +- Scopes: `all-apis offline_access` +- Requires: a Databricks workspace OAuth app (custom integration) with a redirect URI matching what AgentCore Identity exposes + +What you can claim: +- "Genie answers reflect each user's UC permissions" — true +- "UC audit logs name the human end user" — true +- "No data movement" — true + +Caveats to disclose: +- The redirect URI is a two-pass setup (placeholder first, real URI after AgentCore provisions the credential provider). Use `scripts/sync_oauth_redirect.py` to close the loop. +- Refresh-token reuse / per-user TTL is roadmap (v0.2 in the reference repo) + +### `m2m` (booth-demo quick start) + +Every Genie call uses a Databricks service principal. **Do not claim user-level governance in this mode.** + +- `grant_type=client_credentials` +- Scopes: `all-apis` +- Requires: a Databricks SP with workspace + UC read on the Genie space's underlying tables + +What you can claim: +- "Bedrock agent answers governed numerical questions" — true (the SP has UC grants) +- "Metric definitions are consistent" — true (Trusted Assets remain authoritative) + +Caveats to disclose: +- All callers see the same rows the SP can see; no per-user differentiation +- UC audit attributes SQL to the SP, not to the end user +- Acceptable for single-tenant testing; never ship as the production posture + +## OAuth credential provider configuration + +The AgentCore Identity OAuth credential provider is the linchpin. In Terraform (`awscc_bedrockagentcore_oauth_credential_provider`) or CloudFormation (`AWS::BedrockAgentCore::OAuthCredentialProvider`): + +```hcl +resource "awscc_bedrockagentcore_oauth_credential_provider" "databricks" { + name = "${local.name}-databricks-oauth" + credential_provider_vendor = "CustomOauth2" + oauth2_provider_config_input = { + custom_oauth2_provider_config = { + client_id = var.auth_mode == "obo" ? var.databricks_oauth_client_id : var.databricks_client_id + client_secret = var.auth_mode == "obo" ? var.databricks_oauth_client_secret : var.databricks_client_secret + authorization_endpoint = "${var.databricks_host}/oidc/v1/authorize" + token_endpoint = "${var.databricks_host}/oidc/v1/token" + scopes = var.auth_mode == "obo" ? ["all-apis", "offline_access"] : ["all-apis"] + grant_type = var.auth_mode == "obo" ? "authorization_code" : "client_credentials" + } + } +} +``` + +The Databricks OAuth credentials live in Secrets Manager so the Gateway can fetch them at tool-invocation time. The Gateway target IAM role needs `secretsmanager:GetSecretValue` on the secret ARNs and `s3:GetObject` on the schema bucket. + +## Identity flow gotchas + +- **Redirect URI mismatch (OBO):** the most common cause of `403` on UC tables. After Terraform/CFN provisions the credential provider, run `aws bedrock-agentcore-control get-oauth-credential-provider` to read the redirect URI, then update the Databricks OAuth app to match. The reference repo's `scripts/sync_oauth_redirect.py` automates this via the Databricks Account API. +- **`DATABRICKS_OAUTH_PROVIDER_ARN` confusion:** this env var is the AgentCore Identity OAuth credential provider ARN, **not** a Secrets Manager ARN. Treating these as the same thing is a common bug — `register_gateway.py` requires the credential provider ARN. +- **AgentCore schema naming:** `AWS::BedrockAgentCore::OAuthCredentialProvider` and the corresponding `awscc_*` attribute names evolved during preview. If `terraform init` or `cfn deploy` errors on a renamed property, check the current CFN registry. + +## Governance validation + +After deploying in OBO mode, validate that user-level governance actually works before claiming it in a customer demo: + +1. Run the same question as **two distinct end users** with different UC grants on the underlying tables +2. Confirm they get **different rows** (or `403` for the user without grants) +3. Inspect UC audit logs in `system.access.audit` — confirm the SQL is attributed to the **human end user**, not to the OAuth app's client ID + +If any of those checks fail, the OBO wiring is wrong even if smoke tests pass with one user. diff --git a/databricks-skills/databricks-genie-bedrock-agentcore/2-deployment-tf-vs-cfn.md b/databricks-skills/databricks-genie-bedrock-agentcore/2-deployment-tf-vs-cfn.md new file mode 100644 index 00000000..a7b39467 --- /dev/null +++ b/databricks-skills/databricks-genie-bedrock-agentcore/2-deployment-tf-vs-cfn.md @@ -0,0 +1,190 @@ +# Deployment: Terraform vs CloudFormation + +Two IaC paths, picked based on which AWS account you're deploying into. + +## Decision matrix + +| Account type | Path | Why | +| --- | --- | --- | +| External customer AWS account | Terraform | Caller has direct IAM/Bedrock-AgentCore privileges; `awscc` provider gives Terraform-native AgentCore primitives | +| FE Sandbox AWS account | CloudFormation | Caller's SSO role lacks `iam:CreateRole` and `bedrock-agentcore:*` — must deploy via a pre-blessed exec role | +| Other restricted AWS account | CloudFormation | Same reason as FE Sandbox; the exec-role pattern generalizes | +| Customer is mandated CloudFormation-only | CloudFormation | Some enterprises prohibit Terraform — meet them where they are | + +## What both paths provision + +| Resource | Purpose | +| --- | --- | +| S3 bucket | Holds the Genie MCP OpenAPI schema | +| Secrets Manager (M2M) | Databricks SP credentials | +| Secrets Manager (OBO) | Databricks workspace OAuth app credentials | +| IAM role for Gateway target | `secretsmanager:GetSecretValue`, `s3:GetObject` | +| AgentCore OAuth credential provider | Mints Databricks tokens (M2M or OBO depending on `AUTH_MODE`) | +| AgentCore Gateway | Hosts the Genie MCP target | +| IAM role for Bedrock agent | `bedrock:InvokeModel`, `bedrock-agentcore:InvokeGateway` | +| Bedrock agent + alias | The conversational runtime | + +## Terraform path (external AWS) + +Uses the `awscc` provider for AgentCore primitives because the AWS provider doesn't ship native `aws_bedrockagentcore_*` resources at the time of writing. + +```hcl +terraform { + required_providers { + aws = { source = "hashicorp/aws", version = "~> 5.60" } + awscc = { source = "hashicorp/awscc", version = "~> 1.0" } + } +} +``` + +Deploy: + +```bash +cd terraform +terraform init +terraform apply \ + -var "databricks_host=$DATABRICKS_HOST" \ + -var "databricks_client_id=$DATABRICKS_CLIENT_ID" \ + -var "databricks_client_secret=$DATABRICKS_CLIENT_SECRET" \ + -var "databricks_oauth_client_id=$DATABRICKS_OAUTH_CLIENT_ID" \ + -var "databricks_oauth_client_secret=$DATABRICKS_OAUTH_CLIENT_SECRET" \ + -var "genie_space_id=$GENIE_SPACE_ID" \ + -var "auth_mode=$AUTH_MODE" +``` + +Outputs to capture into `.env`: `gateway_id`, `oauth_provider_arn`, `schema_s3_bucket`, `bedrock_agent_id`, `bedrock_agent_alias_id`. + +## CloudFormation path (FE Sandbox / restricted accounts) + +### The pre-blessed exec-role pattern + +The problem: on FE Sandbox the SSO role typically has + +- ✅ `cloudformation:*`, `s3:*` +- ❌ `iam:CreateRole`, `iam:PutRolePolicy` +- ❌ `bedrock-agentcore:CreateGateway`, `CreateOauthCredentialProvider` +- ❌ `bedrock:CreateAgent` + +That blocks Terraform-as-you AND `aws cloudformation deploy --capabilities CAPABILITY_NAMED_IAM` from the SSO role. The workaround (Ioannis Papadopoulos's pattern, originated for the Agent Bricks ↔ Bedrock/AgentCore demo): + +1. **One-time, by an account admin:** create an IAM role `cfn-bedrock-agentcore-deployer` with all the privileges CFN needs. Trust policy lets your SSO role call `sts:AssumeRole`. +2. **Each deploy:** `aws cloudformation deploy --role-arn $CFN_DEPLOYER_ROLE_ARN ...` — CFN executes as the pre-blessed role. +3. **You only need:** `cloudformation:*` and `sts:AssumeRole` on the deployer role. Both usually allowed on FE Sandbox. + +### IAM policy for the deployer role + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "S3SchemaBucket", + "Effect": "Allow", + "Action": [ + "s3:CreateBucket", "s3:DeleteBucket", "s3:GetBucketLocation", + "s3:GetBucketPolicy", "s3:GetBucketPublicAccessBlock", + "s3:GetBucketVersioning", "s3:ListBucket", "s3:PutBucketPolicy", + "s3:PutBucketPublicAccessBlock", "s3:PutBucketVersioning", + "s3:DeleteObject", "s3:GetObject", "s3:PutObject" + ], + "Resource": [ + "arn:aws:s3:::dbx-genie-mcp-*", + "arn:aws:s3:::dbx-genie-mcp-*/*" + ] + }, + { + "Sid": "SecretsManager", + "Effect": "Allow", + "Action": [ + "secretsmanager:CreateSecret", "secretsmanager:DeleteSecret", + "secretsmanager:DescribeSecret", "secretsmanager:GetSecretValue", + "secretsmanager:PutSecretValue", "secretsmanager:TagResource", + "secretsmanager:UntagResource", "secretsmanager:UpdateSecret" + ], + "Resource": "arn:aws:secretsmanager:*:*:secret:dbx-genie-mcp-*" + }, + { + "Sid": "IAM", + "Effect": "Allow", + "Action": [ + "iam:AttachRolePolicy", "iam:CreateRole", "iam:DeleteRole", + "iam:DeleteRolePolicy", "iam:DetachRolePolicy", "iam:GetRole", + "iam:GetRolePolicy", "iam:ListAttachedRolePolicies", + "iam:ListRolePolicies", "iam:PassRole", "iam:PutRolePolicy", + "iam:TagRole", "iam:UntagRole", "iam:UpdateAssumeRolePolicy" + ], + "Resource": ["arn:aws:iam::*:role/dbx-genie-mcp-*"] + }, + { + "Sid": "BedrockAgentCore", + "Effect": "Allow", + "Action": [ + "bedrock-agentcore:CreateGateway", "bedrock-agentcore:DeleteGateway", + "bedrock-agentcore:GetGateway", "bedrock-agentcore:ListGateways", + "bedrock-agentcore:UpdateGateway", + "bedrock-agentcore:CreateGatewayTarget", + "bedrock-agentcore:DeleteGatewayTarget", + "bedrock-agentcore:GetGatewayTarget", + "bedrock-agentcore:ListGatewayTargets", + "bedrock-agentcore:UpdateGatewayTarget", + "bedrock-agentcore:CreateOauthCredentialProvider", + "bedrock-agentcore:DeleteOauthCredentialProvider", + "bedrock-agentcore:GetOauthCredentialProvider", + "bedrock-agentcore:ListOauthCredentialProviders", + "bedrock-agentcore:UpdateOauthCredentialProvider" + ], + "Resource": "*" + }, + { + "Sid": "BedrockAgent", + "Effect": "Allow", + "Action": [ + "bedrock:CreateAgent", "bedrock:CreateAgentAlias", + "bedrock:DeleteAgent", "bedrock:DeleteAgentAlias", + "bedrock:GetAgent", "bedrock:GetAgentAlias", + "bedrock:ListAgents", "bedrock:ListAgentAliases", + "bedrock:PrepareAgent", "bedrock:UpdateAgent", + "bedrock:UpdateAgentAlias", + "bedrock:AssociateAgentKnowledgeBase", + "bedrock:DisassociateAgentKnowledgeBase", + "bedrock:TagResource", "bedrock:UntagResource" + ], + "Resource": "*" + } + ] +} +``` + +The policy is resource-scoped to `dbx-genie-mcp-*` so the deployer role can't be reused against unrelated infra. A copy-paste Slack template for the admin ask is at `cloudformation/REQUEST_DEPLOYER_ROLE.md` in the reference repo. + +### Deploy + +```bash +export CFN_DEPLOYER_ROLE_ARN=arn:aws:iam:::role/cfn-bedrock-agentcore-deployer +cd cloudformation +./deploy.sh +``` + +`deploy.sh` reads `.env`, then runs: + +```bash +aws cloudformation deploy \ + --region "$AWS_REGION" \ + --stack-name "$STACK_NAME" \ + --template-file stack.yaml \ + --capabilities CAPABILITY_NAMED_IAM \ + --role-arn "$CFN_DEPLOYER_ROLE_ARN" \ + --parameter-overrides ... +``` + +Outputs are surfaced via `aws cloudformation describe-stacks --query 'Stacks[0].Outputs' --output table` — capture into `.env`. + +## Picking between the two + +- If you might run this on FE Sandbox at any point, **maintain the CloudFormation path** as the source of truth and treat Terraform as the convenience layer. Sandbox is the harder constraint. +- If you're only deploying to external customer AWS accounts, **stick with Terraform** — it's the more expressive of the two, and customers running Terraform shops will prefer it. +- The reference repo (`databricks-aws-integrations/genie_with_bedrock_agentcore`) keeps both paths to cover both audiences. Drift between them is the primary maintenance cost — when changing one, change the other. + +## Schema-name caveat + +`AWS::BedrockAgentCore::*` resource names and properties evolved through preview. The IaC referenced here was authored against the 2026-05 CFN registry. If `terraform init` errors on `awscc_bedrockagentcore_*`, or `aws cloudformation validate-template` errors on a `BedrockAgentCore::*` property, check the current registry schema and patch. diff --git a/databricks-skills/databricks-genie-bedrock-agentcore/3-quickstart-and-scripts.md b/databricks-skills/databricks-genie-bedrock-agentcore/3-quickstart-and-scripts.md new file mode 100644 index 00000000..9bc884a3 --- /dev/null +++ b/databricks-skills/databricks-genie-bedrock-agentcore/3-quickstart-and-scripts.md @@ -0,0 +1,134 @@ +# Quickstart and Helper Scripts + +End-to-end deployment recipe using the reference repo at `databricks-field-eng/databricks-aws-integrations` (folder: `genie_with_bedrock_agentcore/demo`). Five helper scripts cover the parts that would otherwise live on a manual checklist. + +## Step 0: Capture identifiers + +| Variable | Where to find | +| --- | --- | +| `DATABRICKS_HOST` | Workspace URL (no trailing slash), e.g. `https://e2-demo-field-eng.cloud.databricks.com` | +| `GENIE_SPACE_ID` | Genie UI → open space → URL contains `/spaces/` | +| `DATABRICKS_CLIENT_ID` / `_SECRET` | Workspace → Settings → Identity & access → Service principals → OAuth secrets | +| `DATABRICKS_OAUTH_CLIENT_ID` / `_SECRET` | Account console → Settings → App connections → OAuth application integrations → custom integration | +| `AWS_REGION` | Pick a region with Bedrock + AgentCore availability | +| `CFN_DEPLOYER_ROLE_ARN` | (CFN path only) get from your FE Sandbox admin | + +## Step 1: Optionally provision a sample Genie space + +```bash +DATABRICKS_WAREHOUSE_ID= \ + python scripts/create_demo_genie_space.py +``` + +Creates a TPCH-based demo space with 5 sample NL→SQL questions, instructions, and a glossary via the Databricks Genie Spaces REST API. + +If your workspace doesn't yet expose the create surface (it's been rolling out through preview), the script falls back to printing the JSON payload for you to paste into the Genie UI's "Create new space" dialog. Either way, the resulting `GENIE_SPACE_ID` goes into `.env`. + +## Step 2: Provision AWS + +Pick **one** path: + +```bash +# external AWS account +cd terraform && terraform init && terraform apply ... + +# or FE Sandbox / restricted account +cd cloudformation && ./deploy.sh +``` + +Capture outputs into `.env`: `AGENTCORE_GATEWAY_ID`, `DATABRICKS_OAUTH_PROVIDER_ARN`, `SCHEMA_S3_BUCKET`, `BEDROCK_AGENT_ID`, `BEDROCK_AGENT_ALIAS`. + +## Step 3: Register Genie as a Gateway target + +```bash +pip install -r src/requirements.txt +python src/register_gateway.py +``` + +The script: + +1. Reads `docs/genie-mcp-schema.json`, substitutes your `DATABRICKS_HOST` and `GENIE_SPACE_ID`, uploads to `s3://$SCHEMA_S3_BUCKET/$SCHEMA_S3_KEY` +2. Calls `bedrock-agentcore-control:CreateGatewayTarget` with the OAuth credential provider ARN + +Expected output: + +``` +Uploaded schema -> s3://dbx-genie-mcp-...-schemas/schemas/genie-mcp-.json +Registered target: +{ + "targetId": "...", + ... +} +``` + +## Step 4: Wire the Bedrock agent to the Gateway + +```bash +python scripts/associate_gateway.py +``` + +Creates an action group on the Bedrock agent backed by the AgentCore Gateway via `bedrock-agent:CreateAgentActionGroup` with `actionGroupExecutor.agentCoreGatewayId`. Then calls `prepare_agent` to make the change live. + +If the SDK surface for `agentCoreGatewayId` isn't yet available in your account/region, the script prints the exact console steps to follow: + +1. Open the Bedrock console → Agents → select the provisioned agent +2. Action groups → Add → choose **AgentCore Gateway** as the source +3. Pick the gateway provisioned in Step 2 +4. Save → Prepare → verify alias `live` is on the prepared version + +## Step 5: (OBO only) Sync the OAuth redirect URI + +```bash +python scripts/sync_oauth_redirect.py +``` + +This closes the OAuth two-pass loop: + +1. Fetches the redirect URI from AgentCore Identity (`bedrock-agentcore-control:GetOauthCredentialProvider`) +2. Pushes it to the Databricks custom OAuth app via the Account API (`PATCH /api/2.0/accounts/{account_id}/oauth2/custom-app-integrations/{integration_id}`) +3. If the optional Account API env vars (`DATABRICKS_ACCOUNT_*`) aren't populated, just prints the URI for manual paste into the Account console + +Without this step, OBO mode will `401` at token exchange because the Databricks OAuth app rejects the unfamiliar redirect URI. + +## Step 6: Smoke test + +```bash +python src/bedrock_agent_test.py "What were our top 5 customers by revenue last quarter?" +``` + +Expected: + +- `[trace] tool call -> /tools/query_genie` +- A natural-language answer that cites the SQL Genie generated and the row count + +## Step 7: Validate governance (OBO only) + +Required before claiming user-level governance in a customer demo: + +1. Run the same question as **two distinct end users** with different UC grants +2. Confirm different rows or `403` for the user without grants +3. Inspect `system.access.audit` — confirm the SQL is attributed to the **human end user**, not the OAuth app's client ID + +## Helper script reference + +| Script | What it does | Fallback | +| --- | --- | --- | +| `scripts/create_demo_genie_space.py` | Creates a TPCH-based Genie space with Trusted Assets via the Genie Spaces API | Prints JSON for manual paste into Genie UI | +| `src/register_gateway.py` | Uploads OpenAPI schema to S3, registers Genie as a Gateway target | None — required step | +| `scripts/associate_gateway.py` | Wires the Bedrock agent to the Gateway as an action group | Prints exact Bedrock console steps | +| `scripts/sync_oauth_redirect.py` | Pushes AgentCore redirect URI to the Databricks OAuth app | Prints URI for manual paste into Account console | +| `src/bedrock_agent_test.py` | End-to-end smoke test invoking the agent | None — diagnostic | + +## Local-development bridge (optional) + +`src/genie_mcp_proxy.py` is a FastAPI proxy that wraps the Genie Conversation API directly. Use it when you want to test custom orchestration locally before involving AgentCore Gateway: + +```bash +AUTH_MODE=m2m python src/genie_mcp_proxy.py +# proxy is now at http://localhost:8080 +curl -X POST http://localhost:8080/tools/query_genie \ + -H "Content-Type: application/json" \ + -d '{"question":"top 5 customers by revenue last quarter"}' +``` + +In `obo` mode the proxy expects an end-user bearer token in the `Authorization` header (the default when AgentCore Gateway calls a self-hosted target). diff --git a/databricks-skills/databricks-genie-bedrock-agentcore/SKILL.md b/databricks-skills/databricks-genie-bedrock-agentcore/SKILL.md new file mode 100644 index 00000000..a5f1dfd3 --- /dev/null +++ b/databricks-skills/databricks-genie-bedrock-agentcore/SKILL.md @@ -0,0 +1,137 @@ +--- +name: databricks-genie-bedrock-agentcore +description: "Expose Databricks Genie spaces as a governed MCP tool to Amazon Bedrock agents through AgentCore Gateway. Use when integrating Databricks Genie with Bedrock agents on AWS, when a customer wants Unity Catalog governance on Bedrock-side analytics, or when avoiding data movement into Bedrock Knowledge Bases. Covers two auth modes (OBO and M2M), two IaC paths (Terraform and CloudFormation), and the AWS+Databricks plumbing required end-to-end." +--- + +# Databricks Genie via Amazon Bedrock AgentCore Gateway (MCP) + +Patterns for exposing **Databricks Genie spaces** as a governed **MCP tool** to **Amazon Bedrock agents** through **AgentCore Gateway** — no data movement into Knowledge Bases, no parallel metric definitions, Unity Catalog governance preserved end-to-end. + +## Overview + +Bedrock customers building NL-to-SQL agents typically face two unappealing options: + +1. **Copy data into a Bedrock Knowledge Base** — loses UC governance, forces metric definitions to be reimplemented outside the data platform +2. **Build a custom NL-to-SQL chain** — reinvents what Genie does and drifts from dashboard semantics + +This skill covers the third path: leave the metric definitions and the data in Databricks, expose Genie via the managed MCP endpoint, register it as an AgentCore Gateway target, propagate end-user identity via AgentCore Identity OAuth on-behalf-of-user. + +The Databricks-managed Genie MCP endpoint (`{DATABRICKS_HOST}/api/2.0/mcp/genie/{space_id}`) is registered as a target on an AgentCore Gateway. Any Bedrock agent associated with that gateway can call `query_genie` as a tool. + +## When to use this skill + +- A Bedrock customer wants UC governance for their agent's analytics tools +- A Databricks customer is evaluating Bedrock as the agent surface and needs the Databricks plumbing +- A Field Engineer is preparing a Genie + Bedrock demo (FSI portfolio risk, healthcare clinical ops, retail merch planning, FP&A variance, etc.) + +## Two auth modes + +| Mode | When | What end-user sees | +| --- | --- | --- | +| `obo` (recommended for production) | Customer demos, multi-user systems | Genie answers reflect each user's UC permissions; UC audit attributes SQL to the human | +| `m2m` (booth-demo quick start) | Single-tenant testing, booth demos | Every call uses a service principal token; all callers see the same rows the SP can see | + +**Always disclose auth mode in talk track.** Saying "row-level UC permissions per end user" is a misrepresentation in `m2m` mode. See [1-architecture-and-auth.md](1-architecture-and-auth.md) for full identity-flow detail. + +## Two IaC paths + +| Path | When | +| --- | --- | +| **Terraform** (uses `awscc` provider for AgentCore primitives) | External AWS account where caller has direct IAM/Bedrock-AgentCore privileges | +| **CloudFormation** (deployed via a pre-blessed exec role) | FE Sandbox / restricted AWS accounts where the caller's SSO role lacks `iam:CreateRole` and `bedrock-agentcore:*` | + +The CloudFormation path uses Ioannis Papadopoulos's pre-blessed-exec-role pattern (`aws cloudformation deploy --role-arn $CFN_DEPLOYER_ROLE_ARN`) — see [2-deployment-tf-vs-cfn.md](2-deployment-tf-vs-cfn.md). + +## Quick Start + +End-to-end deployment recipe (per [3-quickstart-and-scripts.md](3-quickstart-and-scripts.md) for full detail): + +```bash +# 1. Clone the reference architecture +git clone https://github.com/databricks-field-eng/databricks-aws-integrations.git +cd databricks-aws-integrations/genie_with_bedrock_agentcore/demo + +# 2. Configure +cp .env.example .env +# Fill in: DATABRICKS_HOST, GENIE_SPACE_ID, AWS_REGION +# For OBO: DATABRICKS_OAUTH_CLIENT_ID/SECRET +# For M2M: DATABRICKS_CLIENT_ID/SECRET + +# 3. (Optional) Stand up a sample Genie space programmatically +DATABRICKS_WAREHOUSE_ID= \ + python scripts/create_demo_genie_space.py + +# 4. Provision AWS — pick one +cd terraform && terraform init && terraform apply # external AWS +# or +cd cloudformation && ./deploy.sh # FE Sandbox + +# 5. Register Genie as a Gateway target (uploads schema to S3, calls +# bedrock-agentcore-control:CreateGatewayTarget) +python src/register_gateway.py + +# 6. Wire the Bedrock agent to the Gateway as an action group +python scripts/associate_gateway.py + +# 7. (OBO only) Sync the AgentCore redirect URI back to the Databricks OAuth app +python scripts/sync_oauth_redirect.py + +# 8. Smoke test +python src/bedrock_agent_test.py "Top 5 customers by revenue last quarter?" +``` + +## Common Patterns + +### Pattern 1: OBO production deployment + +End-user identity propagates from the Bedrock agent into Databricks via AgentCore Identity OAuth U2M. UC enforces row/column-level security per user; audit logs attribute SQL to the human. + +Required: +- Databricks workspace OAuth app (custom integration) with redirect URI matching AgentCore Identity's +- AgentCore Identity OAuth credential provider configured with `grant_type=authorization_code` and scope `all-apis offline_access` +- The OAuth-redirect two-pass setup (placeholder URI first, real URI after AgentCore provisions the credential provider) — `scripts/sync_oauth_redirect.py` automates this + +### Pattern 2: M2M booth demo + +Every Genie call uses a Databricks service principal. Acceptable for single-tenant testing, **never claim user-level governance** in this mode. + +Required: +- Databricks SP with workspace + UC read on the Genie space's underlying tables +- AgentCore Identity OAuth credential provider configured with `grant_type=client_credentials` and scope `all-apis` + +### Pattern 3: FE Sandbox deployment + +When the AWS account is FE Sandbox or otherwise restricted, the SSO caller cannot directly create IAM roles or Bedrock-AgentCore resources. Pattern: + +1. **One-time:** an account admin creates an IAM role (e.g. `cfn-bedrock-agentcore-deployer`) with the privileges CloudFormation needs (`iam:*Role*`, `bedrock-agentcore:*`, `bedrock:*Agent*`, `secretsmanager:*`, `s3:*`). The exact policy is in [2-deployment-tf-vs-cfn.md](2-deployment-tf-vs-cfn.md). +2. **Each deploy:** `aws cloudformation deploy --role-arn $CFN_DEPLOYER_ROLE_ARN ...` so CFN executes as the pre-blessed role. + +The caller only needs `cloudformation:*` and `sts:AssumeRole` on the deployer role — both usually allowed on FE Sandbox. + +### Pattern 4: Multi-space discovery (roadmap) + +For agents that need to choose between multiple Genie spaces, register a `list_genie_spaces` tool alongside `query_genie`. Not in v0.1; documented in the reference repo's roadmap. + +## Reference Files + +- [1-architecture-and-auth.md](1-architecture-and-auth.md) — auth modes, OAuth credential provider configuration, identity flow, governance validation +- [2-deployment-tf-vs-cfn.md](2-deployment-tf-vs-cfn.md) — both IaC paths, the FE Sandbox exec-role pattern, the IAM policy needed +- [3-quickstart-and-scripts.md](3-quickstart-and-scripts.md) — end-to-end deployment recipe, helper scripts, expected outputs + +## Common Issues + +| Issue | Solution | +|-------|----------| +| **`terraform init` errors on `awscc_bedrockagentcore_*` attribute** | AgentCore CFN registry schema renamed a property post-preview; check the current schema and update the Terraform/CFN. The IaC was authored against the 2026-05 registry. | +| **`aws cloudformation deploy` errors `User is not authorized to perform iam:CreateRole`** | You're on a restricted account — use the `--role-arn $CFN_DEPLOYER_ROLE_ARN` path, not direct deploy. See [2-deployment-tf-vs-cfn.md](2-deployment-tf-vs-cfn.md). | +| **`401` from Genie endpoint** | M2M: SP lacks workspace/UC privileges or token URL is wrong. OBO: the bearer token isn't being forwarded — confirm AgentCore Identity is exchanging it correctly. | +| **`403` on UC tables (OBO)** | End-user identity isn't propagating. Most often the Databricks OAuth app's redirect URI doesn't match AgentCore Identity's — re-run `scripts/sync_oauth_redirect.py`. | +| **`query_genie` not visible to agent** | Action group isn't attached. Re-run `scripts/associate_gateway.py`. If the SDK surface isn't yet available in your account/region, the script falls back to printing exact console steps. | +| **README claims OBO governance but agent answers don't reflect user permissions** | Check `AUTH_MODE` in `.env` — the proxy and the gateway target both honor it, and the README's claim only holds when both are set to `obo`. | +| **Genie space not created via `scripts/create_demo_genie_space.py`** | The Genie Spaces API is recent and may not be enabled on every workspace. The script falls back to printing the JSON payload for manual creation in the Genie UI. | + +## See also + +- [databricks-genie](../databricks-genie/SKILL.md) — Genie Space management (create, query, export, import). Use that skill for the Databricks-side Genie work; this skill covers the AWS-side integration. +- [databricks-agent-bricks](../databricks-agent-bricks/SKILL.md) — Knowledge Assistants, Genie Spaces, and Supervisor Agents on Databricks. Use for Databricks-native multi-agent orchestration. +- Reference architecture: [databricks-field-eng/databricks-aws-integrations/genie_with_bedrock_agentcore](https://github.com/databricks-field-eng/databricks-aws-integrations/tree/main/genie_with_bedrock_agentcore) — full deployable demo with Terraform, CloudFormation, and helper scripts.