Skip to content

Bootstrap Guide

Jump to bottom
Joey French edited this page Jan 13, 2026 · 12 revisions

Bootstrap Guide

This guide walks you through bootstrapping a fresh fork of RoboSystems.

Table of Contents

How It Works

RoboSystems uses GitHub OIDC federation for AWS authentication. No AWS credentials are stored in GitHub.

┌─────────────────┐      ┌─────────────────┐      ┌─────────────────┐
│  GitHub Action  │─────▶│  OIDC Token     │─────▶│     AWS STS     │
│  Workflow       │      │  (I am repo X)  │      │  (temp creds)   │
└─────────────────┘      └─────────────────┘      └─────────────────┘
                                                          │
                                                          ▼
                                                  ┌─────────────────┐
                                                  │  Deploy to AWS  │
                                                  │  (1hr session)  │
                                                  └─────────────────┘

Security Benefits:

  • No long-term credentials stored anywhere
  • Credentials scoped to specific repo/branch
  • 1-hour max session (can't be abused if compromised)

Prerequisites

  • AWS IAM Identity Center (SSO) enabled with admin permissions
  • aws CLI v2 (brew install awscli)
  • gh CLI authenticated (brew install gh && gh auth login)
  • jq (brew install jq)
  • direnv (optional - brew install direnv)

GitHub token scopes: repo, admin:org, workflow

Verify you have access to the correct repo:

gh repo view
# Should show your fork, e.g.: HarbingerFinLab/robosystems

Fresh AWS Account Setup

Skip this section if your AWS account already has IAM Identity Center configured.

  1. Log in as root user and enable MFA
  2. Enable IAM Identity Center (SSO)
  3. Create Permission Set:
    • IAM Identity Center → Permission sets → Create
    • Select "Predefined permission set" → "AdministratorAccess"
  4. Create SSO admin user:
    • IAM Identity Center → Users → Add user
    • Complete details and set up MFA
  5. Assign permissions:
    • Users → [your user] → Assign AWS accounts
    • Select account(s) → AdministratorAccess permission set

Bootstrap

# 1. Configure AWS CLI to use SSO
aws configure sso --profile robosystems-sso

# 2. Run the bootstrap
just bootstrap                              # Default profile/region
just bootstrap robosystems-sso              # Custom profile
just bootstrap robosystems-sso eu-west-1    # Custom profile AND region

Bootstrap handles everything:

  1. Deploys OIDC federation CloudFormation stack
  2. Sets GitHub variables (AWS_ROLE_ARN, AWS_ACCOUNT_ID, AWS_REGION)
  3. Creates ECR repository for Docker images
  4. Creates .envrc for automatic profile/region selection
  5. Runs setup-aws to create application secrets (see below)

What gets created in AWS:

  • OIDC Identity Provider for GitHub Actions
  • IAM Role: RoboSystemsGitHubActionsRole (backend)
  • IAM Role: RoboSystemsGitHubActionsFrontendRole (frontend apps)
  • ECR Repository with lifecycle policy

Application Secrets (just setup-aws)

Creates robosystems/prod and robosystems/staging in AWS Secrets Manager with:

  • Auto-generated keys: JWT signing, connection encryption, backup encryption
  • Feature flags: billing, rate limiting, registration, SSE, etc.
  • Integration placeholders: QuickBooks, Plaid, SEC, Turnstile (configure later)

Safe to re-run - existing secrets are never overwritten. To customize later:

# View current values
aws secretsmanager get-secret-value --secret-id robosystems/prod --query SecretString --output text | jq .

# Update values
aws secretsmanager put-secret-value --secret-id robosystems/prod --secret-string "$(cat updated.json)"

Fork-specific: GitHub Actions workflows automatically pass your AWS account ID as a namespace to CloudFormation, creating unique bucket names like robosystems-{account-id}-shared-raw-{env}.

Deploy

just deploy prod    # ~20-30 min for initial setup

Initial deployment creates all infrastructure (VPC, databases, ECS services). Subsequent deploys only update changed resources.

Verify deployment:

just bastion-tunnel prod all    # Connect to all services
# API at http://localhost:8000
# Dagster at http://localhost:3003

Multi-Repository Setup

The OIDC stack creates two roles:

Role Repositories Permissions
RoboSystemsGitHubActionsRole robosystems Full infrastructure
RoboSystemsGitHubActionsFrontendRole robosystems-app, roboledger-app, roboinvestor-app Limited frontend

After bootstrapping the backend, set up frontend apps:

# In robosystems-app, roboledger-app, or roboinvestor-app
npm run setup:bootstrap

Allowed deployment branches: main, release/*, v* tags

API Access Modes

RoboSystems supports three API access modes, configured via API_ACCESS_MODE_PROD / API_ACCESS_MODE_STAGING:

Mode ALB Scheme TLS Domain Required Use Case
internal Internal No No Development, testing, private deployments
public-http Internet-facing No No Quick public access without domain setup
public Internet-facing Yes Yes Production with custom domain

Internal Mode (Default)

No custom domain required. Access via bastion tunnel:

just bastion-tunnel prod all
# API at http://localhost:8000

This is the default - no GitHub variables needed. The ALB is internal (not internet-accessible) and you connect through SSM port forwarding.

Public HTTP Mode

Internet-facing ALB without TLS. Useful for testing or when you don't have a domain:

gh variable set API_ACCESS_MODE_PROD --body "public-http"
just deploy prod

After deployment, access via the ALB DNS name (shown in CloudFormation outputs).

Limitations:

  • No TLS encryption (credentials sent in plain text)
  • OAuth integrations (Google, QuickBooks) won't work (require HTTPS redirect URLs)
  • Not recommended for production with sensitive data

Why no HTTPS without a domain? AWS ACM certificates require domain validation - you must prove you own the domain. The ALB's auto-generated DNS (*.elb.amazonaws.com) is AWS-owned, so you can't get a certificate for it. If you need HTTPS, register a domain (~$10-12/year for .dev or .click) and use public mode.

Public Mode (Custom Domain)

Full production setup with HTTPS and custom domain:

  1. Domain must be hosted in Route53
  2. Configure variables: 
    gh variable set API_ACCESS_MODE_PROD --body "public"
gh variable set API_DOMAIN_NAME_ROOT --body "yourdomain.com"
gh variable set API_DOMAIN_NAME_PROD --body "api.yourdomain.com"
  3. Deploy - ACM certificates and DNS records created automatically

Switching Modes

You can switch modes at any time:

# Switch from internal to public-http
gh variable set API_ACCESS_MODE_PROD --body "public-http"
just deploy prod

# Switch to full public with domain
gh variable set API_ACCESS_MODE_PROD --body "public"
gh variable set API_DOMAIN_NAME_ROOT --body "yourdomain.com"
gh variable set API_DOMAIN_NAME_PROD --body "api.yourdomain.com"
just deploy prod

Note: Switching between internal and internet-facing modes will replace the ALB (causes brief downtime).

Frontend App Access Modes

Frontend apps (robosystems-app, roboledger-app, roboinvestor-app) support the same three access modes, configured via APP_ACCESS_MODE_PROD / APP_ACCESS_MODE_STAGING:

Mode CloudFront ALB Scheme TLS Domain Required Use Case
internal No Internal No No Development via bastion tunnel
public-http Yes (AWS DNS) Internet-facing Yes* No Testing without custom domain
public Yes Internet-facing Yes Yes Production with custom domain

*CloudFront provides HTTPS on its default domain (d123.cloudfront.net)

Internal Mode

No CloudFront or custom domain. Access via bastion tunnel:

just bastion-tunnel prod app
# App at http://localhost:3000

The ALB is internal (not internet-accessible) and you connect through SSM port forwarding. Static assets are served directly from ECS (no S3/CloudFront).

Public HTTP Mode

CloudFront with AWS-provided domain. Useful for testing or when you don't have a domain:

gh variable set APP_ACCESS_MODE_PROD --body "public-http"
just deploy prod

After deployment, access via the CloudFront domain (shown in CloudFormation outputs, e.g., d123abc.cloudfront.net).

Benefits over internal mode:

  • Public access without custom domain
  • HTTPS via CloudFront's default certificate
  • CDN caching for static assets

Limitations:

  • No custom domain - uses CloudFront's auto-generated domain
  • Some OAuth providers may require a custom domain for callbacks

Public Mode (Custom Domain)

Full production setup with custom domain and HTTPS:

  1. Domain must be hosted in Route53
  2. Configure variables: 
    gh variable set APP_ACCESS_MODE_PROD --body "public"
gh variable set DOMAIN_NAME_ROOT --body "yourdomain.com"
gh variable set DOMAIN_NAME_PROD --body "app.yourdomain.com"
  3. Deploy - ACM certificates and DNS records created automatically

Switching Modes

You can switch modes at any time:

# Switch from internal to public-http
gh variable set APP_ACCESS_MODE_PROD --body "public-http"
just deploy prod

# Switch to full public with domain
gh variable set APP_ACCESS_MODE_PROD --body "public"
gh variable set DOMAIN_NAME_ROOT --body "yourdomain.com"
gh variable set DOMAIN_NAME_PROD --body "app.yourdomain.com"
just deploy prod

Note: Switching between internal and internet-facing modes will replace the ALB (causes brief downtime). Switching to/from internal mode also adds/removes CloudFront (may take 10-15 minutes to propagate).

Troubleshooting

SSO: "No accounts found"

Ensure your SSO user has permission sets assigned to accounts.

SSO: "Profile not found"

aws configure sso --profile robosystems-sso

AWS CLI: "You must specify a region"

echo 'export AWS_REGION=us-east-1' >> .envrc && direnv allow

OIDC: "Not authorized to perform sts:AssumeRoleWithWebIdentity"

  1. Verify OIDC stack deployed successfully
  2. Check branch matches allowed conditions (main, release/, v tags)
  3. Ensure workflow has permissions: id-token: write

Quick Reference

# Bootstrap
just bootstrap                    # Default profile/region
just bootstrap my-sso eu-west-1   # Custom profile AND region

# Deploy
just deploy prod                  # Production (~20-30 min initial)
just deploy staging               # Staging

# Connect
just bastion-tunnel prod all      # All tunnels (postgres, valkey, dagster, api)

# Optional / Re-run individually
just setup-aws                    # Application secrets & feature flags
just setup-gha                    # Full GitHub variable control (~120 vars)
just setup-bedrock                # Local AI/Bedrock development

# Verify
gh variable list
aws sts get-caller-identity

CloudFormation Templates

Template Deployed By Purpose
bootstrap-oidc.yaml just bootstrap (local) GitHub OIDC federation
vpc.yaml GitHub Actions VPC and networking
postgres.yaml GitHub Actions RDS PostgreSQL
valkey.yaml GitHub Actions ElastiCache Redis
s3.yaml GitHub Actions S3 buckets
api.yaml GitHub Actions ECS API service
dagster.yaml GitHub Actions ECS Dagster service
graph-*.yaml GitHub Actions LadybugDB infrastructure
bastion.yaml GitHub Actions SSM bastion host

Related Documentation

© 2025 RFS LLC

Clone this wiki locally