Add docs for local development

Author: ZIJ

docs/ce/local-development/backend.mdx

@@ -0,0 +1,61 @@
+---
+title: Backend (orchestrator) local setup
+---
+
+The backend serves orchestration APIs, GitHub app endpoints, and internal APIs the UI relies on.
+
+## Quick start
+
+1. Create `backend/.env` with the essentials (adjust DB URL/ports):
+   ```bash
+   DATABASE_URL=postgres://postgres:root@localhost:5432/postgres?sslmode=disable
+   DIGGER_INTERNAL_SECRET=orchestrator-secret          # must match UI ORCHESTRATOR_BACKEND_SECRET
+   DIGGER_ENABLE_INTERNAL_ENDPOINTS=true               # enables /_internal/*
+   DIGGER_ENABLE_API_ENDPOINTS=true                    # enables /api/*
+   HOSTNAME=http://localhost:3000                      # used for GitHub app callbacks
+   ```
+   Optional but useful:
+   - `GITHUB_APP_ID`, `GITHUB_APP_PEM`, `GITHUB_APP_WEBHOOK_SECRET` if you already have an app.
+   - `GITHUB_ORG` if you want `/github/setup` to target an org.
+2. Start the service (from `backend/`):
+   ```bash
+   set -a; source .env; set +a
+   go run main.go              # or: make start
+   ```
+   Default port: `3000`.
+
+## Make the UI happy
+
+- The UI calls `/api/*` and `/github/*` with `Authorization: Bearer $DIGGER_INTERNAL_SECRET` and `DIGGER_ORG_ID`/`DIGGER_USER_ID` headers.
+- You must upsert the WorkOS org + user the UI is authenticated as:
+  ```bash
+  SECRET=$DIGGER_INTERNAL_SECRET
+  ORG_ID=org_xxx                           # WorkOS org id
+  ORG_NAME=my-org                          # slug shown in backend
+  ADMIN_EMAIL=you@example.com
+  USER_ID=user_xxx                         # WorkOS user id
+  USER_EMAIL=$ADMIN_EMAIL
+
+  # org
+  curl -s -X POST http://localhost:3000/_internal/api/upsert_org \
+    -H "Authorization: Bearer $SECRET" \
+    -H "Content-Type: application/json" \
+    -d '{"org_name":"'"$ORG_NAME"'","external_source":"workos","external_id":"'"$ORG_ID"'","admin_email":"'"$ADMIN_EMAIL"'"}'
+
+  # user
+  curl -s -X POST http://localhost:3000/_internal/api/create_user \
+    -H "Authorization: Bearer $SECRET" \
+    -H "Content-Type: application/json" \
+    -d '{"external_source":"workos","external_id":"'"$USER_ID"'","email":"'"$USER_EMAIL"'","external_org_id":"'"$ORG_ID"'"}'
+  ```
+
+## GitHub app integration
+
+- For a quick install link, set `ORCHESTRATOR_GITHUB_APP_URL` in `ui/.env.local` to your app’s install URL (`https://github.com/apps/<app>/installations/new`).
+- To create a new app via the backend, open `http://localhost:3000/github/setup` (requires `HOSTNAME` set to a reachable URL for callbacks).
+
+## Troubleshooting
+
+- **404 on /api/repos**: ensure `DIGGER_ENABLE_API_ENDPOINTS=true` and the org/user above are created.
+- **401/403**: verify `Authorization` header uses `DIGGER_INTERNAL_SECRET`.
+- **GitHub connect 404**: set `ORCHESTRATOR_GITHUB_APP_URL` as described.

docs/ce/local-development/overview.mdx

@@ -0,0 +1,39 @@
+---
+title: Local development overview
+---
+
+This section explains how to run the three core services locally:
+
+- **Backend** (`backend/`, port 3000 by default) – orchestrator + REST APIs for repos/orgs/jobs.
+- **Statesman** (`taco/cmd/statesman`, port 8080) – state storage API and Terraform Cloud-compatible endpoints.
+- **UI** (`ui/`, port 3030) – TanStack Start frontend that talks to both services and WorkOS.
+
+## Prerequisites
+
+- Go toolchain for backend + statesman, Node 18+ for UI (`pnpm` or `npm`).
+- A WorkOS project with User Management enabled and at least one organization + member (needed for UI auth and org IDs).
+- Optional: GitHub App for repo onboarding (the backend can help you create one via `/github/setup`).
+
+## Shared secrets and ports
+
+- Pick two secrets and reuse them across components:
+  - `ORCHESTRATOR_BACKEND_SECRET` ≡ `DIGGER_INTERNAL_SECRET` (backend) ≡ UI env.
+  - `STATESMAN_BACKEND_WEBHOOK_SECRET` ≡ `OPENTACO_ENABLE_INTERNAL_ENDPOINTS` (statesman) ≡ UI env.
+- Default ports: backend `3000`, statesman `8080`, UI `3030`.
+
+## High-level workflow
+
+1) **Start backend** with internal + API endpoints enabled (so UI can call `/api/*` and `/github/*`).  
+2) **Start statesman** with internal endpoints enabled; use memory storage for quick start.  
+3) **Configure UI** `.env.local` with URLs + secrets + WorkOS creds; run `pnpm dev --host --port 3030`.  
+4) **Sync org/user** into backend and statesman (WorkOS org id and user id/email) via the provided curl snippets in each page.  
+5) (Optional) **GitHub App**: set `ORCHESTRATOR_GITHUB_APP_URL` to your install URL or `http://localhost:3000/github/setup` to create one via the backend.
+
+## Troubleshooting cheatsheet
+
+- **Backend /api/* returns 404**: `DIGGER_ENABLE_API_ENDPOINTS` not `true` or org not upserted.  
+- **Statesman 403**: webhook secret mismatch. **Statesman 404/500 resolving org**: org not synced (missing `external_org_id`).  
+- **UI WorkOS auth succeeds but org is empty**: add membership in WorkOS and resync org/user to services.  
+- **GitHub connect opens 404**: set `ORCHESTRATOR_GITHUB_APP_URL` to a valid install/setup URL.
+
+Continue with the per-service pages for commands and env examples.

docs/ce/local-development/statesman.mdx

@@ -0,0 +1,54 @@
+---
+title: Statesman local setup
+---
+
+Statesman serves state storage + Terraform Cloud-compatible APIs. The UI uses its internal endpoints, so enable webhook auth and sync your org/user.
+
+## Quick start
+
+1. Set env vars:
+   ```bash
+   OPENTACO_ENABLE_INTERNAL_ENDPOINTS=statesman-secret   # must match UI STATESMAN_BACKEND_WEBHOOK_SECRET
+   OPENTACO_AUTH_DISABLE=true                            # skips OIDC for local
+   OPENTACO_STORAGE=memory                               # default; uses SQLite query backend automatically
+   # Optional: OPENTACO_SECRET_KEY for signed URLs; OPENTACO_PORT=8080
+   ```
+2. Run the service (from repo root):
+   ```bash
+   cd taco
+   go run cmd/statesman/main.go -storage memory -auth-disable   # or ./statesman with the same flags
+   ```
+   Default port: `8080`.
+
+## Sync org and user (required for UI)
+
+Statesman resolves orgs by `external_org_id` (your WorkOS org id). If it cannot resolve, `/internal/api/units` will 500.
+
+```bash
+SECRET=$OPENTACO_ENABLE_INTERNAL_ENDPOINTS
+ORG_ID=org_xxx                        # WorkOS org id
+ORG_NAME=digger-org                   # slug to store
+ORG_DISPLAY="Digger Org"
+USER_ID=user_xxx                      # WorkOS user id
+USER_EMAIL=you@example.com
+
+# create/sync org
+curl -s -X POST http://localhost:8080/internal/api/orgs/sync \
+  -H "Authorization: Bearer $SECRET" \
+  -H "X-User-ID: $USER_ID" -H "X-Email: $USER_EMAIL" \
+  -H "Content-Type: application/json" \
+  -d '{"name":"'"$ORG_NAME"'","display_name":"'"$ORG_DISPLAY"'","external_org_id":"'"$ORG_ID"'"}'
+
+# ensure user exists in that org
+curl -s -X POST http://localhost:8080/internal/api/users \
+  -H "Authorization: Bearer $SECRET" \
+  -H "X-Org-ID: '$ORG_ID'" -H "X-User-ID: $USER_ID" -H "X-Email: $USER_EMAIL" \
+  -H "Content-Type: application/json" \
+  -d '{"subject":"'"$USER_ID"'","email":"'"$USER_EMAIL"'"}'
+```
+
+## Troubleshooting
+
+- **403**: webhook secret mismatch (`OPENTACO_ENABLE_INTERNAL_ENDPOINTS` vs UI `STATESMAN_BACKEND_WEBHOOK_SECRET`).
+- **404/500 resolving org**: org not synced; rerun the `orgs/sync` call above.
+- **SQLite quirks**: defaults to SQLite in-process; no config needed for local. For Postgres/MySQL, set `TACO_QUERY_BACKEND` and related envs (see `docs/ce/state-management/query-backend`).

docs/ce/local-development/ui.mdx

@@ -0,0 +1,51 @@
+---
+title: UI local setup
+---
+
+The UI is a TanStack Start app that authenticates via WorkOS and calls both backend and statesman.
+
+## Quick start
+
+1. Copy `.env.example` to `.env.local` in `ui/` and fill the essentials:
+   ```bash
+   # URLs
+   PUBLIC_URL=http://localhost:3030
+   ALLOWED_HOSTS=localhost,127.0.0.1
+
+   # WorkOS (User Management)
+   WORKOS_CLIENT_ID=<your_workos_client_id>
+   WORKOS_API_KEY=<your_workos_api_key>
+   WORKOS_COOKIE_PASSWORD=<32+ random chars>
+   WORKOS_REDIRECT_URI=http://localhost:3030/api/auth/callback
+   WORKOS_WEBHOOK_SECRET=<if using webhooks locally via tunnel>
+
+   # Backend
+   ORCHESTRATOR_BACKEND_URL=http://localhost:3000
+   ORCHESTRATOR_BACKEND_SECRET=orchestrator-secret   # matches backend DIGGER_INTERNAL_SECRET
+   ORCHESTRATOR_GITHUB_APP_URL=http://localhost:3000/github/setup  # or your app install URL
+
+   # Statesman
+   STATESMAN_BACKEND_URL=http://localhost:8080
+   STATESMAN_BACKEND_WEBHOOK_SECRET=statesman-secret  # matches OPENTACO_ENABLE_INTERNAL_ENDPOINTS
+
+   # Optional
+   DRIFT_REPORTING_BACKEND_URL=http://localhost:3004
+   DRIFT_REPORTING_BACKEND_WEBHOOK_SECRET=...
+   POSTHOG_KEY=...
+   ```
+   In WorkOS, add `http://localhost:3030/api/auth/callback` as a redirect.
+2. Install deps and run:
+   ```bash
+   cd ui
+   pnpm install   # or npm install
+   pnpm dev --host --port 3030
+   ```
+   Open `http://localhost:3030` and sign in with a WorkOS user that belongs to at least one org.
+3. Ensure backend + statesman were started and the same secrets are in place (see [Backend](./backend) and [Statesman](./statesman)).
+4. Sync the WorkOS org/user to both services using the curl snippets on those pages (required for repos/units to load).
+
+## Common errors
+
+- **NotFound/Forbidden listing units**: statesman org/user not synced or webhook secret mismatch.
+- **404 on repos or GitHub connect**: backend missing org/user, `DIGGER_ENABLE_API_ENDPOINTS` not set, or `ORCHESTRATOR_GITHUB_APP_URL` points to a non-existent path.
+- **WorkOS login succeeds but dashboard redirects to / or errors**: the signed-in user has no WorkOS org membership; add to an org and resync to services.

docs/docs.json

@@ -165,6 +165,15 @@
               "ce/azure-specific/azure-devops-locking-connection-methods"
             ]
           },
+          {
+            "group": "Local Development",
+            "pages": [
+              "ce/local-development/overview",
+              "ce/local-development/backend",
+              "ce/local-development/statesman",
+              "ce/local-development/ui"
+            ]
+          },
           {
             "group": "Contributing",
             "pages": [
@@ -213,4 +222,4 @@
       "linkedin": "https://www.linkedin.com/company/diggerhq/"
     }
   }
-}
+}