Harden config auth and fix UI test/build flow

.github/workflows/ci.yml

@@ -27,6 +27,17 @@ jobs:
       - name: Run Tests
         run: make GOBIN=$HOME/gopath/bin test
 
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.13'
+
+      - name: Create Python Virtual Environment
+        run: python -m venv .pyvenv
+
+      - name: Run API Unit Tests
+        run: make api-test
+
       - name: Run Coverage Tests
         env:
           COVERALLS_TOKEN: ${{ secrets.COVERALLS_TOKEN }}

.gitignore

@@ -21,6 +21,7 @@ roveralls*
 .specstory
 .pyvenv
 gogen-api/__pycache__
+__pycache__/
 gogen-api/build
 gogen-api/env.json
 ui/node_modules/*
@@ -29,4 +30,4 @@ ui/build/*
 ui/coverage/*
 ui/public/gogen.wasm
 ui/.vite
-*.idea
+*.idea

AGENTS.md

@@ -0,0 +1,116 @@
+# AGENTS.md
+
+This file gives coding agents the repo-specific context needed to work effectively in `gogen`.
+
+## Project Overview
+
+Gogen is a data generator for demo and test data, especially time-series logs and metrics. The repo contains:
+
+- a Go CLI core
+- a Python AWS Lambda API backend in `gogen-api/`
+- a React/TypeScript UI in `ui/`
+
+## Common Commands
+
+### Go
+
+```bash
+make install          # Preferred install path; injects ldflags from Makefile
+make build            # Cross-compiles linux, darwin, windows, wasm
+make test             # go test -v ./...
+go test -v ./internal
+go test -v -run TestName ./internal
+```
+
+Notes:
+
+- Use `make install` instead of bare `go install`; version/build metadata and OAuth settings are injected through `-ldflags`.
+- Dependencies are vendored. After dependency changes, run `go mod vendor`.
+
+### Python API
+
+```bash
+cd gogen-api
+./start_dev.sh
+./setup_local_db.sh
+./deploy_lambdas.sh
+```
+
+Repo-standard Python environment:
+
+```bash
+source /home/clint/local/src/gogen/.pyvenv/bin/activate
+```
+
+Focused API unit tests:
+
+```bash
+make api-test
+```
+
+### UI
+
+```bash
+cd ui
+npm run dev
+npm run build
+npm test
+```
+
+## Architecture
+
+### Go Package Layout
+
+- `main.go`: CLI entry point using `urfave/cli.v1`; maps flags to `GOGEN_*` env vars
+- `internal/`: core config, sample, token, API/share logic
+- `generator/`: generation workers
+- `outputter/`: output workers and destinations
+- `run/`: pipeline orchestration
+- `timer/`: one timer goroutine per sample
+- `rater/`: event-rate control
+- `template/`: output formatting
+- `logger/`: log wrapper
+
+### Data Flow
+
+```text
+YAML/JSON config -> internal.Config singleton
+  -> timer goroutines
+  -> generator worker pool
+  -> outputter worker pool
+  -> output destination
+```
+
+### Config System
+
+- Config is a singleton guarded by `sync.Once`
+- Remote configs default to `https://api.gogen.io` and can be overridden by `GOGEN_APIURL`
+- In Go tests, reset config state with `config.ResetConfig()` before `config.NewConfig()`
+- Tests often use `config.SetupFromString(...)` for inline YAML
+
+### Python API
+
+- Lambda handlers live as separate files in `gogen-api/api/`
+- Backed by DynamoDB + S3
+- Local development uses Docker Compose plus SAM
+- Use `.pyvenv` rather than system Python when running repo Python commands
+
+### UI
+
+- Vite + React 18 + TypeScript + Tailwind
+- Components live in `ui/src/components/`
+- Pages live in `ui/src/pages/`
+- Tests are colocated as `.test.tsx`
+
+## CI/CD
+
+- `.github/workflows/ci.yml` runs Go tests on pushes to `master`/`dev` and on PRs
+- CI also runs `make api-test`
+- Branch builds/deploys happen on `master` and `dev`
+- Release workflow is handled separately in `.github/workflows/release.yml`
+
+## Practical Notes
+
+- Prefer minimal, targeted edits; this repo spans Go, Python, and frontend code in one tree
+- For Python work, prefer adding tests that avoid external AWS dependencies unless the task explicitly needs integration coverage
+- For UI tests, keep them aligned with the current design system rather than hardcoding old color classes

Makefile

@@ -7,7 +7,7 @@ SUMMARY = $(shell git describe --tags --always --dirty)
 DATE = $(shell date --rfc-3339=date)
 
 
-.PHONY: all build deps install test docker splunkapp embed
+.PHONY: all build deps install test api-test docker splunkapp embed
 
 ifeq ($(OS),Windows_NT)
 	dockercmd := docker run -e TERM -e HOME=/go/src/github.com/coccyx/gogen --rm -it -v $(CURDIR):/go/src/github.com/coccyx/gogen -v $(HOME)/.ssh:/root/.ssh clintsharp/gogen bash
@@ -33,6 +33,8 @@ install:
 test:
 	go test -v ./...
 
+api-test:
+	./.pyvenv/bin/python -m unittest gogen-api/test_auth_utils.py gogen-api/test_upsert_auth.py
+
 docker:
 	$(dockercmd)
-

gogen-api/README.md

@@ -149,6 +149,24 @@ This script will:
 4. List objects in the bucket
 5. Download and verify the test file
 
+### Testing API Unit Logic
+
+Focused unit tests are available for request authentication helpers and config ownership enforcement.
+
+From the repo root:
+
+```bash
+make api-test
+```
+
+Or directly with the project virtual environment:
+
+```bash
+/home/clint/local/src/gogen/.pyvenv/bin/python -m unittest \
+  gogen-api/test_auth_utils.py \
+  gogen-api/test_upsert_auth.py
+```
+
 ## Accessing Services
 
 ### API Endpoints
@@ -307,4 +325,4 @@ docker-compose logs createbuckets
 - Document code with clear comments
 - Update SUMMARY.md after completing significant features
 - Each AWS Lambda function should be a separate .py file in the `api` directory
-- Remember that the codebase is being updated from Python 2.7 to Python 3.13 
+- Remember that the codebase is being updated from Python 2.7 to Python 3.13 

gogen-api/api/auth_utils.py

@@ -0,0 +1,42 @@
+from cors_utils import cors_response
+from github_utils import get_github_user
+from logger import setup_logger
+
+logger = setup_logger(__name__)
+
+
+def get_header(event, name):
+    """Return an HTTP header value using case-insensitive lookup."""
+    headers = event.get('headers') or {}
+    target = name.lower()
+
+    for key, value in headers.items():
+        if key.lower() == target:
+            return value
+
+    return None
+
+
+def get_authenticated_username(event):
+    """
+    Authenticate the GitHub token from the request and return the username.
+
+    Returns:
+        tuple[str | None, dict | None]: (username, error_response)
+    """
+    auth_header = get_header(event, 'Authorization')
+    if not auth_header:
+        logger.error("Authorization header not present")
+        return None, cors_response(401, {'error': 'Authorization header not present'})
+
+    user_info, error = get_github_user(auth_header)
+    if error:
+        logger.error(f"Failed to authenticate user: {error}")
+        return None, cors_response(401, {'error': error})
+
+    username = user_info.get('login')
+    if not username:
+        logger.error("Could not get username from GitHub")
+        return None, cors_response(401, {'error': 'Could not get username from GitHub'})
+
+    return username, None

gogen-api/api/delete.py

@@ -2,7 +2,7 @@
 from db_utils import get_dynamodb_client, get_table_name
 from s3_utils import delete_config
 from cors_utils import cors_response
-from github_utils import get_github_user
+from auth_utils import get_authenticated_username
 from logger import setup_logger
 
 logger = setup_logger(__name__)
@@ -28,22 +28,9 @@ def lambda_handler(event, context):
     try:
         logger.debug(f"Received event: {json.dumps(event, indent=2)}")
 
-        # Validate GitHub authorization
-        if 'headers' not in event or 'Authorization' not in event['headers']:
-            logger.error("Authorization header not present")
-            return respond("Authorization header not present", status_code=401)
-
-        # Get user information from GitHub token
-        auth_header = event['headers']['Authorization']
-        user_info, error = get_github_user(auth_header)
-        if error:
-            logger.error(f"Failed to authenticate user: {error}")
-            return respond(error, status_code=401)
-
-        username = user_info.get('login')
-        if not username:
-            logger.error("Could not get username from GitHub")
-            return respond("Could not get username from GitHub", status_code=401)
+        username, auth_error = get_authenticated_username(event)
+        if auth_error:
+            return auth_error
 
         # Extract config name from path
         path_params = event.get('pathParameters', {})

gogen-api/api/my_configs.py

@@ -2,7 +2,7 @@
 from boto3.dynamodb.conditions import Attr
 from db_utils import get_dynamodb_client, get_table_name
 from cors_utils import cors_response
-from github_utils import get_github_user
+from auth_utils import get_authenticated_username
 from logger import setup_logger
 
 logger = setup_logger(__name__)
@@ -28,22 +28,9 @@ def lambda_handler(event, context):
     try:
         logger.debug(f"Received event: {json.dumps(event, indent=2)}")
 
-        # Validate GitHub authorization
-        if 'headers' not in event or 'Authorization' not in event['headers']:
-            logger.error("Authorization header not present")
-            return respond("Authorization header not present", status_code=401)
-
-        # Get user information from GitHub token
-        auth_header = event['headers']['Authorization']
-        user_info, error = get_github_user(auth_header)
-        if error:
-            logger.error(f"Failed to authenticate user: {error}")
-            return respond(error, status_code=401)
-
-        username = user_info.get('login')
-        if not username:
-            logger.error("Could not get username from GitHub")
-            return respond("Could not get username from GitHub", status_code=401)
+        username, auth_error = get_authenticated_username(event)
+        if auth_error:
+            return auth_error
 
         logger.info(f"Fetching configurations for user: {username}")
 

gogen-api/api/upsert.py

@@ -2,7 +2,7 @@
 from db_utils import get_dynamodb_client, get_table_name
 from s3_utils import upload_config
 from cors_utils import cors_response
-from github_utils import validate_github_token
+from auth_utils import get_authenticated_username
 from logger import setup_logger
 
 logger = setup_logger(__name__)
@@ -34,15 +34,9 @@ def lambda_handler(event, context):
             logger.error(f"Invalid JSON in request body: {str(e)}")
             return respond("Invalid JSON in request body")
 
-        # Validate GitHub authorization
-        if 'headers' not in event or 'Authorization' not in event['headers']:
-            logger.error("Authorization header not present")
-            return respond("Authorization header not present")
-
-        # Validate GitHub token
-        is_valid, error_msg = validate_github_token(event['headers']['Authorization'])
-        if not is_valid:
-            return respond(error_msg)
+        username, auth_error = get_authenticated_username(event)
+        if auth_error:
+            return auth_error
 
         # Validate and clean request body
         validated_body = {}
@@ -58,9 +52,9 @@ def lambda_handler(event, context):
         if 'config' in validated_body:
             config_content = validated_body['config']
 
-            # Create S3 path in the format username/sample.yml
-            if 'owner' in validated_body and 'name' in validated_body:
-                s3_path = f"{validated_body['owner']}/{validated_body['name']}.yml"
+            if 'name' in validated_body:
+                validated_body['owner'] = username
+                s3_path = f"{username}/{validated_body['name']}.yml"
 
                 # Upload config to S3
                 logger.info(f"Uploading config to S3 at path: {s3_path}")
@@ -78,13 +72,13 @@ def lambda_handler(event, context):
                 validated_body['s3Path'] = s3_path
 
                 # Set the primary key (gogen = owner/name)
-                validated_body['gogen'] = f"{validated_body['owner']}/{validated_body['name']}"
+                validated_body['gogen'] = f"{username}/{validated_body['name']}"
 
                 # Remove gistID if present (for migration)
                 validated_body.pop('gistID', None)
             else:
-                logger.error("Owner or name missing in request body")
-                return respond("Owner and name are required fields")
+                logger.error("Name missing in request body")
+                return respond("Name is a required field")
         else:
             logger.warning("No config found in request body")
 
@@ -103,4 +97,4 @@ def lambda_handler(event, context):
 
     except Exception as e:
         logger.error(f"Error in lambda_handler: {str(e)}", exc_info=True)
-        return respond(e) 
+        return respond(e)