feat: move stage 1 to nightly build #1940
Draft
+83
−72
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Depends on #1941
Refactor AMI builds to use nightly base image
Summary
This PR refactors the AMI build pipeline to separate platform provisioning (stage 1) from application installation (stage 2). Stage 1 now builds a single version-agnostic base image nightly, which all
stage 2 builds consume.
Changes
New workflow:
.github/workflows/base-image-nightly.yml- Runs daily at 2 AM UTC, builds version-agnostic base stage 1 AMI, replicates to us-east-1 and ap-southeast-1Stage 1 changes:
amazon-arm64-nix.pkr.hcl- Added base-nightly mode with conditional AMI namingebssurrogate/scripts/surrogate-bootstrap-nix.sh- Removed postgres version variablesansible/tasks/setup-postgrest.yml- moved to stage 2ansible/playbook.yml- Move PostgREST installation to stage 2 onlyStage 2 changes:
stage2-nix-psql.pkr.hcl- Search for base-nightly AMI instead of versioned stage 1Workflow updates:
.github/workflows/ami-release-nix.yml- Remove stage 1 build.github/workflows/ami-release-nix-single.yml- Remove stage 1 build.github/workflows/testinfra-ami-build.yml- Remove stage 1 buildRationale
Current state: Every release workflow builds stage 1 from scratch for each postgres version (15, 17, orioledb-17), taking 30+ minutes and creating 3-4 redundant AMIs per release.
Problem: Stage 1 installs OS packages, system dependencies, and tooling that are identical across all postgres versions. Rebuilding this repeatedly is inefficient.
Solution: Build stage 1 once per night as a version-agnostic base image. All postgres versions share the same tested platform base, and stage 2 handles version-specific installation.
Benefits
Testing
Migration Notes