-
Notifications
You must be signed in to change notification settings - Fork 886
docs(docs): add pgfence migration safety integration guide #7564
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
flvmnt
wants to merge
5
commits into
prisma:main
Choose a base branch
from
flvmnt:docs/add-pgfence-integration
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+183
−0
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
d1ca076
docs(docs): add pgfence migration safety integration guide
flvmnt 3415b32
fix: pin pgfence version, remove hidden unicode chars, fix links
flvmnt 5f97f63
fix: resolve CodeRabbit findings — match trigger text, strip hidden c…
flvmnt f43a68d
fix: pin install version, improve stats file docs link
flvmnt 664ee22
fix: correct HIGH risk lock description and add globstar to CI examples
flvmnt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,6 +7,7 @@ | |
| "vercel-deployment", | ||
| "deno", | ||
| "datadog", | ||
| "pgfence", | ||
| "permit-io", | ||
| "shopify", | ||
| "neon-accelerate", | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,164 @@ | ||
| --- | ||
| title: pgfence | ||
| description: Analyze Prisma Migrate SQL files for dangerous lock patterns, risk levels, and safe rewrite recipes before deploying to production | ||
| url: /guides/integrations/pgfence | ||
| metaTitle: How to use pgfence with Prisma Migrate for safe PostgreSQL migrations | ||
| metaDescription: Learn how to analyze Prisma migration SQL files with pgfence to detect dangerous lock patterns, understand risk levels, and get safe rewrite recipes before deploying. | ||
| --- | ||
|
|
||
| ## Introduction | ||
|
|
||
| [pgfence](https://pgfence.dev) is a PostgreSQL migration safety CLI that analyzes SQL migration files and reports lock modes, risk levels, and safe rewrite recipes. It uses PostgreSQL's actual parser ([libpg-query](https://github.com/pganalyze/libpg-query-node)) to understand exactly what each DDL statement does, what locks it acquires, and what it blocks. | ||
|
|
||
| Prisma Migrate generates plain SQL files at `prisma/migrations/*/migration.sql`. pgfence can analyze those files directly, catching dangerous patterns before they reach production. | ||
|
|
||
| Common issues pgfence detects include: | ||
|
|
||
| - `CREATE INDEX` without `CONCURRENTLY` (blocks writes) | ||
| - `ALTER COLUMN TYPE` (full table rewrite with `ACCESS EXCLUSIVE` lock) | ||
| - `ADD COLUMN ... NOT NULL` without a safe default (blocks reads and writes) | ||
| - Missing `lock_timeout` settings (risk of lock queue death spirals) | ||
|
|
||
| For each dangerous pattern, pgfence provides the exact safe alternative -- the expand/contract sequence you should use instead. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - [Node.js v20+](https://nodejs.org/) | ||
| - A Prisma project using PostgreSQL as the database provider | ||
| - Existing migrations in `prisma/migrations/` | ||
|
|
||
| ## 1. Install pgfence | ||
|
|
||
| Add pgfence as a development dependency in your project: | ||
|
|
||
| ```npm | ||
| npm install -D @flvmnt/pgfence@0.2.1 | ||
| ``` | ||
|
|
||
| ## 2. Analyze your migrations locally | ||
|
|
||
| Run pgfence against your Prisma migration files: | ||
|
|
||
| ```bash | ||
| npx --yes @flvmnt/pgfence@0.2.1 analyze prisma/migrations/**/migration.sql | ||
| ``` | ||
|
|
||
| pgfence parses every SQL statement and reports the lock mode, risk level, and any safe rewrites available. | ||
|
|
||
| ### Understanding the output | ||
|
|
||
| pgfence assigns a risk level to each statement based on the PostgreSQL lock it acquires: | ||
|
|
||
| | Risk level | Meaning | | ||
| |------------|---------| | ||
| | **LOW** | Safe operations with minimal locking (e.g., `ADD COLUMN` with a constant default on PG 11+) | | ||
| | **MEDIUM** | Operations that block writes but not reads (e.g., `CREATE INDEX` without `CONCURRENTLY`) | | ||
| | **HIGH** | Operations that block writes and competing DDL, but not plain reads (e.g., `ADD FOREIGN KEY` without `NOT VALID`) | | ||
| | **CRITICAL** | Operations that take `ACCESS EXCLUSIVE` locks on large tables (e.g., `DROP TABLE`, `TRUNCATE`) | | ||
coderabbitai[bot] marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| Here is an example of pgfence analyzing a migration that adds an index without `CONCURRENTLY`: | ||
|
|
||
| ```sql | ||
| -- prisma/migrations/20240115_add_index/migration.sql | ||
| CREATE INDEX "User_email_idx" ON "User"("email"); | ||
| ``` | ||
|
|
||
| ```bash | ||
| npx --yes @flvmnt/pgfence@0.2.1 analyze prisma/migrations/20240115_add_index/migration.sql | ||
| ``` | ||
|
|
||
| pgfence will flag this as a `MEDIUM` risk because `CREATE INDEX` takes a `SHARE` lock, which blocks all writes to the table for the duration of the index build. It will suggest using `CREATE INDEX CONCURRENTLY` instead. | ||
|
|
||
| :::warning | ||
| Prisma Migrate does not generate `CONCURRENTLY` variants automatically. If pgfence flags an index creation, you should manually edit the generated migration SQL file to add `CONCURRENTLY` before applying it. Note that `CREATE INDEX CONCURRENTLY` cannot run inside a transaction, so you will also need to ensure the migration runs outside a transaction block. | ||
| ::: | ||
|
|
||
| ## 3. Use JSON output for programmatic checks | ||
|
|
||
| pgfence supports JSON output, which is useful for integrating with other tools or scripts: | ||
|
|
||
| ```bash | ||
| npx --yes @flvmnt/pgfence@0.2.1 analyze --output json prisma/migrations/**/migration.sql | ||
| ``` | ||
|
|
||
| You can also set a maximum risk threshold for CI pipelines. The command exits with code 1 if any statement exceeds the threshold: | ||
|
|
||
| ```bash | ||
| npx --yes @flvmnt/pgfence@0.2.1 analyze --ci --max-risk medium prisma/migrations/**/migration.sql | ||
| ``` | ||
|
|
||
| ## 4. Add pgfence to your CI pipeline | ||
|
|
||
| Add pgfence as a safety check that runs before `prisma migrate deploy` in your CI/CD pipeline. This catches dangerous migration patterns before they reach your production database. | ||
|
|
||
| Here is a GitHub Actions workflow that runs pgfence on every pull request that includes migration changes: | ||
|
|
||
| ```yaml title=".github/workflows/migration-safety.yml" | ||
| name: Migration safety check | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - prisma/migrations/** | ||
|
|
||
| jobs: | ||
| pgfence: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout repo | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Node.js | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: "20" | ||
|
|
||
| - name: Install dependencies | ||
| run: npm ci | ||
|
|
||
| - name: Run pgfence analysis | ||
| run: | | ||
| shopt -s globstar | ||
| npx --yes @flvmnt/pgfence@0.2.1 analyze --ci --max-risk medium prisma/migrations/**/migration.sql | ||
| ``` | ||
|
|
||
| This workflow only triggers when migration files change. If pgfence detects any statement with risk higher than `MEDIUM`, the check fails and blocks the pull request from merging. | ||
|
|
||
| :::info | ||
| You can adjust the `--max-risk` threshold to match your team's risk tolerance. Options are `low`, `medium`, `high`, and `critical`. | ||
| ::: | ||
|
|
||
| ### Combining pgfence with deploy | ||
|
|
||
| If you have an existing deployment workflow, add pgfence as a step before `prisma migrate deploy`: | ||
|
|
||
| ```yaml title=".github/workflows/deploy.yml" | ||
| - name: Run pgfence migration safety check | ||
| run: | | ||
| shopt -s globstar | ||
| npx --yes @flvmnt/pgfence@0.2.1 analyze --ci --max-risk medium prisma/migrations/**/migration.sql | ||
|
|
||
| - name: Apply pending migrations | ||
| run: npx prisma migrate deploy | ||
| env: | ||
| DATABASE_URL: ${{ secrets.DATABASE_URL }} | ||
| ``` | ||
|
|
||
| ## 5. Size-aware risk scoring (optional) | ||
|
|
||
| pgfence can adjust risk levels based on actual table sizes. A `CREATE INDEX` on a 100-row table is very different from the same operation on a 10-million-row table. | ||
|
|
||
| To use size-aware scoring without giving pgfence direct database access, export a stats snapshot from your database and pass it to pgfence: | ||
|
|
||
| ```bash | ||
| npx --yes @flvmnt/pgfence@0.2.1 analyze --stats-file pgfence-stats.json prisma/migrations/**/migration.sql | ||
| ``` | ||
|
|
||
| The stats file contains row counts and table sizes from `pg_stat_user_tables`. Run `npx --yes @flvmnt/pgfence@0.2.1 extract-stats --db-url <connection-string>` to generate this file, or see the [pgfence README](https://github.com/flvmnt/pgfence#db-size-aware-risk-scoring) for details. | ||
|
|
||
| ## Next steps | ||
|
|
||
| - [pgfence documentation and source code](https://github.com/flvmnt/pgfence) | ||
| - [pgfence on npm](https://www.npmjs.com/package/@flvmnt/pgfence) | ||
| - [Prisma Migrate overview](/orm/prisma-migrate) | ||
| - [Deploying database changes with Prisma Migrate](/orm/prisma-client/deployment/deploy-database-changes-with-prisma-migrate) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.