Skip to content

Commit c661a2c

Browse files
authored
Update for rewriting and schema sharding (#42)
1 parent de0dd54 commit c661a2c

File tree

4 files changed

+73
-8
lines changed

4 files changed

+73
-8
lines changed

docs/configuration/pgdog.toml/.pages

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,6 @@ title: "pgdog.toml"
22
nav:
33
- 'general.md'
44
- 'databases.md'
5+
- 'rewrite.md'
56
- '...'
67
- 'plugins.md'
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
---
2+
icon: material/alpha-r-box-outline
3+
---
4+
5+
# Rewrite
6+
7+
The `rewrite` section controls PgDog's automatic SQL rewrites for sharded clusters. It affects shard-key updates and multi-row INSERT statements, and can be toggled globally or per-policy.
8+
9+
## Options
10+
11+
```toml
12+
[rewrite]
13+
enabled = false
14+
shard_key = "error"
15+
split_inserts = "error"
16+
```
17+
18+
| Field | Description | Default |
19+
| --- | --- | --- |
20+
| `enabled` | Master toggle; when `false`, PgDog parses but never applies rewrite plans. | `false` |
21+
| `shard_key` | Behaviour when an `UPDATE` changes a sharding key.<br>`error` rejects the statement.<br>`rewrite` migrates the row between shards.<br>`ignore` forwards it unchanged. | `"error"` |
22+
| `split_inserts` | Behaviour when a sharded table receives a multi-row `INSERT`.<br>`error` rejects the statement.<br>`rewrite` fans the rows out to their shards.<br>`ignore` forwards it unchanged. | `"error"` |
23+
24+
!!! note "Two-phase commit"
25+
PgDog recommends enabling [`general.two_phase_commit`](general.md#two_phase_commit) when either policy is set to `rewrite`. Without it, rewrites are committed shard-by-shard and can leave partial changes if a shard fails.
26+
27+
## Runtime overrides
28+
29+
The admin database exposes these toggles via `SET`:
30+
31+
```postgresql
32+
SET rewrite_enabled TO true; -- mirrors [rewrite].enabled
33+
SET rewrite_shard_key_updates TO rewrite; -- error | rewrite | ignore
34+
SET rewrite_split_inserts TO rewrite; -- error | rewrite | ignore
35+
```
36+
37+
Switches apply to subsequent sessions once the cluster reloads configuration. Session-level overrides allow canary testing before persisting them in `pgdog.toml`.
38+
39+
## Limitations
40+
41+
* Shard-key rewrites require the `WHERE` clause to resolve to a single row; otherwise PgDog rolls back and raises `rewrite.shard_key="rewrite" is not yet supported ...`.
42+
* Split INSERT rewrites must run outside explicit transactions so PgDog can orchestrate per-shard `BEGIN`/`COMMIT` cycles. Inside a transaction PgDog returns `25001` and leaves the client transaction intact.
43+
* Both features fall back to `error` semantics while `rewrite.enabled = false` or when PgDog cannot determine a target shard.
44+
45+
See [feature docs](../../features/sharding/sharding-functions.md#rewrite-behaviour) for walkthroughs of these flows.

docs/configuration/pgdog.toml/sharded_schemas.md

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -29,27 +29,37 @@ SELECT * FROM customer_a.users -- Prefixed with the schema name.
2929
WHERE admin = true;
3030
```
3131

32+
You can add multiple entries per database. Mappings are matched by schema name first; if none match, PgDog falls back to a default rule.
33+
3234
## Default shard
3335

34-
For queries that don't specify a schema or for which a mapping doesn't exist, the default behavior is to send it to all shards. If this is not desirable, you can configure a default shard, like so:
36+
For queries that don't specify a schema or for which a mapping doesn't exist, the default behavior is to send it to all shards. If this is not desirable, add an entry without a `name` to choose a default shard:
3537

3638
```toml
3739
[[sharded_schemas]]
3840
database = "prod"
3941
shard = 0
4042
```
4143

42-
For example, the following queries will be sent to shard zero:
44+
PgDog now sends any unmapped schema to shard zero, including plain references (`SELECT * FROM pg_stat_activity`) and schemas created after the mapping file was generated.
4345

44-
```postgresql
45-
-- No schema specified.
46-
SELECT * FROM pg_stat_activity;
46+
## Broadcast mappings
4747

48-
-- Schema isn't mapped in the config.
49-
SELECT * FROM customer_c.users
50-
WHERE admin = true;
48+
If you need a single configuration entry to cover “all shards”, set `all = true`. PgDog still accepts a `name` for documentation purposes, but ignores the shard number and forwards the query to every shard:
49+
50+
```toml
51+
[[sharded_schemas]]
52+
database = "prod"
53+
name = "reporting"
54+
all = true
5155
```
5256

57+
This is useful for schemas that host reference tables replicated everywhere.
58+
59+
## DDL routing
60+
61+
Schema mappings apply to both DDL and DML. Fully-qualified statements such as `CREATE TABLE customer_b.users (...)` use the same shard resolution as regular queries, keeping schema changes aligned across shards.
62+
5363
## Manual routing
5464

5565
If you need to query a specific shard or can't specify the schema name in the query, you can add it to a comment, for example:

docs/features/sharding/sharding-functions.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -180,6 +180,15 @@ shard = 0
180180

181181
This will send all queries that don't specify a schema or use a schema without a mapping to shard zero.
182182

183+
## Rewrite behaviour
184+
185+
PgDog can transparently move writes between shards when [`rewrite`](../../configuration/pgdog.toml/rewrite.md) is enabled.
186+
187+
* **Shard-key updates** (`rewrite.shard_key = "rewrite"`) delete the matching row from its current shard and re-insert it on the shard implied by the new key. Exactly one row must match the `WHERE` clause; PgDog aborts rewrites that affect multiple rows or unresolved shards.
188+
* **Split INSERTs** (`rewrite.split_inserts = "rewrite"`) decompose multi-row `INSERT` statements so each shard receives only the rows it owns. PgDog opens per-shard transactions and can escalate to two-phase commit when configured to preserve atomicity across shards.
189+
190+
Both features require `rewrite.enabled = true`, operate only on sharded tables, and fall back to returning errors when PgDog cannot determine a safe rewrite plan. Running them alongside [`general.two_phase_commit`](../../configuration/pgdog.toml/general.md#two_phase_commit) is recommended to guarantee atomic outcomes.
191+
183192
## Read more
184193

185194
- [COPY command](copy.md)

0 commit comments

Comments
 (0)