Add fundamental development configuration

Author: parisk

.example.env

@@ -0,0 +1,3 @@
+POSTGRES_CIDR=172.55.32.0/24
+POSTGRES_SUBNET=172.55.0.0/16
+POSTGRES_GATEWAY=172.55.32.254

.gitignore

@@ -0,0 +1 @@
+.env

Dockerfile

@@ -0,0 +1,16 @@
+FROM postgres:16.2 as base
+
+
+FROM base as primary
+
+COPY ./postgres/primary/docker-entrypoint-initdb.d/ docker-entrypoint-initdb.d/
+
+
+FROM base as secondary
+
+COPY ./postgres/secondary/bin/ /usr/local/bin/
+ENTRYPOINT [ "docker-entrypoint-override.sh" ]
+
+CMD ["postgres"]
+
+FROM primary

README.md

@@ -1,7 +1,55 @@
 # Postgres with LOGIC
 
-Production-ready Postgres with Docker that does not suck, with [LOGIC](https://withlogic.co).
+This repository contains the configuration for production-grade Postgres with Docker, with [LOGIC](https://withlogic.co).  It was first presented on 24 January 2024, at Docker Athens during the presentation [Production grade Postgres with Docker](https://www.youtube.com/watch?v=tJegTc-oLtk)[^1].
+
+The configuration in this repository sets up a primary and secondary Postgres server with streaming replication. This means that the primary server accepts all write queries, which are in turn replicated to the secondary server. Therefore, read queries can be load balanced and performed on both servers.
+
+## Requirements
+
+- Docker Engine 25.0.0 or newer
+- Docker Compose 2.24.9 or newer, for development
+
+## Configuration
+
+The setup of Postgres with LOGIC is configured with environment variables and Docker Secrets for sensitive data.
+
+### Environment variables
+
+- `POSTGRES_CIDR`: The CIDR block from which to allocate IPs in the Docker network and also allow replication from (default: `172.54.32.0/24`)
+- `POSTGRES_GATEWAY`: The gateway to use in the Docker network (default: `172.54.32.254`)
+- `POSTGRES_SUBNET`: The subnet to allocate for the Docker network (default: `172.54.0.0/16`)
+
+For convenience, in development these environment variables can be set in a `.env` environment file. Example file available in [`.example.env`](./.example.env)
+
+## Development
+
+To kick off and evaluate the setup locally, all you have to do is run
+
+```console
+docker compose up
+```
+
+After all containers start, you can validate the setup with the following steps:
+
+1. Create a table on the primary server
+    ```console
+    docker compose exec primary psql -U postgres -c "CREATE TABLE people (name varchar(40));"
+    ```
+2. Insert a couple of rows in the primary server
+    ```console
+    docker compose exec primary psql -U postgres -c "INSERT INTO people VALUES ('grace');"
+    docker compose exec primary psql -U postgres -c "INSERT INTO people VALUES ('alan');"
+    ```
+3. Validate that data can be read from both servers
+    ```console
+    docker compose exec primary psql -U postgres -c "SELECT * FROM people;"
+    docker compose exec secondary psql -U postgres -c "SELECT * FROM people;"
+    ```
+
+---
 
 <p align="center">
-  <img src="https://github.com/withlogicco/postgres/assets/1188592/79797352-66a1-436e-82ac-a6ed33bc5aa8" />
+  <i>🦄 Built with <a href="https://withlogic.co/">LOGIC</a>. 🦄</i>
 </p>
+
+[^1]: Presentation on YouTube: https://www.youtube.com/watch?v=tJegTc-oLtk

compose.yml

@@ -0,0 +1,45 @@
+x-base:
+  &base
+    secrets:
+      - postgres-password
+      - replicator-password
+    environment:
+      POSTGRES_PASSWORD_FILE: /run/secrets/postgres-password
+      POSTGRES_CIDR: ${POSTGRES_CIDR:-172.54.32.0/24}
+    command: ["postgres", "-c", "log_statement=all"]
+
+services:
+  primary:
+    <<: *base
+    build:
+      context: .
+      target: primary
+    volumes:
+      - primary_data:/var/lib/postgresql/data
+
+  secondary:
+    <<: *base
+    build:
+      context: .
+      target: secondary
+    restart: on-failure:3
+    volumes:
+      - secondary_data:/var/lib/postgresql/data
+
+secrets:
+  postgres-password:
+   file: dev/secrets/postgres-password
+  replicator-password:
+   file: dev/secrets/replicator-password
+
+networks:
+  default:
+    ipam:
+      config:
+        - subnet: ${POSTGRES_SUBNET:-172.54.0.0/16}
+          ip_range: ${POSTGRES_CIDR:-172.54.32.0/24}
+          gateway: ${POSTGRES_GATEWAY:-172.54.32.254}
+
+volumes:
+  primary_data:
+  secondary_data:

dev/secrets/postgres-password

@@ -0,0 +1 @@
+development_password

dev/secrets/replicator-password

@@ -0,0 +1 @@
+development_replicator_password

postgres/primary/docker-entrypoint-initdb.d/10-setup-replication-user.sh

@@ -0,0 +1,4 @@
+set  -ex
+
+REPLICATOR_PASSWORD=$(cat /run/secrets/replicator-password)
+psql -c "CREATE ROLE replicator REPLICATION LOGIN PASSWORD '$REPLICATOR_PASSWORD'"

postgres/primary/docker-entrypoint-initdb.d/20-setup-replication-auth.sh

@@ -0,0 +1,6 @@
+set -ex
+
+POSTGRES_CIDR=${POSTGRES_CIDR:-172.54.32.0/24}
+PGDATA=${PGDATA:-/var/lib/postgresql/data}
+
+echo "host replication replicator $POSTGRES_CIDR scram-sha-256" >> $PGDATA/pg_hba.conf

postgres/secondary/bin/10-base-backup.sh

@@ -0,0 +1,6 @@
+set -ex
+
+export PGPASSWORD=$(cat /run/secrets/replicator-password)
+
+pg_basebackup -w -h primary -D $PGDATA -U replicator -P -v -X stream
+chmod -R 0750 $PGDATA