feat(storage): migrate all data mounts to versioned 16/main subdirectories#1649
Draft
marceloneppel wants to merge 2 commits into16/edgefrom
Draft
feat(storage): migrate all data mounts to versioned 16/main subdirectories#1649marceloneppel wants to merge 2 commits into16/edgefrom
marceloneppel wants to merge 2 commits into16/edgefrom
Conversation
…ories
Introduce versioned path constants (POSTGRESQL_DATA_DIR, ARCHIVE_DATA_DIR,
LOGS_DATA_DIR, TEMP_DATA_DIR = <storage-root>/16/main) alongside the existing
storage-root constants. All internal path references are updated to use the
versioned paths.
On snap refresh, _ensure_storage_layout() performs a one-time, idempotent
migration of existing files from each storage root into the versioned
subdirectory, recording completion via a `storage_layout_migrated` flag in
the peer-relation databag. _repair_pg_wal_symlink() updates the pg_wal
symlink to point at the new WAL path after migration.
_reconcile_storage_permissions() runs unconditionally on every refresh to
heal storage root permissions (0755) and pg_wal symlink ownership
(_daemon_:_daemon_), fixing units migrated by earlier builds.
On the primary, _ensure_temp_tablespace_location() migrates the temp
tablespace location in the PostgreSQL catalog. The connection requires
autocommit=True since DROP/CREATE TABLESPACE cannot run inside a transaction
block. _clear_pg_version_dirs() purges PG_* version directories from the
target path before CREATE TABLESPACE to prevent ObjectInUse errors.
patroni.yml.j2 now uses a {{ wal_dir }} template variable instead of the
hardcoded storage-root WAL path.
Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
github-actions
Bot
added
the
Libraries: OK
After the PR introduced versioned storage paths (16/main), three bugs prevented temp tablespaces from recovering after a reboot: 1. TEMP_DATA_DIR was not recreated on tmpfs after the migration flag was already set. Add an else-branch in _ensure_storage_layout that calls mkdir(parents=True, exist_ok=True) without setting owner/mode. The resulting root-owned directory triggers set_up_database's permission check, which calls _handle_temp_tablespace_on_reboot to reinitialise the tablespace directory structure for the tmpfs case. 2. As a belt-and-suspenders fallback, _ensure_temp_tablespace_location now detects an empty TEMP_DATA_DIR (no PG_<ver>_<catver>/ subdir) and drops then recreates the tablespace so PostgreSQL rebuilds the internal directory structure. 3. _ensure_temp_tablespace_location_if_primary raised PostgreSQLUndefinedHostError when primary_endpoint was None during early startup. Return True early when the endpoint is not yet available. The persistent storage integration test assertion for the library's "Fixed permissions" log message is softened to a warning: that message only appears when permissions needed fixing, but units initialised by _migrate_storage_mount already have correct owner/mode so the fix path is not triggered. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue
Solution
Introduce versioned path constants (POSTGRESQL_DATA_DIR, ARCHIVE_DATA_DIR, LOGS_DATA_DIR, TEMP_DATA_DIR = /16/main) alongside the existing storage-root constants. All internal path references are updated to use the versioned paths.
On snap refresh, _ensure_storage_layout() performs a one-time, idempotent migration of existing files from each storage root into the versioned subdirectory, recording completion via a
storage_layout_migratedflag in the peer-relation databag. _repair_pg_wal_symlink() updates the pg_wal symlink to point at the new WAL path after migration._reconcile_storage_permissions() runs unconditionally on every refresh to heal storage root permissions (0755) and pg_wal symlink ownership (daemon:daemon), fixing units migrated by earlier builds.
On the primary, _ensure_temp_tablespace_location() migrates the temp tablespace location in the PostgreSQL catalog. The connection requires autocommit=True since DROP/CREATE TABLESPACE cannot run inside a transaction block. clear_pg_version_dirs() purges PG* version directories from the target path before CREATE TABLESPACE to prevent ObjectInUse errors.
patroni.yml.j2 now uses a {{ wal_dir }} template variable instead of the hardcoded storage-root WAL path.
Checklist