[VDO-5752] Merge Feature/add-vdoformat-into-kernel into main branch

[bjohnsto]

Draft cover letter for upstream submission. Skips test and user mode changes

---

Subject: [PATCH 0/9] dm vdo: add in-kernel VDO formatting support

This series moves the formatting capability of the userspace tool vdoformat into the kernel dm-vdo target itself, allowing a VDO volume to be created and formatted in a single dmsetup create invocation. Currently, creating a new VDO volume requires a two-step process: first running the vdoformat tool to write the on-disk metadata structures, and then loading the dm target via dmsetup create.

The kernel determines whether a device needs formatting by reading the geometry block (block 0 on the storage device) during initialization. If the block is entirely zeroed, the device is treated as unformatted and the target formats it. Otherwise, the existing VDO is loaded as before. The actual writes of the formatted metadata to disk are deferred to the dm target's pre-resume operation rather than being performed during dmsetup create.

The series is organized as follows:

Patches 1-2 add initialization functions for the two primary on-disk metadata structures. vdo_initialize_volume_geometry() populates the geometry block, computing the index size from the UDS configuration to determine region boundaries. vdo_initialize_component_states() populates the super block, including location information for the slab depot, block map, and recovery journal. The uds_compute_index_size() function is added to calculate the space required for the uds indexer on disk.

Patch 3 extends the dm table line with three new optional parameters: indexMemory (UDS index memory size), indexSparse (dense vs sparse index), and slabSize (blocks per allocation slab). These values are parsed, validated, and stored in the device configuration for use during formatting. The slab size constants are reworked from the single MAX_VDO_SLAB_BITS into explicit MIN_VDO_SLAB_BLOCKS, MAX_VDO_SLAB_BLOCKS, and DEFAULT_VDO_SLAB_BLOCKS values. The target version is bumped from 9.1.0 to 9.2.0 to reflect this table line change.

Patch 4 adds an upfront validation check that the logical size passed via the table line does not exceed MAXIMUM_VDO_LOGICAL_BLOCKS.

Patch 5 adds a vdo_encode_volume_geometry() function that writes the magic number, header, geometry fields, and CRC32 checksum into a buffer so that it can be written to disk.

Patch 6 introduces a vdo_geometry_block structure containing a vio and buffer, mirroring the existing vdo_super_block structure. Both are now initialized at VDO startup and freed at shutdown, establishing the infrastructure needed to read and write the geometry block using the same mechanisms as the super block.

Patch 7 adds vdo_submit_metadata_vio_wait(), a synchronous I/O submission helper that blocks until completion. This is needed for I/O during early initialization before work queues are available. The existing read_geometry_block() is refactored to use it.

Patch 8 adds the core formatting logic. The vdo_format() function generates a UUID and nonce, initializes the volume geometry and component states, and marks the VDO as needing formatting. The initialization path is updated to read the geometry block and branch: if all zeros, format; otherwise, parse the existing geometry and load.

Patch 9 completes the feature by writing the formatted metadata to disk. New vdo_save_super_block() and vdo_save_geometry_block() functions perform asynchronous writes of the super block and geometry block respectively. A vdo_clear_layout() function zeroes the UDS index's first block, the block map partition, and the recovery journal partition. These operations are driven by new phases in the pre-load state machine (PRE_LOAD_PHASE_FORMAT_*), ensuring that disk writes happen during pre-resume rather than during dmsetup create.

Diffstat across the series:

12 files changed, 645 insertions(+), 133 deletions(-)

constants.h | 17 +-
dm-vdo-target.c | 139 ++++++++++++++-
encodings.c | 229 +++++++++++++++++++++++----
encodings.h | 18 ++-
index-layout.c | 6 +-
indexer.h | 2 -
io-submitter.c | 31 ++++
io-submitter.h | 4 +
status-codes.c | 2 +
types.h | 3 +
vdo.c | 308 +++++++++++++++++++++++++++----------
vdo.h | 19 +++

---

In addition, the remaining commits provide testing (user mode and functional) of the new feature, as well as vdoformat refactoring to use the formatting functions that now exist in encodings.[ch].

[lorelei-sakai]

Just a couple minor comments on commit messages. The one for patch 8 () is more important since it'll be going upstream. The cover letter draft does mention a reason for that patch, but having it in the commit message for that patch would be good.

The cover letter draft similarly is pretty good. I'd make some minor adjustments but there's nothing critical:
In the first paragraph, I'd swap the sentences. Start with what the series does ("This series moves the formatting capability...", then move on to why. ("Currently, ...")
The second paragraph, detailing how it works, looks good.

The patch-by -patch breakdown seems a little verbose, but fine. There's a missing backtick in the patch 9 paragraph, but the real cover letter will need to be all text (no markup) for e-mail, so that'll all have to get stripped out anyway.

In general, though, I'd say that if there is something interesting to say about a specific patch, the information should be in the commit message for that patch rather than (or possibly in addition to) in the cover letter. Anyone reading the cover letter should also have the entire series available to them, but the patches will get committed and the cover letter won't. So a later reader will have to depend on the commit messages. To the extent that this per-patch information is useful, could you try to make sure it's reflected in the commit messages themselves, too?

[lorelei-sakai]

Oh, commit messages can also be patched up/massaged when we submit this upstream, but I would prefer to do it sooner if possible.

[lorelei-sakai]

Two minor but possibly worrisome message edits.
Not even sure these are worth fussing over.

[lorelei-sakai]

Great, let's do this.