[Feature Request] Generic metadata sidecar export for completed downloads

[EdLaughton]

Generic metadata sidecar export for completed downloads

I’d like to propose a generic metadata sidecar/export system for completed downloads, built around DownloadTask rather than a single downstream tool or a single metadata provider.

I previously worked on a more CWA/Hardcover-specific prototype on this branch:

• https://github.com/EdLaughton/shelfmark/tree/feature/cwa-sidecar-payload

I also opened a PR based on that prototype, but I’m planning to close it in favour of this broader feature request.

Why this would be useful

There is clear value in being able to emit structured metadata alongside delivered files for downstream consumers such as:

• CWA
• Audiobookshelf-style metadata import
• Grimmory / BookLore-style workflows
• other future consumers

The key point is not to hard-code one JSON shape, but to provide a generic export layer that can support multiple output formats.

Proposed direction

Build this around a normalized internal export model attached to completed DownloadTasks, then render one or more target-specific output formats from that model.

High-level flow:

search/release source -> DownloadTask -> normalized export metadata -> formatter(s) -> sidecar file(s)

Why DownloadTask seems like the right seam

DownloadTask already appears to carry much of the right information for this layer, including:

• source
• title / author / year / format
• content type
• original / retry download URLs and protocol
• output-related fields
• user / request linkage

That makes it a more natural abstraction point than building this around one specific search source or one specific output format.

Suggested design

A reasonable structure would be:

1. Normalized internal export model

   • title / subtitle
   • authors
   • series name / position
   • content type / format
   • source / release provenance
   • known identifiers
   • final delivered filename/path

2. DownloadTask integration

   • build export metadata from the final resolved download/delivery state

3. Formatter layer

   • formatter(s) that turn the normalized model into tool-specific outputs, e.g.:
     • CWA
     • Audiobookshelf-style metadata
     • Grimmory / BookLore-compatible metadata
     • potentially other JSON output formats later

4. Configurable output

   • metadata export disabled by default
   • one or more output formats enabled explicitly

Suggested initial scope

A reasonable first pass would be:

1. a normalized internal export model
2. DownloadTask integration
3. a formatter interface
4. one concrete formatter (for example CWA)
5. config to enable metadata export

Important constraints

• only emit identifiers/metadata that are genuinely known
• do not infer fake exact IDs at sidecar-write time
• write sidecars only once the final delivered file path is known
• keep ordinary delivery behavior unchanged when metadata export is disabled
• keep the core model generic rather than tied to one provider or one consumer

Non-goals for a first pass

• solving every downstream metadata format up front
• speculative provider-specific inference
• broad downloader/search refactors unrelated to metadata export
• forcing all consumers into one single JSON schema
• locking in a metadata shape before the broader metadata/sidecar direction settles

Prior art / relevant context

The earlier prototype sidecar writer already handled a few useful things well:

• writing next to the final delivered file
• partial provenance handling
• safe/atomic-style write behavior

That seems reusable, even if the final feature should sit behind a more generic formatter/export layer rather than being hard-coded for one tool.

If this direction makes sense, I’d be happy to revisit the earlier prototype later from a fresh branch based on current main, but shaped around a generic export model rather than a CWA-specific implementation.

[EdLaughton]

This is intended as a feature request rather than a bug report. I opened it to capture a possible future direction after closing the earlier prototype PR, so please feel free to relabel as enhancement/feature request if that fits better.