Questions about moving and merging individual snapshots

[gyurmogyuri]

Hello everyone,

I’m glad I found this project! It looks really interesting, and I’m currently exploring its features.

I have a specific use case I'd like to ask about: is it possible to migrate a single backup snapshot to another machine, rather than moving the entire repository? Additionally, would it be possible to merge a snapshot from a third location back into this setup?

How would you recommend doing this? I'd appreciate any advice or suggested workflows you might have.

Thanks for your help!

[Ian2020]

+1 but also to add this is difficult in the similar tool restic. The user either has to start an new dest repo from scratch, copying "chunker params" from the source repo or restore the snapshot somewhere and then back it up again to the destination repo. That is to say a feature like this could be difficult to add if not considered early in the repo design (I assume).

For more on this:

• Ensuring deduplication for copied snapshots (restic manual)
• Copy snapshots between repositories without --copy-chunker-params (restic forum)

[goebbe]

@gyurmogyuri Thanks for posting your question.
Could you elaborate what the use case would be for this type of feature?
What would be the objective of "migrating" a single snapshot to another machine?

You could, of course, do a complete restore of a singe snapshot and do a backup of this very restore to a different machine. But I really have a hard time to understand why you would like to-do this?

What would be the objective of "migrating" and why would you like to "merging" a snapshot from one machine with another repository?

[gyurmogyuri]

@goebbe Thank you for the questions and your interest! I'd be happy to elaborate on the specific use cases.

The classic "textbook" backup model usually consists of many small clients and a single, high-capacity central server that stores everything. In my case, however, I am working with a multi-stage, distributed environment.

Here is what I mean by these concepts in practice:

1. "Merge" (Merging / Syncing without extraction):
Imagine a large file system where I continuously work on projects. I make periodic backups of a selected subset of these projects to a portable, local device (e.g., a mobile phone). When I want to ingest this snapshot into the main backup server, I want a simple merge with the existing repository instead of a traditional "migrate" (or restore + new backup).
Why? When a backup is created from live data, the software chunks, encrypts, and hashes the data, and builds the metadata database. I would configure the cryptographic and chunking parameters to be compatible across devices. This way, when copying to the server, I want to avoid the computationally expensive "extract -> re-read -> re-encrypt" process. The goal is to simply transfer the already deduplicated chunks/blocks directly into the server's repository.

2. "Moving / Extracting" (Moving a snapshot to ANOTHER backup server):
This is the reverse direction. I have a massive central backup (repo) containing a huge amount of data. I want to extract a specific project (a snapshot) from it and move it to the repository of another, independent backup server, strictly in a backup (compressed, deduplicated) format.
Important note: I do not want to restore the data to client machines for actual work here; rather, I want to move snapshots between backup servers.
The goal of this is selective redundancy. If the main backup server goes down, I can still restore the project from another backup server. However, I do not want (and it would be unnecessary) to mirror/replicate the entire massive main repo bit-by-bit to secondary servers. I strictly want to transfer the individual snapshots of the most critical projects between the servers.

3. "Snapshot Shaping" (Reshaping directory structure at the metadata level):
The purpose of this feature is to allow the original directory structure to be freely modified, or even to save a completely new, rearranged snapshot on the server during the porting (moving or merging) of snapshots and sources.
The key here is that this rearrangement should happen exclusively through changes in the database (metadata). I absolutely do not want to physically extract (restore) the files, move them at the file system level, and then re-process (re-chunk) them. The modification should consist purely of rearranging the pointers within the deduplication database.

In short: My goal is to be able to freely move, merge, and shape snapshots between different backup repos (servers) without having to restore anything at the file system level or re-process the data.