Skip to content

Conversation

hcpss-gdelrosario
Copy link

This change will allow PVE to utilize ZFS Send -c compression option to speed up intra-cluster migration and replication.

The current implementation does not compress streams which significantly slows down migration on VMs/CTs with large data sets (ie, databases, filestorage ...). For example, if a VM is assigned a 512GB ZFS dataset and is migrated into another host, the ZFS Send will transmit 512GB of data regardless if the VM internally used 1GB of disk space or +500GB of space. Using the -c option, upon migrating the VM, due to compression only used datablocks are sent significantly increasing speed.

Note that even if the underlying ZFS source and destination datasets have the setting compression=on (the default setting in PVE), ZFS Send without the -c option will "unpack" the blocks and send the blocks uncompressed to the receiver.

I've tested this on a production cluster on PVE 8.3.3 with many instances of VMs and CTs with ZFS datasets ranging from 16GB all the way up to 1TB+. Migration and replication works as they did before without any issues.

utilize ZFS Send compression "-c" option to speed up intra-cluster migration
@Fantu
Copy link

Fantu commented Feb 26, 2025

Hi, if you didn't see it on the main page:

How-To Contribute
Proxmox uses a mailing list approach for development and won't accept GitHub based pull-requests. Please check our Developer Documentation for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants