The weight columns in variants.parquet are all written as f64 today.
That is more precision than these channels need, and it makes the metadata side heavier than necessary.
What to check:
- downcast weight channels to
f32
- measure on-disk size and decode cost before and after
- look at float-focused compression for these columns
Could use pco here. Could also just switch to f32 and keep the current Parquet path if that gets most of the gain.
The weight columns in
variants.parquetare all written asf64today.That is more precision than these channels need, and it makes the metadata side heavier than necessary.
What to check:
f32Could use
pcohere. Could also just switch tof32and keep the current Parquet path if that gets most of the gain.