What's New
Turbo Mode (--turbo)
30-55x faster encode speed with ~2% ratio tradeoff. Same .dcx format — decompression is unchanged.
datacortex compress logs.ndjson --turbo # 99 MB/s encode
datacortex bench corpus/ -m fast --turbo # benchmark turboBenchmarks (Apple M-series):
| File | Normal Encode | Turbo Encode | Speedup |
|---|---|---|---|
| GH Archive 10MB | 2.7 MB/s | 169 MB/s | 55x |
| NDJSON 10K rows 3.3MB | 2.3 MB/s | 68 MB/s | 30x |
| Twitter API 617KB | 2.2 MB/s | 87 MB/s | 40x |
Improved Compression Ratio
- Raised brotli quality to 10 for 1-16MB files (was 9)
- GH Archive: 12.9% → 12.5% ratio (closed 78% of the brotli-11 gap)
- Overall corpus: 9.8% → 9.6% ratio
Full Changelog
feat: add turbo mode for 30-55x faster encode speedperf: raise brotli quality to 10 for 1-16MB files- 3 new turbo mode tests (393 total)
- Updated README with turbo docs and benchmarks