feat: add --direct-io CLI flag for FUSE page cache bypass#28
Merged
Conversation
Add opt-in FOPEN_DIRECT_IO support via --direct-io flag. When enabled, every read/write goes through the FUSE handler instead of being served from the kernel page cache. Useful for benchmarking real CAS throughput. Not recommended for production: disables efficient mmap caching, which is critical for safetensors/transformers workloads where tensors are memory-mapped and re-read across forward passes.
POSIX Compliance (pjdfstest) |
Benchmark Results |
When direct_io is enabled, the prefetch buffer drains consumed bytes after serving reads. Re-reads at already-consumed offsets must refetch from CAS, preventing the buffer from acting as a read cache. Without direct_io (default), the buffer retains data for re-reads, which works well with the kernel page cache handling cross-call caching.
- Pass args.direct_io to VirtualFs::new (missing arg caused build failure) - Disable seek window in forward-only mode (try_serve_seek returns None, drain_to_seek skips seek population) so re-reads truly refetch - Guard drain on to_read > 0 so zero-length reads are no-ops
- open()/create(): only set FOPEN_DIRECT_IO on read-only opens to avoid surfacing EBADF on read-after-write with streaming write handles - setup(): pass direct_io=false when is_nfs, since NFS has no equivalent and it would misleadingly enable forward-only prefetch
- open()/create(): only skip DIRECT_IO for simple streaming write handles; advanced-write O_RDWR handles are local files and support reads - setup(): log warning when --direct-io is used with NFS (no equivalent)
Only O_RDWR opens in simple (non-advanced) mode need DIRECT_IO skipped (streaming handles can't read). O_WRONLY and create() can safely use it.
Mirror the open() guard: create() with O_RDWR in simple mode produces a streaming write-only handle, so DIRECT_IO would surface EBADF on reads.
Request FUSE_DIRECT_IO_ALLOW_MMAP in init() when --direct-io is active, so mmap() works on files with FOPEN_DIRECT_IO (Linux 6.6+). Without this, safetensors and other mmap-based readers fail with EINVAL.
Log a warning instead of silently proceeding when the kernel doesn't support FUSE_DIRECT_IO_ALLOW_MMAP, so users know mmap may fail.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--direct-ioCLI flag (default: off) that setsFOPEN_DIRECT_IOon file open/createUseful for benchmarking real CAS throughput without page cache or buffer inflation. Not recommended for production (disables efficient mmap caching for safetensors workloads).
The bench script (PR #25) will use
HF_DIRECT_IO=1to enable this during fio runs.