Skip to content

fix: prevent poll from deleting inodes with open file handles#74

Merged
XciD merged 4 commits intomainfrom
fix/poll-open-handles
Mar 26, 2026
Merged

fix: prevent poll from deleting inodes with open file handles#74
XciD merged 4 commits intomainfrom
fix/poll-open-handles

Conversation

@XciD
Copy link
Copy Markdown
Member

@XciD XciD commented Mar 26, 2026

Summary

Fixes REVIEW.md finding 2-#69: the poll thread could remove an inode from the table while file handles were still referencing it, causing read()/getattr()/release() on those handles to fail with ENOENT.

Root cause: apply_poll_diff called inode_table.remove(ino) without checking whether any OpenFile entries referenced that inode.

Fix

  • Wrap open_files in Arc so it can be shared with the poll task
  • Pass open_files to apply_poll_diff, skip deletion when handles exist
  • Files with open handles stay in the inode table until all handles are released

Test

  • poll_skips_deletion_with_open_handles: opens a file, simulates remote deletion via poll, verifies the inode survives while the handle is open

The poll thread could remove an inode from the table while file handles
were still referencing it, causing read/getattr/release to fail with
ENOENT. Now checks open_files before removing, keeping the inode alive
until all handles are released.

- Wrap open_files in Arc so it can be shared with the poll task
- Pass open_files to apply_poll_diff, skip deletion when handles exist
- Add poll_skips_deletion_with_open_handles test
@github-actions
Copy link
Copy Markdown

POSIX Compliance (pjdfstest)

============================================================
  pjdfstest POSIX Compliance Results
------------------------------------------------------------
  Files: 130/130 passed    Tests: 832 total (0 subtests failed)
  Result: PASS
------------------------------------------------------------
  Category               Passed    Total   Status
  -------------------- -------- -------- --------
  chflags                     5        5       OK
  chmod                       8        8       OK
  chown                       6        6       OK
  ftruncate                  13       13       OK
  granular                    5        5       OK
  mkdir                       9        9       OK
  open                       19       19       OK
  posix_fallocate             1        1       OK
  rename                     10       10       OK
  rmdir                      11       11       OK
  symlink                    10       10       OK
  truncate                   13       13       OK
  unlink                     11       11       OK
  utimensat                   9        9       OK
============================================================

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 26, 2026

Benchmark Results

============================================================
  Benchmark — 50MB
------------------------------------------------------------
  Metric                                 FUSE          NFS
  ------------------------------ ------------ ------------
  Sequential read                    236.0 MB/s     247.4 MB/s
  Sequential re-read                1606.2 MB/s    2265.8 MB/s
  Range read (1MB@25MB)               37.9 ms         0.2 ms
  Random reads (100x4KB avg)          31.9 ms         0.0 ms
  Sequential write (FUSE)           1409.5 MB/s
  Close latency (CAS+Hub)            0.634 s
  Write end-to-end                    74.7 MB/s
  Dedup write                       1672.7 MB/s
  Dedup close latency                0.088 s
  Dedup end-to-end                   423.0 MB/s
============================================================
============================================================
  Benchmark — 200MB
------------------------------------------------------------
  Metric                                 FUSE          NFS
  ------------------------------ ------------ ------------
  Sequential read                   1045.9 MB/s     950.5 MB/s
  Sequential re-read                1762.1 MB/s    2383.5 MB/s
  Range read (1MB@25MB)               32.8 ms         0.2 ms
  Random reads (100x4KB avg)          31.9 ms         0.0 ms
  Sequential write (FUSE)           1531.3 MB/s
  Close latency (CAS+Hub)            0.111 s
  Write end-to-end                   827.7 MB/s
  Dedup write                       1487.8 MB/s
  Dedup close latency                0.127 s
  Dedup end-to-end                   766.3 MB/s
============================================================
============================================================
  Benchmark — 500MB
------------------------------------------------------------
  Metric                                 FUSE          NFS
  ------------------------------ ------------ ------------
  Sequential read                   1626.2 MB/s    1372.7 MB/s
  Sequential re-read                1710.7 MB/s    2469.6 MB/s
  Range read (1MB@25MB)               35.6 ms         0.2 ms
  Random reads (100x4KB avg)          33.2 ms         0.0 ms
  Sequential write (FUSE)           1420.9 MB/s
  Close latency (CAS+Hub)            0.107 s
  Write end-to-end                  1088.6 MB/s
  Dedup write                       1432.4 MB/s
  Dedup close latency                0.115 s
  Dedup end-to-end                  1077.3 MB/s
============================================================
============================================================
  fio Benchmark Results
------------------------------------------------------------
  Job                        FUSE MB/s   NFS MB/s  FUSE IOPS   NFS IOPS
  ------------------------- ---------- ---------- ---------- ----------
  seq-read-100M                  502.5      398.4                      
  seq-reread-100M               2500.0       19.0                      
  rand-read-4k-100M                0.1        0.1         18         19
  seq-read-5x10M                 568.2      769.2                      
  rand-read-10x1M                  0.1        0.1         27         37
  Random Read Latency           FUSE avg      NFS avg
  ------------------------- ------------ ------------
  rand-read-4k-100M           56190.7 us   53188.2 us
  rand-read-10x1M             37337.7 us   27174.6 us
============================================================

- Extract has_open_handles_for() free function (shared by VirtualFs and poll)
- Fix missing continue when inode already removed between snapshot and write lock
- Simplify deletion guard with match expression
- Test now verifies second poll cleans up after handle release
@XciD XciD marked this pull request as ready for review March 26, 2026 12:01
XciD added 2 commits March 26, 2026 13:14
…en handles

Instead of skipping the deletion entirely (leaving the file visible by
name), unlink the pathname via unlink_one() but keep the inode as an
orphan (nlink=0). This way open handles can still read/fstat, but the
file disappears from lookup/readdir. release() cleans up the orphan.
@XciD XciD force-pushed the fix/poll-open-handles branch from a80e0a2 to 891d656 Compare March 26, 2026 12:23
@XciD XciD merged commit 3e10136 into main Mar 26, 2026
6 checks passed
@XciD XciD deleted the fix/poll-open-handles branch March 26, 2026 12:37
This was referenced Mar 26, 2026
XciD added a commit that referenced this pull request Mar 26, 2026
Bump version to 0.1.1.

## Changes since v0.1.0

### Bug fixes
- fix: prevent file descriptor exhaustion on bulk NFS deletes (#68)
- fix: schedule flush after NFS exclusive create with empty files (#71)
- fix: re-check dirty status before poll deletes inode, TOCTOU race
(#67)
- fix: prevent poll from deleting inodes with open file handles (#74)
- fix: prevent apply_commit from clobbering size/hash on generation
mismatch (#75)
- fix: serialize setattr truncate with write to prevent inode size race
(#76)
- fix: tolerate rename Phase 3 failure after remote mutation (#77)

### Performance
- perf: chunk upload_files to bound FD usage during flush (#72)
- perf: skip redundant HEAD revalidation after flush commit (#73)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant