Skip to content

[ENHANCEMENT] support folder quota #22

@Vicente-Cheng

Description

@Vicente-Cheng

Folder quota would be a key feature to support k8s volumes.


Implementation Plan

Context

Arctic Wolf NFS server needs folder quota support for Kubernetes PVC size enforcement. Each PVC maps to a first-level subdirectory under the export root, and the quota limits the total byte usage of that directory.

Design Decisions

  • Granularity: First-level subdirectory quota (one per PVC)
  • Quota dimension: Bytes only (file/inode count is reported from the real filesystem but not enforced)
  • Calculation: Tracking-based (in-memory counters + redb persistence)
  • Error code: NFS3ERR_DQUOT (69)
  • Persistence: redb embedded KV store (crash-safe ACID, zero external dependencies, pure Rust)
  • Startup strategy: Trust stored values, start serving immediately, reconcile in background
  • FSSTAT: Bytes come from quota (or libc::statvfs when no quota); files always come from libc::statvfs
  • Tracked operations: write, remove, setattr_size (truncate down only), rename (cross-quota-dir only)
    • Not tracked: create, mkdir, rmdir, symlink, mknod, setattr_size extend (sparse file, write will catch real usage)

Implementation Stages

Each stage is an independent commit that builds and passes tests on its own.


Stage 1: Config infrastructure

Commit: config: add quota configuration and size parsing helper

  • Cargo.toml: add redb = \"3\" dependency
  • src/config.rs:
    • Add QuotaConfig struct (enabled: bool, db_path: PathBuf)
    • Add pub quota: QuotaConfig to Config
    • Add parse_size() helper (supports B, KB, MB, GB, TB)
    • Unit tests for parse_size variants and TOML parsing

Verification: make build, make test


Stage 2: QuotaManager core module

Commit: fsal: add QuotaManager with redb persistence

  • src/fsal/quota.rs (new):
    • QuotaManager struct (redb database + in-memory cache)
    • QuotaEntry struct (limit, usage)
    • redb table definition (key = first-level dir name, value = (limit_bytes, usage_bytes))
    • Methods: new, resolve_quota_dir, check_quota, add_usage, sub_usage, set_quota, remove_quota, get_quota_info
    • Unit tests: CRUD, redb persistence across reopen, boundary checks
  • src/fsal/mod.rs: add pub mod quota;

Verification: make test; redb file can be created and reopened


Stage 3: statvfs trait method (without quota wiring yet)

Commit: fsal: add statvfs trait method with real filesystem stats

  • src/fsal/mod.rs:
    • Add FsStats struct (bytes + files fields)
    • Add async fn statvfs() to Filesystem trait
  • src/fsal/local/mod.rs: implement statvfs() using libc::statvfs
  • src/nfs/fsstat.rs: replace hardcoded values with filesystem.statvfs()
  • Unit tests

Verification: FSSTAT returns real filesystem stats; df shows accurate values


Stage 4: Wire QuotaManager into LocalFilesystem

Commit: fsal: integrate QuotaManager into LocalFilesystem

  • src/fsal/local/mod.rs:
    • Add quota_manager: Option<QuotaManager> field
    • new() accepts Option<&QuotaConfig>
    • Update statvfs(): for paths inside a quota directory, return quota-based bytes with real file counts; fall back to libc::statvfs otherwise
  • src/fsal/mod.rs: BackendConfig carries optional quota config
  • src/main.rs: pass &config.quota to the backend
  • Tests

Verification: Server starts and loads the quota DB; FSSTAT inside a quota directory returns quota-based values


Stage 5: Quota enforcement + NFS error mapping

Commit: fsal: enforce folder quota on write/remove/truncate/rename

  • src/fsal/local/mod.rs: add quota checks and usage updates:
    • write(): check max(0, offset + len - current_size) before write; add actual delta after
    • remove(): capture file size before delete; subtract after
    • setattr_size(): if truncating down, subtract the delta (extend is not tracked)
    • rename(): if crossing quota directories, check target and transfer usage
  • NFS handler error mapping (\"Quota exceeded\"NFS3ERR_DQUOT):
    • src/nfs/write.rs
    • src/nfs/setattr.rs
    • src/nfs/rename.rs
  • Unit tests: over-limit rejection, space freeing, cross-quota move

Verification: make test; manual test confirms DQUOT is returned on over-limit writes


Stage 6: Background reconciliation

Commit: fsal: add background quota reconciliation task

  • src/fsal/quota.rs: add scan_and_reconcile() (per directory) and reconcile_all()
  • src/fsal/local/mod.rs: add start_quota_reconciliation() that spawns a background task
  • src/main.rs: call after server startup, without blocking service
  • Tests

Verification: Server logs show reconciliation progress; out-of-band filesystem changes are corrected after the next scan


Stage 7: Integration tests

Commit: test: add NFS folder quota integration tests

  • tests/test_nfs_quota.py:
    • WRITE up to the limit → assert NFS3ERR_DQUOT (69)
    • REMOVE a file → subsequent WRITE succeeds
    • FSSTAT returns quota-based byte values
    • Truncate down frees quota
  • Update tests/configs/ if a quota DB path is needed

Verification: make nfstest


Critical Files

  • Cargo.toml — redb dependency
  • src/config.rs — QuotaConfig, parse_size
  • src/fsal/quota.rs (new) — QuotaManager + redb
  • src/fsal/mod.rs — FsStats, statvfs trait
  • src/fsal/local/mod.rs — quota enforcement
  • src/nfs/fsstat.rs — dynamic space reporting
  • src/nfs/write.rs, src/nfs/setattr.rs, src/nfs/rename.rs — DQUOT error mapping
  • src/main.rs — config wiring, background reconciliation

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions