Skip to content

Rewrite of the local cache component#3121

Open
bbockelm wants to merge 15 commits intoPelicanPlatform:mainfrom
bbockelm:local-cache-refactor
Open

Rewrite of the local cache component#3121
bbockelm wants to merge 15 commits intoPelicanPlatform:mainfrom
bbockelm:local-cache-refactor

Conversation

@bbockelm
Copy link
Collaborator

This amounts to approximately a complete rewrite of the local cache component.

The component now implements a wide swath of HTTP 1.1's cache protocol (include Cache-Control, ETags) and can work at the block level.

A much more thorough test suite is included, including work with ETags from the origin.

@bbockelm bbockelm force-pushed the local-cache-refactor branch from b7b0535 to 21f5693 Compare February 15, 2026 20:25
@bbockelm bbockelm linked an issue Feb 22, 2026 that may be closed by this pull request
@bbockelm bbockelm added enhancement New feature or request cache Issue relating to the cache component labels Feb 22, 2026
Most important item is the fix for director responses which allowed
reuse of a full URL, not just the URL prefix.
- Concurrent range-based downloaders could all try the HEAD in parallel
- Have modest retries on metadata update conflicts
- Do not cache file descriptors between reads on tests; allows us to
  reliably change the disk cache from outside the process.
@bbockelm bbockelm marked this pull request as ready for review February 27, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cache Issue relating to the cache component enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Overhaul the local cache

1 participant