Skip to content

feat(client): ETag-aware manifest cache + immutable-tag policy (#258)#259

Merged
gerchowl merged 1 commit intodevfrom
feature/258-etag-manifest-cache
Apr 30, 2026
Merged

feat(client): ETag-aware manifest cache + immutable-tag policy (#258)#259
gerchowl merged 1 commit intodevfrom
feature/258-etag-manifest-cache

Conversation

@gerchowl
Copy link
Copy Markdown
Contributor

Summary

Issue #258 — replaces the never-invalidated manifest disk cache with an ETag-aware conditional GET across all four reference clients, plus formalises the immutable-tag policy in README + CHANGELOG.

  • Cache holds a sibling .manifest.etag; first .manifest access in a client lifecycle sends If-None-Match. 304 → cached body served without refetch; 200 → both body and ETag replaced atomically.
  • Defensive cold-start when origin omits an ETag (e.g. air-gapped mirror): body cached, no etag file written, next lifecycle refetches unconditionally rather than risk a stale-etag deadlock.
  • Immutable-tag note in README's Versioning section + CHANGELOG ### Notes subsection: once a CalVer tag is published the data at that revision will not change. New data → new CalVer.
  • All four client packages aligned to 0.6.3 (patch — backward-compat behavior change; existing pinned-tag callers see one extra HTTP HEAD per session worst case).

Mechanism summary across clients

Client Storage Conditional GET
Python (clients/python/src/mat_vis_client/client.py) $cache_dir/<tag>/.manifest.{json,etag} _get_with_etag(url, etag=…) via stdlib urllib.request, mirrors _get's retry/backoff envelope; 304 short-circuits before the rate-limit handler
JS (clients/js/mat-vis-client.mjs) Filesystem in Node ($MAT_VIS_CACHE/<tag>/.manifest.{json,etag}); in-memory only in browser (no IndexedDB to avoid a runtime dep) fetch(url, { headers: { 'If-None-Match': etag } }), branches on resp.status === 304
Rust (clients/rust/src/main.rs) $MAT_VIS_CACHE/<tag>/.manifest.{json,etag} (skipped when no HOME resolves) reqwest::blocking::Client.get(url).header(IF_NONE_MATCH, etag), branches on resp.status() == NOT_MODIFIED
Shell (clients/mat-vis.sh) $CACHE/$TAG/.manifest.{json,etag} curl -H "If-None-Match: $etag" -w '%{http_code}', branches on 304 vs 200 (and treats 000 from file:// as 200 so structural tests still pass)

Policy paragraphs

README.md (under ## Versioning)

Release tags are immutable. Once a CalVer tag is published (e.g.
v2026.04.2), the data at that revision will not change — bytes pinned
to a tag stay pinned. New upstream snapshots, fixes, or rebakes ship as
a new CalVer tag, never as an in-place rewrite of an existing one. This
contract is what lets clients use cheap If-None-Match conditional GETs
on the manifest (#258) and trust pinned-tag deployments across long
intervals without re-validating every byte.

CHANGELOG.md (under ## mat-vis-client 0.6.3### Notes)

Release tags are immutable. Once a CalVer tag is published (e.g.
v2026.04.2), the data at that revision will not change. New
upstream snapshots, fixes, or rebakes ship as a new CalVer tag, never
as an in-place rewrite of an existing one. This contract is what lets
the new ETag cache trust 304 responses on pinned tags — the manifest
bytes simply cannot drift under a pinned tag in the first place.

Test plan

  • Python: 5 new tests in TestManifestEtagCache (cold start, in-process memo, fresh-process 304, fresh-process 200 mutation, no-ETag-from-server). Existing manifest tests migrated from _get_json to _get_with_etag patches. MAT_VIS_SKIP_LIVE_TESTS=1 pytest clients/python/test_client.py clients/python/tests/ tests/ -q605 passed, 30 skipped.
  • JS: existing structural tests upgraded to expose text() + headers.get('etag') on the fetch stub; all 10 tests pass.
  • Rust: cargo test21 passed.
  • Shell: bash clients/test_client.sh13 passed.
  • Live regression for bugfix(test): bump physicallybased proof + LIVE_TAG default to v2026.04.2 (#250 follow-up) #252 / brittle physicallybased fetch: MAT_VIS_LIVE_TESTS=1 pytest clients/python/test_client.py::test_proof_phase_2_fetch_physicallybased_index_from_hfpassed against gerchowl/mat-vis@v2026.04.2.
  • Live default-tag end-to-end: TestLiveDefaultTag2 passed.

Out of scope

Replace the never-invalidated manifest disk cache with one conditional
GET per client lifecycle across all four reference clients (Python, JS,
Rust, shell). Cache holds a sibling .manifest.etag; on the first
.manifest access we send If-None-Match and serve the cached body on
304, refetch on 200. Defensive cold-start when the origin omits an
ETag — body cached, no etag file written, next lifecycle refetches
unconditionally.

Also formalises the immutable-tag policy in README + CHANGELOG: once a
CalVer tag is published (e.g. v2026.04.2) the data at that revision
will not change. New data → new CalVer. This contract is what lets the
ETag cache trust 304 responses on pinned tags.

All four client packages aligned to 0.6.3.
@github-actions github-actions Bot added area:docs Documentation, README, guides area:testing Test infrastructure, BATS, pytest area:client Python/JS/Rust client packages labels Apr 30, 2026
@gerchowl gerchowl enabled auto-merge (squash) April 30, 2026 16:45
@gerchowl gerchowl merged commit 246d455 into dev Apr 30, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:client Python/JS/Rust client packages area:docs Documentation, README, guides area:testing Test infrastructure, BATS, pytest

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant