bugfix(baker): bake_scalar_source updates release-manifest.json (#251)#255
Merged
bugfix(baker): bake_scalar_source updates release-manifest.json (#251)#255
Conversation
Pre-#251 ``bake_scalar_source`` only pushed ``<source>.json`` and never touched ``release-manifest.json``. Pre-#239 the client reconstructed the manifest from a tree listing, so the missing write was invisible. After #239 the client reads the static manifest directly — meaning every scalar bake (today: physicallybased) silently left its source absent from the manifest, hiding it from clients. We caught this on ``gerchowl/mat-vis@v2026.04.2`` and patched prod manually (commit 63e84e2). This change is structural only — the prod manifest is already correct. Apply the same CAS retry pattern ``bake_one_per_file`` uses for the catalog + manifest commit: - Push the catalog as before (one commit, via ``push_to_hf``). - Then enter a CAS loop: fetch the current manifest + parent SHA, merge in ``{"scalar": {"complete": True}}`` under the source entry, commit with ``parent_commit=<sha>``. Retry on 412/409, exhaust at 6. Reuses ``_fetch_manifest_with_parent`` and ``_merge_manifest_for_source`` from ``hf_bake_per_file`` and ``_create_commit_with_backoff`` from ``hf_retry`` — single contract evolves across both bakers. Drops the stale "clients derive the manifest from the dataset tree" docstring line. Tests pin three properties: 1. Two commits land — catalog (one file) and manifest (one file). 2. The manifest payload carries ``{"physicallybased": {"catalog": "physicallybased.json", "tiers": {"scalar": {"complete": true}}}}``. 3. Pre-existing sources in the manifest are preserved on merge, and the manifest commit opts into ``parent_commit`` for CAS.
github-actions
Bot
added
area:testing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
bake_scalar_sourcewas writing<source>.jsonbut never touchingrelease-manifest.json. Pre-bugfix(client): read release-manifest.json directly to fix scale bug (#238) #239 the client reconstructed the manifest from a tree listing, so this was invisible. Post-bugfix(client): read release-manifest.json directly to fix scale bug (#238) #239 the client reads the static manifest directly, so every scalar bake (today: physicallybased) silently left its source absent from the manifest. We caught this ongerchowl/mat-vis@v2026.04.2and patched prod manually (commit63e84e2); this PR is structural only.bake_scalar_sourcenow enters the same CAS retry loopbake_one_per_fileuses: fetch the current manifest + parent SHA, merge in{"scalar": {"complete": true}}under the source entry, commit withparent_commit=<sha>, retry on 412/409, exhaust at 6. Reuses_fetch_manifest_with_parent,_merge_manifest_for_source, and_create_commit_with_backoff— no duplication.Test plan
tests/test_bake_scalar_source.pywith three cases:{"physicallybased": {"catalog": "physicallybased.json", "tiers": {"scalar": {"complete": true}}}}.parent_commitfor CAS.tests/test_bake_one_routing.pystill green (return dict still carriesmaterials).uv run --extra baker pytest tests/ -q --ignore=tests/e2e --deselect tests/test_version_sync.py::test_client_runtime_version_matches_pyproject→ 369 passed, 5 skipped, 1 deselected.ruff check+ruff format --checkclean on changed files.Closes #251.