BUILD-10724: Import GitHub cache to S3 (migration fallback)#45
BUILD-10724: Import GitHub cache to S3 (migration fallback)#45julien-carsique-sonarsource wants to merge 1 commit intomasterfrom
Conversation
…tion mode)
When the S3 backend is used and the S3 cache misses, automatically attempt
to restore the cache from GitHub using the original unprefixed key. The S3
post-job step will then save the restored content to S3, pre-provisioning it
for subsequent runs.
The feature is enabled by default. Resolution order to disable it:
1. Action input `import-github-cache: 'false'`
2. Environment variable `CACHE_IMPORT_GITHUB=false`
(can be set from a repository variable via ${{ vars.CACHE_IMPORT_GITHUB }})
3. Default: true
`fail-on-cache-miss` and `lookup-only` are correctly propagated to the
GitHub fallback step. When `fail-on-cache-miss` is set and import mode is
active, failure is deferred until both S3 and GitHub have been tried.
Also adds `.github/workflows/check-cache-migration.yml`: a manually-triggered
workflow that compares GitHub cache entries to S3 objects across target branches
(main, master, branch-*, dogfood-on-*, feature/long/*), ignoring transient
keys (build-number-*, mise-*). When 100% of entries are found in S3, it
automatically sets the CACHE_IMPORT_GITHUB=false repository variable to
disable the import fallback.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SummaryImplements a GitHub-to-S3 cache migration fallback for the The key behavior: S3 miss → (if import enabled) try GitHub → persist to S3 on post-job save. The What reviewers should knowStart with
Then review the migration workflow (
Non-obvious decisions:
|
SonarQube reviewer guide
|
SonarQube reviewer guide
|
There was a problem hiding this comment.
Conclusion: The migration logic is sound and fits the existing patterns well, but there is one security issue that needs fixing before merge.
SonarCloud recommendations: Even though action.yml:191 is not failing the quality gate, I strongly recommend fixing it because ${{ inputs.key }} is interpolated directly into the shell run: block inside double quotes — a key value containing a " character can break out of the string and execute arbitrary shell commands. Fix by passing the key through an environment variable:
env:
CACHE_KEY: ${{ inputs.key }}
run: |
echo "::error::Cache miss: no cache found in S3 or GitHub for key '${CACHE_KEY}'"
exit 1




Summary
When migrating from GitHub cache to S3, the S3 bucket starts empty. This causes all runners to re-download dependencies from scratch until their first S3 cache save completes. This PR addresses that by automatically importing existing GitHub cache entries into S3 during the migration window.
Changes
action.yml— newimport-github-cacheinput + migration stepsNew input:
import-github-cache(default: enabled)Resolution order (mirrors the existing
CACHE_BACKEND/backendpattern):import-github-cacheCACHE_IMPORT_GITHUB(can be set from a repo variable via${{ vars.CACHE_IMPORT_GITHUB }})trueNew steps (S3 path only):
actions/cache/restorewith the original unprefixed key when S3 misses and import mode is active. The S3 post-job save then persists the restored content to S3.fail-on-cache-miss: trueis still respected.fail-on-cache-missandlookup-onlyare correctly propagated to the GitHub fallback step..github/workflows/check-cache-migration.yml— migration completion detectorManually triggered (
workflow_dispatch) workflow to determine when the migration is complete and automatically opt out of the import fallback.It:
main,master,branch-*,dogfood-on-*,feature/long/*) and excluding transient keys (build-number-*,mise-*){ref}/{key}in S3CACHE_IMPORT_GITHUB=false, disabling the fallback automaticallyBehaviour matrix
fail-on-cache-missTesting
Dogfood PR: SonarSource/sonar-dummy-js#125
Jira
BUILD-10724 — child of BUILD-10684