Recommended Caching method

[KieranFleckney]

Question

Hello,

I did ask a similar question in Discussions but not sure anyone looks there.

What is the current recommended caching/storing cache between runs. I see #49 talks about removing some of the caching docs. But currently there is not helpful docs on this. Is it the case of using buildkit caching like --cache-to and --cache-from? If so how do even do that with earthbuild? or is there another way. Because if you can't keep the cache it defeats one of the main benefits to a tool like this when running in a CI pipeline like github action, without running selfhosted runners.

I guess just little confused on how to best approach it.

[janishorsts]

Sorry for the delay in responding to your request, @KieranFleckney.

Caching is on our radar and is fundamental to @EarthBuild. In our weekly meetings, we discuss it. However, right now, we are working through "quick wins".

At the moment, there is no easy/nice way to handle caching in a non-self-hosted environment, e.g. GHA.

@kmannislands, anything to add? What was the setting you mentioned in the weekly meeting recently? We need Gemini to summarise our meetings. 😄

The caching relates to (depends on) EarthBuild/buildkit#5, which is a massive challenge.

[janishorsts]

I do not recommend this, but I you like tinkering like me. This caches the buildkit volume used by @EarthBuild in https://github.com/translate-agent/intl/blob/main/.github/workflows/check.yaml.

       # ...
       - name: Earth Cache
         uses: actions/cache@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5
         id: earth-cache
         with:
           path: earth-cache.tar.zst
           key: earth-cache-${{ matrix.go-version }}-${{ github.sha }}
           restore-keys: |
             earth-cache-${{ matrix.go-version }}-
       - name: Uncompress Earth Cache
         run: |
           if [ -f "earth-cache.tar.zst" ]; then
             echo "Cache file found. Restoring..."
             sudo mkdir -p /var/lib/docker/volumes/intl-cache/_data
             sudo tar -xf earth-cache.tar.zst -C /var/lib/docker/volumes/intl-cache/_data
           else
             echo "Cache file not found. Skipping restore."
           fi
       # ...
       # last steps
       - name: Stop Buildkit
         run: docker stop intl-buildkitd
       - name: Compress Earth Cache
         if: steps.earth-cache.outputs.cache-hit != 'true'
         run: sudo tar -c --use-compress-program="zstd -T0" -f earth-cache.tar.zst -C /var/lib/docker/volumes/intl-cache/_data .

[kmannislands]

At $day_job we've discovered and enabled usage of a feature: "remove layer caching" in our image registry. This would suit GHA well since built-in GHCR could be used as the primary buildkit cache.

This was documented in earthly v0.7 that was pulled from the docs by v0.8. I'm speculating here but that feature decreased the motivation for OSS users to switch to earthly's paid offering when this was a commercial software so it may have been removed from docs for business reasons? The feature still seems to work.

Interestingly, Amazon's announcement of support for this buildkit feature in ECR mentions earthly explicitly so at some point this was better-known.

I've tried this for earthbuild's own CI here: #328

The idea is that if we can get that working well then we can make it a recommended approach for caching in Earthbuild and update the docs. TBD.

[kmannislands]

As @janishorsts alluded to, the supported buildkit caching backends are highly related to our upstream buildkit version so when we bring that up to date we should have more options like first-party support gha in addition to the registry mode I mentioned above ^

[KieranFleckney]

@kmannislands Thank you for the insight, I will look into these