feat: create git bundle snapshots periodically#39
Merged
alecthomas merged 1 commit intomainfrom Jan 16, 2026
Merged
Conversation
Interestingly this doesn't seem to result in significant gains. It's also quite costly in terms of storage, so it's disabled by default. That said, it likely does reduce CPU utilisation on the cache server, as it can just stream the bundle back and not have to compute it on the fly. I was planning to automatically inject bundle-uri commands in the prototocol response, but I think given the minimal improvements I'll defer that. ``` ~/dev/cachew $ rm -rf linux-source ; time git clone http://127.0.0.1:8080/git/github.com/torvalds/linux linux-source Cloning into 'linux-source'... remote: Enumerating objects: 11276289, done. remote: Counting objects: 100% (98/98), done. remote: Compressing objects: 100% (56/56), done. remote: Total 11276289 (delta 68), reused 49 (delta 42), pack-reused 11276191 (from 1) Receiving objects: 100% (11276289/11276289), 5.65 GiB | 7.55 MiB/s, done. Resolving deltas: 100% (9153747/9153747), done. Updating files: 100% (92194/92194), done. git clone http://127.0.0.1:8080/git/github.com/torvalds/linux linux-source 486.00s user 112.07s system 67% cpu 14:50.60 total ~/dev/cachew $ rm -rf linux-source ; time git clone http://127.0.0.1:8080/git/github.com/torvalds/linux linux-source Cloning into 'linux-source'... remote: Enumerating objects: 11084152, done. remote: Counting objects: 100% (11084152/11084152), done. remote: Compressing objects: 100% (2034375/2034375), done. remote: Total 11084152 (delta 8997999), reused 11084152 (delta 8997999), pack-reused 0 (from 0) Receiving objects: 100% (11084152/11084152), 5.58 GiB | 101.10 MiB/s, done. Resolving deltas: 100% (8997999/8997999), done. Updating files: 100% (92194/92194), done. git clone http://127.0.0.1:8080/git/github.com/torvalds/linux linux-source 410.10s user 86.08s system 176% cpu 4:40.95 total ~/dev/cachew $ rm -rf linux-source ; time git clone --bundle-uri http://127.0.0.1:8080/git/github.com/torvalds/linux.git/bundle http://127.0.0.1:8080/git/github.com/torvalds/linux.git linux-source Cloning into 'linux-source'... remote: Enumerating objects: 908, done. remote: Counting objects: 100% (908/908), done. remote: Compressing objects: 100% (908/908), done. remote: Total 908 (delta 0), reused 908 (delta 0), pack-reused 0 (from 0) Receiving objects: 100% (908/908), 414.26 KiB | 17.26 MiB/s, done. Checking connectivity: 908, done. Updating files: 100% (92194/92194), done. git clone --bundle-uri linux-source 358.40s user 81.89s system 197% cpu 3:42.47 total ```
53f67e8 to
b307863
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Interestingly this doesn't seem to result in significant gains. It's also quite costly in terms of storage, so it's disabled by default. That said, it likely does reduce CPU utilisation on the cache server, as it can just stream the bundle back and not have to compute it on the fly.
I was planning to automatically inject
bundle-uricommands in the prototocol response, but I think given the minimal improvements I'll defer that.Completely uncached clone:
Using the mirror on the cache server:
Using the mirror on the cache server AND the bundle: