Build Proces: Cache build.sh install dependencies

[tjsturos]

An optimization that could be done for the build process is that, during the build-node section of the build, the go and apt packages are not cached, each rebuild redownloads the updated go version and packages and reruns the apt install commands.

Being able to cache these dependencies in the build.sh script would massively speed up the build-node process.

Note: I do not believe critical it is to keep the build command inside the build.sh script-- rather the commands in the script can be moved into the Dockerfile(s) so Docker can more readily cache those commands into layers.

[hhamud]

I think converting the entire dev environment to using docker would be the way to go. build, test and serve all via docker would be a lot cleaner.

[tjsturos]

The dev process generally is all docker, although it has the task components to stub out the different options. You make your modifications, run the task build_* commands and then get a binary output which you can pass to your nodes to run. The serve part of your thesis is a bit harder to pin down as to the benefits/steps that would come from Dockerizing testing and serving vs just running ./node. Proposing to have users setup/install Docker to run nodes adds complexity to running a node because there are multiple layers now in diagnosing where somebody's node is not operating properly. Also, in general, when provisioning a device, there is really not much difference in a clean slate (if trying to create a clean operating environment for the binary to run) vs a clean OS install. Docker is FROM Ubuntu:24.04 while a normal server provision is a fresh install of the ISO. This is what the vast majority of nodes will be, and those that aren't are still left with the hassle of wrangling docker installs. Testing, should be run in a docker container, just as it can handle the build process for a clean build to use in the testing, and have all the testing dependencies installed.

…

On Friday, December 26th, 2025 at 3:38 PM, Hamza Hamud ***@***.***> wrote: hhamud left a comment [(QuilibriumNetwork/monorepo#497)](#497 (comment)) I think converting the entire dev environment to using docker would be the way to go. build, test and serve all via docker would be a lot cleaner. — Reply to this email directly, [view it on GitHub](#497 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/ANGGZJZWJDJ26JVAYGK6DA34DXIJRAVCNFSM6AAAAACPQCVVV2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMOJTGUZDENRSGM). You are receiving this because you authored the thread.Message ID: ***@***.***>

[blacks1ne]

In general, I don't mind considering all-Docker approach if we make straightforward instead of complicating thinks further, like @tjsturos mentioned.
Speaking of the Docker-build-run approach, I see the following notable points of attention:

1. How do we ensure support for MacOS? The build on MacOS is already outside Docker, not to mention the run. I suppose we could run the Linux node on MacOS within Docker, but I'm not sure about possible performance implications compared to the native binary.
2. We've long had the ambition of supporting node on Windows. With Docker on Windows running a Linux node, this seems like a valid approach.

As the all-Docker approach offers some advantages, I provided detailed feedback on #501 . It would be great to get @CassOnMars opinion on this.

[blacks1ne]

Maybe we should move the all-Docker discussion to #501 as it is counterintuitive to search for it in this issue.

[blacks1ne]

Speaking of the original point in this issue, I am not sure I fully understand it.
Reflecting on the "go and apt packages are not cached" statement, after the Docker file refactoring done by @hhamud on my side the only stage that gets rebuilt upon node code changes is build-node. All other stages are cached including go-base that installs Go and base that updates and installs packages via apt.
Focusing on the build-node stage, the only repeating aspect is the download of the go modules that represent dependencies. But those dependencies can change with the updated node source (e.g. module versions can change or new modules can be introduced). I do not immediately see how those can be cached in a stage before node source copying. Trying to download them to Go cache with go mod download still depends on copying node source in the previous step.