Skip to content

make bake definitions extensible for downstream overrides #393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
crazy-max wants to merge 1 commit into
base: main
Choose a base branch
from
+103 −24
Open

make bake definitions extensible for downstream overrides #393

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ ___
* [Build](#build)
* [Requirements](#requirements)
* [Usage](#usage)
* [Remote override](#remote-override)
* [Contributing](#contributing)

## Release
Expand Down Expand Up @@ -81,6 +82,54 @@ $ PKG_REF=v0.24.0 docker buildx bake pkg-buildx-*
$ docker buildx bake --push --set *.tags=dockereng/packaging:buildx-v0.24.0 release-buildx
```

### Remote override

Downstream repositories can reuse this repository as a remote Bake definition
and extend it with a local `docker-bake.override.hcl` file instead of forking
the full `docker-bake.hcl`.

Example local override:

```hcl
variable "PKGS_EXTRA" {
default = ["my-package"]
}

variable "PKG_PLATFORMS_EXTRA" {
default = {
my-package = ["linux/amd64"]
}
}

variable "PKG_CONTEXTS_EXTRA" {
default = {
my-package = "cwd://pkg/my-package"
}
}

target "_pkg-my-package" {
args = {
PKG_NAME = "my-package"
PKG_REPO = "https://github.com/example/my-package.git"
PKG_REF = "main"
}
}
```

Example invocation from the downstream repository:

```shell
$ docker buildx bake \
-f docker-bake.hcl \
-f cwd://docker-bake.override.hcl \
"https://github.com/docker/packaging.git#main" \
pkg-my-package-debian12
```

Use the `cwd://` prefix for local files and directories when combining a local
override with a remote Bake definition. Without it, Bake resolves paths relative
to the remote context instead of the current working directory.

## Contributing

Want to contribute? Awesome! You can find information about contributing to
Expand Down
78 changes: 54 additions & 24 deletions docker-bake.hcl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,8 @@ variable "DISTROS" {
]
}

variable "PKGS" {
description = "List of packages to build from ./pkg directory. Don't forget to update _pkg-* target if you add/remove a package."
variable "PKGS_BASE" {
description = "Base list of packages built from this repo. Use PKGS_EXTRA in an override file to add downstream packages."
default = [
"buildx",
"compose",
Expand All @@ -56,6 +56,49 @@ variable "PKGS" {
]
}

variable "PKGS_EXTRA" {
description = "Additional packages to append from an override bake definition."
default = []
}

variable "PKGS" {
description = "Full list of packages to build from ./pkg directory or an override-provided package context."
default = concat(PKGS_BASE, PKGS_EXTRA)
}

variable "PKG_PLATFORMS_BASE" {
description = "Base package platform mapping. Use PKG_PLATFORMS_EXTRA in an override bake definition to add downstream packages."
default = {
# https://github.com/docker/buildx/blob/0c747263ef1426f5fa217fcdb616eddf33da6c2d/docker-bake.hcl#L156-L174
buildx = ["darwin/amd64", "darwin/arm64", "linux/amd64", "linux/arm/v6", "linux/arm/v7", "linux/arm64", "linux/ppc64le", "linux/riscv64", "linux/s390x", "windows/amd64", "windows/arm64"]
# https://github.com/docker/compose/blob/c626befee1596abcc74578cb10dd96ae1667f76f/docker-bake.hcl#L112-L124
compose = ["darwin/amd64", "darwin/arm64", "linux/amd64", "linux/arm/v6", "linux/arm/v7", "linux/arm64", "linux/ppc64le", "linux/riscv64", "linux/s390x", "windows/amd64", "windows/arm64"]
# https://github.com/containerd/containerd/blob/87742bd35f6ddc47c638a448c271b7ccf8df9010/.github/workflows/ci.yml#L145-L165
# https://github.com/containerd/containerd/blob/87742bd35f6ddc47c638a448c271b7ccf8df9010/.github/workflows/ci.yml#L135-L137
containerd = ["linux/amd64", "linux/arm/v6", "linux/arm/v7", "linux/arm64", "linux/ppc64le", "linux/s390x", "windows/amd64", "windows/arm64", "windows/arm/v7"]
# https://github.com/docker/docker-credential-helpers/blob/f9d3010165b642df37215b1be945552f2c6f0e3b/docker-bake.hcl#L56-L66
credential-helpers = ["darwin/amd64", "darwin/arm64", "linux/amd64", "linux/arm/v6", "linux/arm/v7", "linux/arm64", "linux/ppc64le", "linux/s390x", "windows/amd64"]
# https://github.com/docker/cli/blob/84038691220e7ba3329a177e4e3357b4ee0e3a52/docker-bake.hcl#L30-L42
docker-cli = ["darwin/amd64", "darwin/arm64", "linux/386", "linux/amd64", "linux/arm/v6", "linux/arm/v7", "linux/arm64", "linux/ppc64le", "linux/riscv64", "linux/s390x", "windows/amd64", "windows/arm64"]
# https://github.com/moby/moby/blob/83264918d3e1c61341511e360a7277150b914b3f/docker-bake.hcl#L82-L91
docker-engine = ["linux/amd64", "linux/arm/v6", "linux/arm/v7", "linux/arm64", "linux/ppc64le", "linux/s390x", "windows/amd64", "windows/arm64"]
# https://github.com/docker/model-runner/blob/039f7a31c0365f9161c9b9b6bb3888161d16e388/cmd/cli/Makefile#L39-L43
model = ["darwin/amd64", "darwin/arm64", "linux/amd64", "linux/arm64", "linux/arm/v7", "windows/amd64", "windows/arm64"]
# https://github.com/docker/docker-agent/blob/5b9feaabe743a5ad577f2247ed55d5dcb2678e8b/Taskfile.yml#L79
agent = ["darwin/amd64", "darwin/arm64", "linux/amd64", "linux/arm64", "windows/amd64", "windows/arm64"]
}
}

variable "PKG_PLATFORMS_EXTRA" {
description = "Additional package platform mapping to merge from an override bake definition."
default = {}
}

variable "PKG_CONTEXTS_EXTRA" {
description = "Additional package context mapping to merge from an override bake definition."
default = {}
}

variable "DISTRO_NAME" {
description = "Name of the distro."
default = null
Expand Down Expand Up @@ -589,25 +632,12 @@ target "_pkg-agent" {
# Returns the list of supported platforms for a given package.
function "pkgPlatforms" {
params = [pkg]
result = lookup({
# https://github.com/docker/buildx/blob/0c747263ef1426f5fa217fcdb616eddf33da6c2d/docker-bake.hcl#L156-L174
buildx = ["darwin/amd64", "darwin/arm64", "linux/amd64", "linux/arm/v6", "linux/arm/v7", "linux/arm64", "linux/ppc64le", "linux/riscv64", "linux/s390x", "windows/amd64", "windows/arm64"]
# https://github.com/docker/compose/blob/c626befee1596abcc74578cb10dd96ae1667f76f/docker-bake.hcl#L112-L124
compose = ["darwin/amd64", "darwin/arm64", "linux/amd64", "linux/arm/v6", "linux/arm/v7", "linux/arm64", "linux/ppc64le", "linux/riscv64", "linux/s390x", "windows/amd64", "windows/arm64"]
# https://github.com/containerd/containerd/blob/87742bd35f6ddc47c638a448c271b7ccf8df9010/.github/workflows/ci.yml#L145-L165
# https://github.com/containerd/containerd/blob/87742bd35f6ddc47c638a448c271b7ccf8df9010/.github/workflows/ci.yml#L135-L137
containerd = ["linux/amd64", "linux/arm/v6", "linux/arm/v7", "linux/arm64", "linux/ppc64le", "linux/s390x", "windows/amd64", "windows/arm64", "windows/arm/v7"]
# https://github.com/docker/docker-credential-helpers/blob/f9d3010165b642df37215b1be945552f2c6f0e3b/docker-bake.hcl#L56-L66
credential-helpers = ["darwin/amd64", "darwin/arm64", "linux/amd64", "linux/arm/v6", "linux/arm/v7", "linux/arm64", "linux/ppc64le", "linux/s390x", "windows/amd64"]
# https://github.com/docker/cli/blob/84038691220e7ba3329a177e4e3357b4ee0e3a52/docker-bake.hcl#L30-L42
docker-cli = ["darwin/amd64", "darwin/arm64", "linux/386", "linux/amd64", "linux/arm/v6", "linux/arm/v7", "linux/arm64", "linux/ppc64le", "linux/riscv64", "linux/s390x", "windows/amd64", "windows/arm64"]
# https://github.com/moby/moby/blob/83264918d3e1c61341511e360a7277150b914b3f/docker-bake.hcl#L82-L91
docker-engine = ["linux/amd64", "linux/arm/v6", "linux/arm/v7", "linux/arm64", "linux/ppc64le", "linux/s390x", "windows/amd64", "windows/arm64"]
# https://github.com/docker/model-runner/blob/039f7a31c0365f9161c9b9b6bb3888161d16e388/cmd/cli/Makefile#L39-L43
model = ["darwin/amd64", "darwin/arm64", "linux/amd64", "linux/arm64", "linux/arm/v7", "windows/amd64", "windows/arm64"]
# https://github.com/docker/docker-agent/blob/5b9feaabe743a5ad577f2247ed55d5dcb2678e8b/Taskfile.yml#L79
agent = ["darwin/amd64", "darwin/arm64", "linux/amd64", "linux/arm64", "windows/amd64", "windows/arm64"]
}, pkg, [])
result = lookup(merge(PKG_PLATFORMS_BASE, PKG_PLATFORMS_EXTRA), pkg, [])
}

function "pkgContext" {
params = [pkg]
result = lookup(PKG_CONTEXTS_EXTRA, pkg, "./pkg/${pkg}")
}

#
Expand Down Expand Up @@ -652,7 +682,7 @@ target "pkg" {
pkg = PKGS
distro = DISTROS
}
context = "./pkg/${pkg}"
context = pkgContext(pkg)
contexts = {
scripts = "./hack/scripts"
}
Expand All @@ -675,7 +705,7 @@ target "verify" {
pkg = PKGS
distro = DISTROS
}
context = "./pkg/${pkg}"
context = pkgContext(pkg)
dockerfile = "verify.Dockerfile"
contexts = {
scripts = "./hack/scripts"
Expand Down Expand Up @@ -708,7 +738,7 @@ target "metadata" {
matrix = {
pkg = PKGS
}
context = "./pkg/${pkg}"
context = pkgContext(pkg)
contexts = {
scripts = "./hack/scripts"
}
Expand Down