From e198d52e2434e16124528157540471f577126c05 Mon Sep 17 00:00:00 2001 From: Norio Nomura Date: Mon, 18 Aug 2025 18:33:34 +0900 Subject: [PATCH 1/2] feature: use Rosetta AOT Caching with CDI This change introduces device configuration to enable Rosetta AOT Caching in Docker VMs. - Modify Rosetta Caching Options from Abstract Socket to Unix Domain Socket: Unix Domain Socket can be mounted within a container using the Container Device Interface (CDI) mechanism. This requires merging the following pull request: https://github.com/Code-Hex/vz/pull/195. - Register Rosettad AOT Caching Daemon as a service: - `/etc/systemd/system/rosettad.service` on systemd - `/etc/init.d/rosettad` on OpenRC - Add "lima.io/rosetta=cached" device specification to `{~/.config,/etc}/cdi/rosetta.yaml` see: https://github.com/cncf-tags/container-device-interface/blob/main/SPEC.md - Add `{~/.config,/etc}/docker/daemon.json` to `docker{,-rootful}.yaml` - `.features.cdi = true` to enable CDI - Add `enable_cdi = true` to `{~/.config,/etc}/containerd/config.toml` To enable Rosetta AOT Caching in docker, use `--device=lima.io/rosetta=cached`. see: https://docs.docker.com/build/building/cdi/ Signed-off-by: Norio Nomura # Conflicts: # templates/docker-rootful.yaml --- .../boot/40-install-containerd.sh | 4 + pkg/driver/vz/boot/05-rosetta-volume.sh | 78 +++++++++++++++++++ .../vz/rosetta_directory_share_arm64.go | 2 +- templates/docker-rootful.yaml | 4 +- templates/docker.yaml | 4 +- 5 files changed, 89 insertions(+), 3 deletions(-) diff --git a/pkg/cidata/cidata.TEMPLATE.d/boot/40-install-containerd.sh b/pkg/cidata/cidata.TEMPLATE.d/boot/40-install-containerd.sh index aa7843b2c28..0ab8e364de5 100644 --- a/pkg/cidata/cidata.TEMPLATE.d/boot/40-install-containerd.sh +++ b/pkg/cidata/cidata.TEMPLATE.d/boot/40-install-containerd.sh @@ -45,6 +45,8 @@ if [ "${LIMA_CIDATA_CONTAINERD_SYSTEM}" = 1 ]; then mkdir -p /etc/containerd /etc/buildkit cat >"/etc/containerd/config.toml" <"${LIMA_CIDATA_HOME}/.config/containerd/config.toml" </etc/init.d/rosettad </etc/systemd/system/rosettad.service </etc/cdi/rosetta.yaml < Date: Sat, 30 Aug 2025 15:01:21 +0900 Subject: [PATCH 2/2] Add documentation about Rosetta AOT Caching Signed-off-by: Norio Nomura --- templates/docker-rootful.yaml | 17 +++++++ templates/docker.yaml | 17 +++++++ website/content/en/docs/config/multi-arch.md | 49 ++++++++++++++++++++ 3 files changed, 83 insertions(+) diff --git a/templates/docker-rootful.yaml b/templates/docker-rootful.yaml index 4d948eb253e..3061d77fc85 100644 --- a/templates/docker-rootful.yaml +++ b/templates/docker-rootful.yaml @@ -69,5 +69,22 @@ message: | docker context use lima-{{.Name}} docker run hello-world ------ + {{- if .Instance.Config.VMOpts.VZ.Rosetta.Enabled}} + Rosetta is enabled in this VM, so you can run x86_64 containers on Apple Silicon. + You can use Rosetta AOT Caching with the CDI spec: + - To run a container, add `--device=lima-vm.io/rosetta=cached` to your `docker run` command: + ------ + docker run --platform=linux/amd64 --device=lima-vm.io/rosetta=cached ... + ------ + - To build an image, add `# syntax=docker/dockerfile:1-labs` at the top of your Dockerfile, + and use `--device=lima-vm.io/rosetta=cached` in the `RUN` command: + ------ + # syntax=docker/dockerfile:1-labs + FROM ... + ... + RUN --device=lima-vm.io/rosetta=cached + ------ + See: https://lima-vm.io/docs/config/multi-arch/#rosetta-aot-caching + {{- end}} param: containerdSnapshotter: false diff --git a/templates/docker.yaml b/templates/docker.yaml index 669357374ba..de356d48cb2 100644 --- a/templates/docker.yaml +++ b/templates/docker.yaml @@ -76,5 +76,22 @@ message: | docker context use lima-{{.Name}} docker run hello-world ------ + {{- if .Instance.Config.VMOpts.VZ.Rosetta.Enabled}} + Rosetta is enabled in this VM, so you can run x86_64 containers on Apple Silicon. + You can use Rosetta AOT Caching with the CDI spec: + - To run a container, add `--device=lima-vm.io/rosetta=cached` to your `docker run` command: + ------ + docker run --platform=linux/amd64 --device=lima-vm.io/rosetta=cached ... + ------ + - To build an image, add `# syntax=docker/dockerfile:1-labs` at the top of your Dockerfile, + and use `--device=lima-vm.io/rosetta=cached` in the `RUN` command: + ------ + # syntax=docker/dockerfile:1-labs + FROM ... + ... + RUN --device=lima-vm.io/rosetta=cached + ------ + See: https://lima-vm.io/docs/config/multi-arch/#rosetta-aot-caching + {{- end}} param: containerdSnapshotter: false diff --git a/website/content/en/docs/config/multi-arch.md b/website/content/en/docs/config/multi-arch.md index ee77f07228f..bbe90ede19e 100644 --- a/website/content/en/docs/config/multi-arch.md +++ b/website/content/en/docs/config/multi-arch.md @@ -86,3 +86,52 @@ rosetta: ``` {{% /tab %}} {{< /tabpane >}} + +### [Enable Rosetta AOT Caching with CDI spec](#rosetta-aot-caching) +| ⚡ Requirement | Lima >= 2.0, macOS >= 14.0, ARM | +|-------------------|----------------------------------| + +Rosetta AOT Caching speeds up containers by saving translated binaries, so they don't need to be translated again. +Learn more: [WWDC2023 video](https://developer.apple.com/videos/play/wwdc2023/10007/?time=721) + +**How to use Rosetta AOT Caching:** + +- **Run a container:** + Add `--device=lima-vm.io/rosetta=cached` to your `docker run` command: + ```bash + docker run --platform=linux/amd64 --device=lima-vm.io/rosetta=cached ... + ``` + +- **Build an image:** + Add `# syntax=docker/dockerfile:1-labs` at the top of your Dockerfile to enable the `--device` option. + Use `--device=lima-vm.io/rosetta=cached` in your `RUN` command: + ```Dockerfile + # syntax=docker/dockerfile:1-labs + FROM ... + ... + RUN --device=lima-vm.io/rosetta=cached + ``` + +- **Check if caching works:** + Look for cache files in the VM: + ```bash + limactl shell {{.Name}} ls -la /var/cache/rosettad + docker run --platform linux/amd64 --device=lima-vm.io/rosetta=cached ubuntu echo hello + limactl shell {{.Name}} ls -la /var/cache/rosettad + # You should see *.aotcache files here + ``` + +- **Check if Docker recognizes the CDI device:** + Look for CDI info in the output of `docker info`: + ```console + docker info + ... + CDI spec directories: + /etc/cdi + /var/run/cdi + Discovered Devices: + cdi: lima-vm.io/rosetta=cached + ``` + +- **Learn more about CDI:** + [CDI spec documentation](https://github.com/cncf-tags/container-device-interface/blob/main/SPEC.md)