feature: "yq" provision mode #3892

norio-nomura · 2025-08-22T04:07:11Z

based on #3908

# `yq` creates or edits a file in the guest filesystem by using `${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq`.
# `${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq` has the same functionality as https://github.com/mikefarah/yq.
# The file specified by `path` will be updated with the specified `expression` using `yq --inplace`.
# If the file does not exist, it will be created using `yq --null-input`.
# `format` is optional and defaults to "auto", which is passed to `yq` by `--input-format <format>`, `--output-format <format>`.
# One of the following is expected to work:
#   "auto", "a", "ini", "i", "json", "j", "props", "p", "xml", "x", "yaml", "y"
# `yq` v4.47.1 can write .ini, .json, .properties, .xml, .yaml, and .yml files.
# See https://github.com/mikefarah/yq for more info.
# Any missing directories will be created as needed using `mkdir -p`.
# The file permissions will be set to the specified value using `chmod`.
# The file and directory creation will be performed with the specified user privileges.
# If the existing file is not writable by the specified user, the operation will fail.
# `path` and `expression` are required.
# `user` and `permissions` are optional. Defaults to "root" and 644.
- mode: yq
  path: "{{.Home}}/.config/docker/daemon.json"
  expression: ".features.containerd-snapshotter = {{.Param.containerdSnapshotter}}"
  format: auto
  user: "{{.User}}"
  permissions: 644

updated:

based on Add yq subcommands to limactl and lima-guestagent #3908
dropped provisionTool

AkihiroSuda · 2025-08-22T04:45:00Z

Why not just apt-get yq in a shell script?

jandubois · 2025-08-22T04:48:00Z

I think this is too specialized to be built into Lima given that you can already do this with a regular provisioning script:

provision:
- mode: user
  script: |
    if ! command -v yq >/dev/null 2>&1; then
      apt-get update
      apt install -y yq
    fi
    mkdir -p ...
    touch ...
    yq ...

norio-nomura · 2025-08-22T04:50:00Z

Why not just apt-get yq in a shell script?

Because yq installed by apt-get is https://github.com/kislyuk/yq
This PR installs https://github.com/mikefarah/yq

jandubois · 2025-08-22T04:53:27Z

Because yq installed by apt-get is https://github.com/kislyuk/yq

Ok, that is unfortunate. I was wondering why it was dragging in Python.

But yq is a standalone binary, so you can also curl/wget it from the GitHub release page.

jandubois · 2025-08-22T04:56:46Z

If the guestagent was already using yqlib, then I would be sympathetic to adding a lima-guestagent yq subcommand for convenience, but afaict only limactl uses yqlib.

And we would rather want to bring down the size of the guestagents, so I don't think we should be doing that either.

AkihiroSuda · 2025-08-22T05:01:18Z

I was wondering why it was dragging in Python.

Python implementation: 2017 kislyuk/yq@ce72e72
- Older Python implementation: 2014 abesto/yq@efbddc9
Go implementation: 2018 mikefarah/yq@f2e10f2

Dragging in the Python implementation makes a sense, as it seems more "authentic" version

jandubois · 2025-08-22T05:13:24Z

Dragging in the Python implementation makes a sense, as it seems more "authentic" version

Fair enough. Unfortunately on homebrew yq is the standalone Go version, creating a bit of a mess. 😞

Homebrew/homebrew-core#29529

jandubois · 2025-08-22T06:33:20Z

If we want to support adding large local files to the cidata volume, then I would suggest the way to do this would be to extend the data provisioning mode. I did try this:

base: template://docker
provision:
- mode: data
  path: /usr/local/bin/yq
  file: ~/Downloads/yq_linux_arm
  permissions: 755

It didn't work because of the tilde path (I just created a PR to fix that), but also because template embedding would attempt to include the file as a base64 encoded string, which exceeds the (somewhat arbitrary) max size of 4MB we have for lima.yaml.

I still think that once created a Lima instance should not reference other local files outside the instance directory. We could get rid of the file size limit (not a good idea, imo), or we could have some mechanism that files would be copied into a cache directory inside the instance directory, and the file reference would be rewritten to point to it.

I don't think the use case is particularly strong though when you can also just fetch the file from GitHub in a provisioning script. We do this with all manner of other prerequisites as well. But I can see the utility when you want to bundle something that is not available for download.

norio-nomura · 2025-08-22T06:40:07Z

This PR adds --provision-tool yq=<local path to executable> option to hostagent to install the executable as provision/tool/yq in cidata.

norio-nomura · 2025-08-22T06:50:43Z

provisionTool:
  yq:
  - location: "~/Downloads/yq_linux_amd64"
    arch: "x86_64"
    digest: "sha256:..."

The example above is from .containerd.archives, but it doesn’t support ~ expansion.

afbjorklund · 2025-08-22T10:09:44Z

@norio-nomura : I think you should add an issue for what you are trying to accomplish here, before diving straight into the implementation and the PR? My gut feeling is that this will meet the same fate as the "ansible" provisioning.

jandubois · 2025-08-22T18:43:38Z

I know this is just an example, but I just realized that you are editing a JSON file, so you can just use jq (which is already installed in Ubuntu).

jandubois · 2025-08-22T19:29:57Z

snap.tmpl using built-in jq:

base: template://docker
provision:
- mode: user
  script: |
    set -eux -o pipefail
    DAEMON="{{.Home}}/.config/docker/daemon.json"
    mkdir -p "$(dirname "$DAEMON")"
    [[ ! -e $DAEMON ]] && echo "{}" > "$DAEMON"
    chmod 644 "$DAEMON"
    EXPR='.features["containerd-snapshotter"] = {{.Param.containerdSnapshotter}}'
    jq "$EXPR" "$DAEMON" >"$DAEMON.new"
    mv "$DAEMON.new" "$DAEMON"
param:
  containerdSnapshotter: true

Seems to work fine:

❯ limactl start --yes --name snap ./snap.tmpl

❯ limactl shell snap sh -c "cat ~/.config/docker/daemon.json"
{
  "features": {
    "containerd-snapshotter": true
  }
}

jandubois · 2025-08-22T19:35:18Z

Unrelated, but I also think we should just always use the containerdSnapshotter in the docker template. I'm not aware of any reason to use the legacy one.

Even Docker Desktop has been using it by default for over a year now for all new installations. It is not considered experimental anymore.

norio-nomura · 2025-08-24T04:43:54Z

snap.tmpl using built-in jq:

jq is not built-in on Ubuntu-Minimal that I'm using.

Seems to work fine:

Yup, I also have some lima.yaml files that contain similar provision scripts.

I want to make those tasks much simpler like this PR does.

norio-nomura · 2025-08-24T05:33:33Z

@norio-nomura : I think you should add an issue for what you are trying to accomplish here, before diving straight into the implementation and the PR? My gut feeling is that this will meet the same fate as the "ansible" provisioning.

If I make an issue before I write a PR, I expect to receive an answer like #3892 (comment) It came.

Being able to make limactl with this function at my local Mac is more valuable to me than this PR is merged. I just prioritized the time to realize this function rather than the time spent communicating with the issue.

norio-nomura · 2025-08-24T07:03:25Z

Added format field. Now, it can manipulate systemd's unit files like the following:

- mode: yq
  path: /etc/systemd/system/docker.service.d/override.conf
  format: ini
  expression: .Socket.SocketUser="{{.User}}"

afbjorklund · 2025-08-25T18:11:36Z

Being able to make limactl with this function at my local Mac is more valuable to me than this PR is merged. I just prioritized the time to realize this function rather than the time spent communicating with the issue.

That is OK, but it also runs into the risk of 1) not realizing there is a simpler way to accomplish the same thing and 2) risking that the PR is never merged - or maybe merged and then deprecated and removed again, like Ansible

jandubois · 2025-08-25T18:25:30Z

I like the provision.mode: yq feature, but think the provisionTool design is too specialized, and would have to either piggy-back on the provision.mode: data mechanism or need to be hardcoded into Lima like the nerdctl archives.

I think a better approach would be to bundle yq into our existing binaries. That way we can re-use all the bundled runtime library code (and even yqlib in the case of limactl), and don't have to worry about versions/locations etc.

I've created #3908 to show how simple this is to implement. What do you think about using that approach @norio-nomura?

norio-nomura · 2025-08-25T23:59:01Z

What do you think about using that approach

~~It was a method that was initially considered and rejected as "unacceptable to the community". If it is accepted, there is no problem.~~
That's how I first considered and rejected it in anticipation that "it doesn't seem acceptable to the community". If my prediction is wrong and acceptable, I will not object to using that method.

The provisionTool is just a little more versatile to the function that bundles yq, and it is not the main purpose, so it doesn't matter if it's gone.

norio-nomura · 2025-08-26T00:09:34Z

I will separate the provisionTool related code to a separate commit.

norio-nomura · 2025-08-26T01:56:49Z

updated:

based on Add yq subcommands to limactl and lima-guestagent #3908
dropped provisionTool

```yaml # `yq` creates or edits a file in the guest filesystem by using `${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq`. # `${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq` has the same functionality as https://github.com/mikefarah/yq. # The file specified by `path` will be updated with the specified `expression` using `yq --inplace`. # If the file does not exist, it will be created using `yq --null-input`. # `format` is optional and defaults to "auto", which is passed to `yq` by `--input-format <format>`, `--output-format <format>`. # One of the following is expected to work: # "auto", "a", "ini", "i", "json", "j", "props", "p", "xml", "x", "yaml", "y" # `yq` v4.47.1 can write .ini, .json, .properties, .xml, .yaml, and .yml files. # See https://github.com/mikefarah/yq for more info. # Any missing directories will be created as needed using `mkdir -p`. # The file permissions will be set to the specified value using `chmod`. # The file and directory creation will be performed with the specified user privileges. # If the existing file is not writable by the specified user, the operation will fail. # `path` and `expression` are required. # `user` and `permissions` are optional. Defaults to "root" and 644. - mode: yq path: "{{.Home}}/.config/docker/daemon.json" expression: ".features.containerd-snapshotter = {{.Param.containerdSnapshotter}}" format: auto user: "{{.User}}" permissions: 644 Signed-off-by: Norio Nomura <norio.nomura@gmail.com>

jandubois · 2025-08-27T19:38:15Z

hack/test-templates.sh

@@ -57,6 +57,7 @@ declare -A CHECKS=(
 	["mount-path-with-spaces"]=""
 	["provision-data"]=""
 	["param-env-variables"]=""
+	["provision-yq"]=""


This is a tiny nitpick, so please ignore unless there are other changes needed:

I would keep the provision-data and provision-yq tests close together in the source because they are related (both provisioning tests).

jandubois · 2025-08-27T19:42:26Z

pkg/cidata/cidata.TEMPLATE.d/boot.sh

@@ -77,6 +77,64 @@ if [ -d "${LIMA_CIDATA_MNT}"/provision.data ]; then
 	done
 fi

+if [ -d "${LIMA_CIDATA_MNT}"/provision.yq ]; then
+	yq="${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq"
+	if ${yq} --version >/dev/null; then


I don't think this test is needed since yq is built into the guest agent.

jandubois · 2025-08-27T19:44:28Z

pkg/cidata/cidata.TEMPLATE.d/boot.sh

+if [ -d "${LIMA_CIDATA_MNT}"/provision.yq ]; then
+	yq="${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq"
+	if ${yq} --version >/dev/null; then
+		for expression in "${LIMA_CIDATA_MNT}"/provision.yq/*; do


I find expression misleading, as it really is a filename. All the other provisioning loops use f, so I think this should use the same:

Suggested change

for expression in "${LIMA_CIDATA_MNT}"/provision.yq/*; do

for f in "${LIMA_CIDATA_MNT}"/provision.yq/*; do

jandubois · 2025-08-27T19:48:26Z

pkg/cidata/cidata.TEMPLATE.d/boot.sh

+			format=$(deref "LIMA_CIDATA_YQ_PROVISION_${filename}_FORMAT")
+			path=$(deref "LIMA_CIDATA_YQ_PROVISION_${filename}_PATH")
+			permissions=$(deref "LIMA_CIDATA_YQ_PROVISION_${filename}_PERMISSIONS")
+			user=$(deref "LIMA_CIDATA_YQ_PROVISION_${filename}_USER")


I think the yq provision mode should use owner instead of user to match the data provision mode. The path and permissions properties use the same name, but user is different from owner, which is confusing.

Suggested change

user=$(deref "LIMA_CIDATA_YQ_PROVISION_${filename}_USER")

owner=$(deref "LIMA_CIDATA_YQ_PROVISION_${filename}_OWNER")

Of course this applies to all other places in the PR too.

user of mode: yq and owner of mode: data behave a little differently.

For the directory created by mkdir -p in mode: data, root will be the owner regardless of the owner.
If you create {{.Home}}/.config/docker/daemon.json with mode: data, sudo chown -R {{. User}} ~/.config is required for mode: user.

If user is aligned with owner, I want to align the behavior of mkdir -p of mode: data so that it is executed by the user of owner. 🤔

Maybe mode: data should be changed to do the same thing mode: yq does now for mkdir -p? It makes no sense to me that these 2 modes would behave differently.

I'll change to remove user and use owner instead.
I'll align the behavior of mode: data.
owner also includes group, but it is difficult to apply group to the directory to be created with mkdir -p alone.
I will probably leave a comment as TODO and use mkdir -p as it is. 🤔

jandubois · 2025-08-27T20:01:02Z

pkg/cidata/cidata.TEMPLATE.d/boot.sh

@@ -77,6 +77,64 @@ if [ -d "${LIMA_CIDATA_MNT}"/provision.data ]; then
 	done
 fi

+if [ -d "${LIMA_CIDATA_MNT}"/provision.yq ]; then
+	yq="${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq"


Guestagent is only installed when LIMA_CIDATA_PLAIN is not set. I assume the provisioning mode should work for plain instance too, so need to execute it from the cidata mount point.

Also need to make sure the guestagent is included in the CIDATA iso for plain instances if there is at least one yq provisioning entry.

Suggested change

yq="${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq"

yq="${LIMA_CIDATA_MNT}/lima-guestagent yq"

jandubois · 2025-08-27T20:45:41Z

pkg/limayaml/validate.go

+			for _, ptr := range []*string{
+				// mode: data
+				p.Content, p.Owner, p.Path,


Good catch of the additional templates. Maybe add &p.Script as well, then you can eliminate the check above as well.

jandubois · 2025-08-27T20:48:51Z

templates/default.yaml

+# # `yq` creates or edits a file in the guest filesystem by using `${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq`.
+# # `${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq` has the same functionality as https://github.com/mikefarah/yq.


I think that is too much internals knowledge. Just say it uses yq. We already use yq in other places, like the --set CLI options.

Suggested change

# # `yq` creates or edits a file in the guest filesystem by using `${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq`.

# # `${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq` has the same functionality as https://github.com/mikefarah/yq.

# # Create or edit a file in the guest filesystem by using `yq`.

jandubois · 2025-08-27T20:50:04Z

templates/default.yaml

+# # `${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq` has the same functionality as https://github.com/mikefarah/yq.
+# # The file specified by `path` will be updated with the specified `expression` using `yq --inplace`.
+# # If the file does not exist, it will be created using `yq --null-input`.
+# # `format` is optional and defaults to "auto", which is passed to `yq` by `--input-format <format>`, `--output-format <format>`.


Just explain what it does, not how it is implemented.

Suggested change

# # `format` is optional and defaults to "auto", which is passed to `yq` by `--input-format <format>`, `--output-format <format>`.

# # `format` is optional and defaults to "auto".

jandubois · 2025-08-27T20:52:06Z

templates/default.yaml

+# # Any missing directories will be created as needed using `mkdir -p`.
+# # The file permissions will be set to the specified value using `chmod`.
+# # The file and directory creation will be performed with the specified user privileges.


Suggested change

# # Any missing directories will be created as needed using `mkdir -p`.

# # The file permissions will be set to the specified value using `chmod`.

# # The file and directory creation will be performed with the specified user privileges.

# # Any missing directories will be created as needed.

# # The file permissions will be set to the specified value.

# # The file and directory creation will be performed as the specified owner.

jandubois · 2025-08-27T20:54:52Z

templates/docker-rootful.yaml

@@ -68,3 +67,5 @@ message: |
  docker context use lima-{{.Name}}
  docker run hello-world
  ------
+param:
+  containerdSnapshotter: false


I understand you probably don't want to change the default behaviour, but I would be supportive of changing the default to true.

Are there any scenarios where this would break something?

norio-nomura mentioned this pull request Aug 22, 2025

feature: use Rosetta AOT Caching with CDI #3858

Open

norio-nomura force-pushed the yq-provision-mode branch 2 times, most recently from cc10a18 to 6cb8a7e Compare August 22, 2025 05:56

norio-nomura force-pushed the yq-provision-mode branch from 6cb8a7e to 06b81fd Compare August 22, 2025 06:41

norio-nomura force-pushed the yq-provision-mode branch from 06b81fd to bb0337b Compare August 24, 2025 07:01

norio-nomura force-pushed the yq-provision-mode branch 4 times, most recently from 9a9c7c4 to 420f6ee Compare August 25, 2025 12:53

jandubois mentioned this pull request Aug 25, 2025

Add yq subcommands to limactl and lima-guestagent #3908

Merged

norio-nomura force-pushed the yq-provision-mode branch from 420f6ee to 03e90a4 Compare August 25, 2025 23:45

norio-nomura force-pushed the yq-provision-mode branch from 03e90a4 to 84e10a3 Compare August 25, 2025 23:48

norio-nomura force-pushed the yq-provision-mode branch from 84e10a3 to 08c6556 Compare August 26, 2025 01:55

norio-nomura force-pushed the yq-provision-mode branch 6 times, most recently from a71bd73 to 8f59a28 Compare August 27, 2025 08:41

norio-nomura force-pushed the yq-provision-mode branch from 8f59a28 to 71563bd Compare August 27, 2025 09:39

norio-nomura marked this pull request as ready for review August 27, 2025 10:07

jandubois requested changes Aug 27, 2025

View reviewed changes

	for expression in "${LIMA_CIDATA_MNT}"/provision.yq/*; do
	for f in "${LIMA_CIDATA_MNT}"/provision.yq/*; do

	user=$(deref "LIMA_CIDATA_YQ_PROVISION_${filename}_USER")
	owner=$(deref "LIMA_CIDATA_YQ_PROVISION_${filename}_OWNER")

	yq="${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq"
	yq="${LIMA_CIDATA_MNT}/lima-guestagent yq"

		# # `yq` creates or edits a file in the guest filesystem by using `${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq`.
		# # `${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq` has the same functionality as https://github.com/mikefarah/yq.

	# # `yq` creates or edits a file in the guest filesystem by using `${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq`.
	# # `${LIMA_CIDATA_GUEST_INSTALL_PREFIX}/bin/lima-guestagent yq` has the same functionality as https://github.com/mikefarah/yq.
	# # Create or edit a file in the guest filesystem by using `yq`.

	# # `format` is optional and defaults to "auto", which is passed to `yq` by `--input-format <format>`, `--output-format <format>`.
	# # `format` is optional and defaults to "auto".

feature: "yq" provision mode #3892

Are you sure you want to change the base?

feature: "yq" provision mode #3892

Conversation

norio-nomura commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AkihiroSuda commented Aug 22, 2025

Uh oh!

jandubois commented Aug 22, 2025

Uh oh!

norio-nomura commented Aug 22, 2025

Uh oh!

jandubois commented Aug 22, 2025

Uh oh!

jandubois commented Aug 22, 2025

Uh oh!

AkihiroSuda commented Aug 22, 2025

Uh oh!

jandubois commented Aug 22, 2025

Uh oh!

jandubois commented Aug 22, 2025

Uh oh!

norio-nomura commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

norio-nomura commented Aug 22, 2025

Uh oh!

afbjorklund commented Aug 22, 2025

Uh oh!

jandubois commented Aug 22, 2025

Uh oh!

jandubois commented Aug 22, 2025

Uh oh!

jandubois commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

norio-nomura commented Aug 24, 2025

Uh oh!

norio-nomura commented Aug 24, 2025

Uh oh!

norio-nomura commented Aug 24, 2025

Uh oh!

afbjorklund commented Aug 25, 2025

Uh oh!

jandubois commented Aug 25, 2025

Uh oh!

norio-nomura commented Aug 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

norio-nomura commented Aug 26, 2025

Uh oh!

norio-nomura commented Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jandubois Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

norio-nomura commented Aug 22, 2025 •

edited

Loading

norio-nomura commented Aug 22, 2025 •

edited

Loading

jandubois commented Aug 22, 2025 •

edited

Loading

norio-nomura commented Aug 25, 2025 •

edited

Loading

norio-nomura commented Aug 26, 2025 •

edited

Loading

jandubois Aug 27, 2025 •

edited

Loading