boot,bootloader: add support for shim fallback and setting EFI boot variables on install by olivercalder · Pull Request #13205 · canonical/snapd

olivercalder · 2023-09-19T02:10:30Z

We want to boot Ubuntu Core from EFI/ubuntu/shim*.efi, setup fallback boot properly, and manually set EFI boot variables on install so that the correct efi asset is used without going through a long multi-fallback boot process.

This relates to the following changes in the pc-gadget snap:

22 Support shim fallback bootchain pc-gadget#106

This is meant to be a revival/replacement of the following existing snapd PRs:

boot: set EFI boot variables on install #13025 (my naïve first attempt)
bootloader/grub: Add support for shim with fallback #11437
bootloader/grub: Add support for shim with fallback #11448

codecov-commenter · 2023-09-19T02:29:30Z

Codecov Report

Merging #13205 (ba950c1) into master (e91e93b) will decrease coverage by 0.01%.
The diff coverage is 73.68%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

@@            Coverage Diff             @@
##           master   #13205      +/-   ##
==========================================
- Coverage   78.81%   78.80%   -0.01%     
==========================================
  Files        1020     1021       +1     
  Lines      127158   127308     +150     
==========================================
+ Hits       100214   100321     +107     
- Misses      20670    20701      +31     
- Partials     6274     6286      +12

Flag	Coverage Δ
unittests	`78.80% <73.68%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
boot/makebootable.go	`70.68% <100.00%> (+0.40%)`	⬆️
bootloader/bootloader.go	`75.00% <ø> (ø)`
gadget/update.go	`85.62% <ø> (ø)`
bootloader/grub.go	`81.28% <72.72%> (-0.61%)`	⬇️
boot/setefibootvars_linux.go	`72.58% <72.58%> (ø)`

... and 3 files with indirect coverage changes

📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

The fallback grub EFI asset is already in EFI/ubuntu/, but the non-fallback grub asset should be in EFI/BOOT/ as well. The sources for these grub assets are identical. This corresponds to work in snapd PR #13205. See the following for more details: - canonical/snapd#13205 Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

olivercalder

I have a question about the connection between the shim fallback path, the recovery boot chain, and trusted assets in general. Perhaps @alfonsosanchezbeato or @valentindavid could help provide some insight? Thanks!

olivercalder · 2023-09-26T02:36:51Z

I still need to add unit tests and spread tests, but the logic is here and I would appreciate getting the review process started.

alfonsosanchezbeato

This is nice. I have some some comments. I am also wondering how this would work in different scenarios:

In most cases, a UC image is flashed to a system to the expected disk target. In that case, with the proposed changes to the gadget, fbx64.efi would automatically create the needed boot entry in EFI NVRAM, so the code for handling the variables would not be used, at least in normal cases.
The second scenario is for hybrid images. There, either subiquity or snapd (via installer API) will need to set appropriately the EFI boot entry. Iirc it was decided that snaps would do this, maybe @pedronis could confirm. But I'm not sure if the glue code for this is already here.

Other things that come to my mind:

Should we be follow the same algorithm as fallback.efi from shim (https://github.com/rhboot/shim/blob/main/fallback.c) when creating the variables, for consistency?
In the boot partition we have EFI/boot/grubx64.efi, maybe we should move it to a different path now.

valentindavid · 2023-10-23T12:52:58Z

Why do we test for that?

I just want to make sure the ARCH variable was set correctly, otherwise later parts of the test fail in not fun to debug ways. But it's true, maybe a test -n $ARCH would be better.

Alternatively, if there's a better way to compute the ARCH string used in EFI assets (i.e. the x64 in shimx64.efi), that would be even better. In addition to being set, I also wanted to make sure that the logic which sets the ARCH variable was successful.

olivercalder · 2023-10-24T21:20:18Z

I've added a simpler spread test that invokes directly the code which sets EFI boot variables. This test passes locally, so it seems the remaining problems with the original nested test relate to if/when/how the code is invoked during the installation process.

I am essentially out of ideas as to why the variables are not being set in install mode. Any guidance from @alfonsosanchezbeato @valentindavid or @pedronis would be greatly appreciated, so we can try to get this landed before the sprint.

alfonsosanchezbeato

Added a remark. Also, a TODO about changing EFI variables when a gadget update happens should be added, maybe in update.go.

olivercalder · 2023-10-27T06:45:28Z

I believe the latest changes should finally fix the problems of the EFI variable-setting code not running.

The MacOS quick checks still need to be fixed, since go-efilib/linux is used throughout the code -- this isn't a problem since the code should only ever be run on Ubuntu Core, which is... not MacOS.

In the future, an additional PR should be opened to set EFI boot variables in gadget/update.go whenever the gadget snap is updated, since that may change where the EFI binaries are located.

alfonsosanchezbeato

Some comments, but overall it is in the good direction I'd say

Meulengracht

Initial pass of PR, overall looks super nice!

Meulengracht · 2023-11-02T09:01:21Z

Nitpick: could this be moved to a seperate function?

Yes, I like this idea. I've moved it into a new function SetUbuntuSeedEfiBootVariables (not thrilled about the name) in setefibootvars.go.

alfonsosanchezbeato

Thanks for the changes! I have a some additional comments now

alfonsosanchezbeato

LGTM, thanks for all the changes! I have a minor comment if you want to address.

alfonsosanchezbeato · 2023-11-03T07:39:42Z

This function does not need to be exported

Good catch, thanks! I much prefer this to be unexported, with just the more general SetEfiBootVariables exported.

olivercalder · 2023-11-03T07:57:03Z

I still need to fix MacOS quick checks before this can be merged.

Meulengracht

Thanks for the changes, lgtm.

When gadget uses shim fallback mode, the trusted assets chain is different. Add support to detect that. LP: #1962182 Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com> Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: renamed trustedShimFallbackBinary to seedShimPath Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: refactored setting EFI boot variables at install Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: adjusted variable names and fixed variable initialization Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: improve setting Boot#### EFI variable Notably, splits off the process of reading a Boot#### variable and extracting its DevicePath into its own function `readBootVariable` which can be mocked and otherwise simplifies the `setBootNumberVariable` function. Also, fixes behavior around the final BootFFFF variable. Previously, it was not possible to select the BootFFFF variable if it was unused, due to overflow concerns on uint16. Now, the behavior around BootFFFF is identical to that of any other boot variable, by using an int internally instead of uint16, which also allows a more robust check for whether there were no matching variables. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: added unit tests for setting EFI Boot#### variable Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: refactored setting EFI boot variables Rewrote EFI boot variable functions to more closely match the behavior of shim fallback: https://github.com/rhboot/shim/blob/main/fallback.c In particular, the following have changed: 1. Existing Boot#### variables must fully match the new load option to be considered a match. In particular, the load option attributes, label, and device path must all be byte-for-byte identical. Previously, only the device paths were compared. 2. Matching Boot#### variables are no longer overwritten. Since the variable data must now byte-for-byte match the new load option, there is no need to overwrite the existing variable. 3. Since existing Boot#### variables are no longer overwritten, the variable attributes are no longer checked for those variables. Instead, it is assumed that the Boot#### variable attributes are viable for it to be used as a boot option. This matches the behavior of `rhboot/shim/fallback.c`, for better or for worse. 4. When modifying the BootOrder variable, boot option numbers are no longer pruned if there is no matching Boot#### variable. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot,bootloader: introduce UefiBootloader to build EFI load options Previously, the path of the shim binary relative to the EFI partition was passed into `SetEfiBootVariables`. However, different bootloaders may wish to set up `OptionalData` in the load option. Additionally, not all `TrustedAssetBootloaders` will attempt to set EFI boot variables, and not all bootloaders which should set EFI boot variables necessarily support secure boot. Thus, these should be decoupled. This commit adds a new `UefiBootloader` interface with the `ConstructShimEfiLoadOption` method, which builds an EFI load option from the shim path for the given bootloader. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot,bootloader: fixed linting errors and improved EFI boot variable test clarity Signed-off-by: Oliver Calder <oliver.calder@canonical.com> bootloader: improved unit test for grub EFI load option creation Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: set EFI boot variables in `MakeRunnableSystem` Previously, attempted to set boot variables in `MakeRecoverySystemBootable`, which is called by `MakeBootableImage`, which is called when building the image file, rather than during install mode. `MakeRunnableSystem` is called on first boot during install mode, and thus should be responsible for setting EFI boot variables. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: use seed bootloader when setting EFI variables In install mode, the bootloader located in ubuntu-seed should be used when setting the EFI boot variables. Previously, the bootloader in ubuntu-boot was accidentally re-used. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> tests: added simple test to execute setefibootvar.go code Signed-off-by: Oliver Calder <oliver.calder@canonical.com> tests: fixed standalone set EFI vars code test to work with different layouts Signed-off-by: Oliver Calder <oliver.calder@canonical.com> tests: moved simple setefibootvar.go check to nested test Signed-off-by: Oliver Calder <oliver.calder@canonical.com> tests: added check for idempotence when setting EFI boot variables Signed-off-by: Oliver Calder <oliver.calder@canonical.com> bootloader: adjust comments, organization, and add TODO Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot,bootloader: fix setting EFI boot variables Make function to search for EFI asset device path and construct load option common so each UefiBootloader does not have to re-implement it. Instead, the bootloader returns the description, asset file path, and optional data, which can then be used to create the EFI load option. Also, in `makeRunnableSystem`, the bootloader in ubuntu-seed must have `NoSlashBoot` in order to correctly find the grub.cfg file and thus the grub bootloader. This commit fixes this bug, and refactors a bit to account for the changes in responsibilities between the bootloader and the setefibootvars.go code. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> bootloader: fixed grub EFI load option test with tmp rootdir Signed-off-by: Oliver Calder <oliver.calder@canonical.com> go.mod: move golang.org/x/text import next to other golang.org/x/ imports Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: adjust opts to look for recovery bootloader when setting EFI variables Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: do not overwrite BootOrder if unchanged, and unexport EFI variable helper functions Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: unexport `setEfiBootOrderVariable` Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: move code to detect bootloader and set EFI variables accordingly into dedicated function Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: unexport `setUbuntuSeedEfiBootVariables` and accompanying error Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot,bootloader: ensure nil optionalData for EFI variable is equivalent to 0-length slice Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: handle empty boot order and other boot var improvements Signed-off-by: Oliver Calder <oliver.calder@canonical.com> boot: make setefibootvars functions linux-only Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

The test checks that EFI boot variables exist for the following: 1. A Boot#### variable pointing to the shim file path. 2. A BootOrder variable with the #### from the above Boot#### as first. Since the layout of EFI assets is dependent on the gadget snap, the test downloads and unpacks the gadget, then modifies the contents so that one variant has the shim and grub binaries in `EFI/boot/` and another variant has the shim and grub binaries in `EFI/ubuntu/` and the fallback binary in `EFI/boot/`. After building a core image around that modified gadget, the VM is booted and the test checks that the EFI variables are set correctly. Then, the test modifies the gadget to match the other variant's initial layout, and then installs the newly modified gadget. This should trigger re-setting EFI boot variables as well. Signed-off-by: Oliver Calder <oliver.calder@canonical.com> tests: fix problems in spread test for setting EFI boot variables Signed-off-by: Oliver Calder <oliver.calder@canonical.com> tests: disabled TPM on EFI boot vars test and separated gadget script Signed-off-by: Oliver Calder <oliver.calder@canonical.com> tests: fixed EFI vars test to use correct toolbox and include all EFI assets Signed-off-by: Oliver Calder <oliver.calder@canonical.com> tests: modify-gadget.sh re-use existing gadget so edition is incremented Signed-off-by: Oliver Calder <oliver.calder@canonical.com> tests: fix mangled EFI var search string and other improvements Signed-off-by: Oliver Calder <oliver.calder@canonical.com> tests: polish tests for setting EFI boot variables Notably, allow tests/nested/core/core20-set-efi-boot-variables to run on arm64 as well as amd64, simplify setefivars.go to search for multiple assets on multiple architectures, and allow tests/nested/manual/core20-set-efi-boot-vars to run on any ubuntu-2*. Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

pedronis

some remarks and questions

pedronis · 2023-11-08T17:09:56Z

 		return nil, err
 	}
-	return []string{assets.shimBinary, assets.grubBinary}, nil
+	if osutil.FileExists(filepath.Join(g.rootdir, assets.fallbackBinary)) {


what happens during installation or refresh with this condition? does it change from false to true at a reasonable point? should it based on what is in the rootdir or what is in the gadget? should that vary if the bootloader was produced via Find vs ForGadget?

In doInstall the assets are updated before the resealing happens:
https://github.com/snapcore/snapd/blob/a90520700167916c204872c93ec74a14c517af86/overlord/snapstate/snapstate.go#L548-L563
("update-gadget-assets" before "update-gadget-cmdline") so in principle the file should be there already before resealing.

I think the confusing thing is that it is already broken. Because we call NewBootFile("", ta, RoleRecovery) and NewBootFile("", ta, RoleRunMode), we probably do not do a proper first reseal (the one with all bootchains).

We do pass the kernel path to BootChain and RecoveryBootChain. But not the gadget. So we are missing some information here.

sorry, what is exactly already broken? if the first sealing was broken we would not be able to boot the first time? at installation time the gadget assets reach the seal code via the TrustedAssetsInstallObserver that is then passed to MakeRunnableSystem but you are probably referring to something different but I'm not sure what

during a refresh/ assets update the initial reseal is done via TrustedAssetsUpdateObserver and then a final seal is done after reboot I think, via observeSuccessfulBootAssets in the boot package

@pedronis I have PR #13398 with a fix for that.

It seems I cannot push to this PR. Should I just open that other PR to continue there?

I wonder why you can't push, at the bottom it says it's allowed. Anyway should I look at reviewing #13398? that one seems already very big though, so I wouldn't push more there but start something on top if neeeded

It is commit cb88e79

I have rebased that commit to master in #13412. I will make sure the tests are passing. And I will ping you then.

But there will be some conflict resolution needed in this PR.

@pedronis I have opened #13412. Maybe you can have a quick pass to see if the approach is reasonable.

pedronis · 2023-11-08T17:10:30Z

+		logger.Debugf("%v", err)
+	} else if err != nil {
+		logger.Debugf("WARNING: %v", err)


why the decision to only log errors here?

pedronis · 2023-11-08T17:12:03Z

 		return fmt.Errorf("cannot record %q as a recovery capable system: %v", recoverySystemLabel, err)
 	}
+
+	err = setUbuntuSeedEfiBootVariables()


I think this should be closer to where we call bootloader.InstallBootConfig

ernestl · 2024-03-13T09:56:53Z

This PR is superseded by #13511. Closing. @olivercalder @valentindavid

olivercalder force-pushed the shim-fallback-efi-vars branch 2 times, most recently from fb65a92 to 3f258f3 Compare September 21, 2023 20:47

olivercalder changed the title ~~boot,bootloader: add support for shim, fallback, and setting EFI boot variables on install~~ boot,bootloader: add support for shim fallback and setting EFI boot variables on install Sep 22, 2023

olivercalder mentioned this pull request Sep 22, 2023

boot: set EFI boot variables on install #13025

Closed

olivercalder mentioned this pull request Sep 25, 2023

22 Support shim fallback bootchain canonical/pc-gadget#106

Open

olivercalder commented Sep 25, 2023

View reviewed changes

Comment thread boot/makebootable.go Outdated

olivercalder marked this pull request as ready for review September 26, 2023 02:36

olivercalder requested review from alfonsosanchezbeato and valentindavid September 26, 2023 02:38

olivercalder added this to the 2.61 milestone Sep 26, 2023

alfonsosanchezbeato reviewed Sep 26, 2023

View reviewed changes

Comment thread bootloader/grub.go Outdated

Comment thread boot/setefibootvars.go Outdated

Comment thread boot/setefibootvars.go Outdated

Comment thread boot/setefibootvars.go Outdated

github-actions Bot added the Run Nested -auto- Label automatically added in case nested tests need to be executed label Sep 30, 2023

mvo5 modified the milestones: 2.61, 2.62 Oct 4, 2023

olivercalder mentioned this pull request Oct 5, 2023

24 Support shim fallback bootchain canonical/pc-gadget#107

Open

alfonsosanchezbeato reviewed Oct 5, 2023

View reviewed changes

Comment thread tests/nested/core/core20-set-efi-boot-vars/task.yaml Outdated

Comment thread tests/nested/core/core20-set-efi-boot-vars/task.yaml Outdated

olivercalder requested review from Meulengracht and alfonsosanchezbeato October 16, 2023 04:38

valentindavid reviewed Oct 19, 2023

View reviewed changes

Comment thread boot/setefibootvars.go Outdated

valentindavid reviewed Oct 19, 2023

View reviewed changes

Comment thread boot/setefibootvars.go Outdated

olivercalder force-pushed the shim-fallback-efi-vars branch from 1ed9dc6 to e96d6f3 Compare October 19, 2023 14:40

olivercalder requested a review from valentindavid October 20, 2023 07:05

valentindavid reviewed Oct 23, 2023

View reviewed changes

olivercalder requested review from pedronis and valentindavid October 24, 2023 21:20

alfonsosanchezbeato reviewed Oct 25, 2023

View reviewed changes

Comment thread bootloader/bootloader.go Outdated

Comment thread boot/makebootable.go Outdated

olivercalder requested a review from alfonsosanchezbeato October 27, 2023 06:45

alfonsosanchezbeato reviewed Oct 30, 2023

View reviewed changes

Meulengracht reviewed Nov 2, 2023

View reviewed changes

alfonsosanchezbeato reviewed Nov 2, 2023

View reviewed changes

Comment thread boot/setefibootvars.go Outdated

Comment thread boot/setefibootvars.go Outdated

Comment thread boot/setefibootvars.go Outdated

alfonsosanchezbeato approved these changes Nov 3, 2023

View reviewed changes

olivercalder requested a review from Meulengracht November 3, 2023 14:04

Meulengracht approved these changes Nov 7, 2023

View reviewed changes

olivercalder force-pushed the shim-fallback-efi-vars branch 3 times, most recently from 8ea377d to 9d7985d Compare November 7, 2023 14:51

olivercalder added 3 commits November 7, 2023 19:22

tests: skip EFI boot vars test after gadget update -- TODO: revert

ba950c1

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

olivercalder force-pushed the shim-fallback-efi-vars branch from 9d7985d to ba950c1 Compare November 7, 2023 17:30

pedronis reviewed Nov 8, 2023

View reviewed changes

pedronis added the ⛔ Blocked label Nov 8, 2023

olivercalder assigned valentindavid Nov 16, 2023

This was referenced Dec 1, 2023

boot/grub.go: return all possible boot chains #13398

Closed

boot/grub.go: add new bootchain #13412

Merged

valentindavid mentioned this pull request Jan 23, 2024

boot,bootloader: add support for shim fallback and setting EFI boot variables on install #13511

Merged

ernestl closed this Mar 13, 2024

Conversation

olivercalder commented Sep 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Sep 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

olivercalder left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

olivercalder commented Sep 26, 2023

Uh oh!

alfonsosanchezbeato left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

olivercalder commented Oct 24, 2023

Uh oh!

alfonsosanchezbeato left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

olivercalder commented Oct 27, 2023

Uh oh!

alfonsosanchezbeato left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Meulengracht left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alfonsosanchezbeato left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alfonsosanchezbeato left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

olivercalder commented Nov 3, 2023

Uh oh!

olivercalder commented Sep 19, 2023 •

edited

Loading

codecov-commenter commented Sep 19, 2023 •

edited

Loading

alfonsosanchezbeato left a comment •

edited

Loading