Skip to content

[sp-sim] configurable initial contents of the cabooses #8734

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

karencfv
Copy link
Contributor

@karencfv karencfv commented Jul 31, 2025

This is a proposal for configurable cabooses in sp-sim and omicron-dev run-all environments. I wasn't really sure how configurable we wanted the tests that start up an sp-sim to be. I'm open to opinions/suggestions :)

Manual testing with the following configuration file

<...>

[[simulated_sps.sidecar]]
serial_number = "SimSidecar0"
manufacturing_root_cert_seed = "01de01de01de01de01de01de01de01de01de01de01de01de01de01de01de01de"
device_id_cert_seed = "01de000000000000000000000000000000000000000000000000000000000000"

[simulated_sps.sidecar.cabooses.sp_slot_0]
board = "sidecar-b"
git_commit = "437c053649220611cdffc8a8d8d4033c0ef4b89c"
name = "sidecar-b"
version = "1.0.44"

[simulated_sps.sidecar.cabooses.sp_slot_1]
board = "sidecar-b"
git_commit = "437c053649220611cdffc8a8d8d4033c0ef4b89c"
name = "sidecar-b"
version = "1.0.44"

[simulated_sps.sidecar.cabooses.rot_slot_a]
board = "oxide-rot-1"
git_commit = "6edf9b5e6aa5c928a5462bda1f7a4c6f3caa40ab"
name = "oxide-rot-1"
version = "1.0.35"
sign = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"

[simulated_sps.sidecar.cabooses.rot_slot_b]
board = "oxide-rot-1"
git_commit = "6edf9b5e6aa5c928a5462bda1f7a4c6f3caa40ab"
name = "oxide-rot-1"
version = "1.0.35"
sign = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"

[simulated_sps.sidecar.cabooses.stage0]
board = "oxide-rot-1"
git_commit = "bdf56dd950b934360df596ed5b2d8b8813c92168"
name = "oxide-rot-1"
version = "1.4.0"
sign = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"

[simulated_sps.sidecar.cabooses.stage0_next]
board = "oxide-rot-1"
git_commit = "bdf56dd950b934360df596ed5b2d8b8813c92168"
name = "oxide-rot-1"
version = "1.4.0"
sign = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"

<...>

[[simulated_sps.gimlet]]
serial_number = "SimGimlet00"
manufacturing_root_cert_seed = "01de01de01de01de01de01de01de01de01de01de01de01de01de01de01de01de"
device_id_cert_seed = "01de000000000000000000000000000000000000000000000000000000000002"

[simulated_sps.gimlet.cabooses.sp_slot_0]
board = "gimlet-e"
git_commit = "437c053649220611cdffc8a8d8d4033c0ef4b89c"
name = "gimlet-e"
version = "1.0.44"

[simulated_sps.gimlet.cabooses.sp_slot_1]
board = "gimlet-e"
git_commit = "437c053649220611cdffc8a8d8d4033c0ef4b89c"
name = "gimlet-e"
version = "1.0.44"

[simulated_sps.gimlet.cabooses.rot_slot_a]
board = "oxide-rot-1"
git_commit = "6edf9b5e6aa5c928a5462bda1f7a4c6f3caa40ab"
name = "oxide-rot-1"
version = "1.0.35"
sign = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"

[simulated_sps.gimlet.cabooses.rot_slot_b]
board = "oxide-rot-1"
git_commit = "6edf9b5e6aa5c928a5462bda1f7a4c6f3caa40ab"
name = "oxide-rot-1"
version = "1.0.35"
sign = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"

[simulated_sps.gimlet.cabooses.stage0]
board = "oxide-rot-1"
git_commit = "bdf56dd950b934360df596ed5b2d8b8813c92168"
name = "oxide-rot-1"
version = "1.4.0"
sign = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"

[simulated_sps.gimlet.cabooses.stage0_next]
board = "oxide-rot-1"
git_commit = "bdf56dd950b934360df596ed5b2d8b8813c92168"
name = "oxide-rot-1"
version = "1.4.0"
sign = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"

<...>

Against a simulated omicron environment

$ ./target/debug/omdb --dns-server [::1]:56966 db inventory collections show latest sp
<...>

Sled SimGimlet00
    part number: i86pc
    power:    A0
    revision: 0
    MGS slot: Sled 0
    found at: 2025-08-15T05:48:05.243Z from http://[::1]:58847
    host phase 1 active slot: A
    host phase 1 hashes:
        SLOT HASH                                                             
        A    356bef6edc8fc80f2270b63b66abc37b293c5542acc9f40f5a96d914a23f6113 
        B    0b2fa98f5133b288a2be7e502eaae086c270bd2ba446d61a5aeba1fd8c1a9ac5 
    cabooses:
        SLOT       BOARD       NAME        VERSION GIT_COMMIT                               SIGN                                                             
        SpSlot0    gimlet-e    gimlet-e    1.0.44  437c053649220611cdffc8a8d8d4033c0ef4b89c n/a                                                              
        SpSlot1    gimlet-e    gimlet-e    1.0.44  437c053649220611cdffc8a8d8d4033c0ef4b89c n/a                                                              
        RotSlotA   oxide-rot-1 oxide-rot-1 1.0.35  6edf9b5e6aa5c928a5462bda1f7a4c6f3caa40ab 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 
        RotSlotB   oxide-rot-1 oxide-rot-1 1.0.35  6edf9b5e6aa5c928a5462bda1f7a4c6f3caa40ab 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 
        Stage0     oxide-rot-1 oxide-rot-1 1.4.0   bdf56dd950b934360df596ed5b2d8b8813c92168 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 
        Stage0Next oxide-rot-1 oxide-rot-1 1.4.0   bdf56dd950b934360df596ed5b2d8b8813c92168 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 
    RoT pages:
        SLOT         DATA_BASE64                         
        Cmpa         Z2ltbGV0LWNtcGEAAAAAAAAAAAAAAAAA... 
        CfpaActive   Z2ltbGV0LWNmcGEtYWN0aXZlAAAAAAAA... 
        CfpaInactive Z2ltbGV0LWNmcGEtaW5hY3RpdmUAAAAA... 
        CfpaScratch  Z2ltbGV0LWNmcGEtc2NyYXRjaAAAAAAA... 
    RoT: active slot: slot A
    RoT: persistent boot preference: slot A
    RoT: pending persistent boot preference: -
    RoT: transient boot preference: -
    RoT: slot A SHA3-256: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    RoT: slot B SHA3-256: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb

Sled SimGimlet01
    part number: i86pc
    power:    A0
    revision: 0
    MGS slot: Sled 1
    found at: 2025-08-15T05:48:05.254Z from http://[::1]:58847
    host phase 1 active slot: A
    host phase 1 hashes:
        SLOT HASH                                                             
        A    356bef6edc8fc80f2270b63b66abc37b293c5542acc9f40f5a96d914a23f6113 
        B    0b2fa98f5133b288a2be7e502eaae086c270bd2ba446d61a5aeba1fd8c1a9ac5 
    cabooses:
        SLOT       BOARD        NAME         VERSION GIT_COMMIT SIGN                                                             
        SpSlot0    SimGimletSp  SimGimlet    0.0.2   ffffffff   n/a                                                              
        SpSlot1    SimGimletSp  SimGimlet    0.0.1   fefefefe   n/a                                                              
        RotSlotA   SimRot       SimGimletRot 0.0.4   eeeeeeee   11594bb5548a757e918e6fe056e2ad9e084297c9555417a025d8788eacf55daf 
        RotSlotB   SimRot       SimGimletRot 0.0.3   edededed   11594bb5548a757e918e6fe056e2ad9e084297c9555417a025d8788eacf55daf 
        Stage0     SimRotStage0 SimGimletRot 0.0.200 ddddddddd  11594bb5548a757e918e6fe056e2ad9e084297c9555417a025d8788eacf55daf 
        Stage0Next SimRotStage0 SimGimletRot 0.0.200 dadadadad  11594bb5548a757e918e6fe056e2ad9e084297c9555417a025d8788eacf55daf 
    RoT pages:
        SLOT         DATA_BASE64                         
        Cmpa         Z2ltbGV0LWNtcGEAAAAAAAAAAAAAAAAA... 
        CfpaActive   Z2ltbGV0LWNmcGEtYWN0aXZlAAAAAAAA... 
        CfpaInactive Z2ltbGV0LWNmcGEtaW5hY3RpdmUAAAAA... 
        CfpaScratch  Z2ltbGV0LWNmcGEtc2NyYXRjaAAAAAAA... 
    RoT: active slot: slot A
    RoT: persistent boot preference: slot A
    RoT: pending persistent boot preference: -
    RoT: transient boot preference: -
    RoT: slot A SHA3-256: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    RoT: slot B SHA3-256: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb

Switch SimSidecar0
    part number: FAKE_SIM_SIDECAR
    power:    A2
    revision: 0
    MGS slot: Switch 0
    found at: 2025-08-15T05:48:05.223Z from http://[::1]:58847
    cabooses:
        SLOT       BOARD       NAME        VERSION GIT_COMMIT                               SIGN                                                             
        SpSlot0    sidecar-b   sidecar-b   1.0.44  437c053649220611cdffc8a8d8d4033c0ef4b89c n/a                                                              
        SpSlot1    sidecar-b   sidecar-b   1.0.44  437c053649220611cdffc8a8d8d4033c0ef4b89c n/a                                                              
        RotSlotA   oxide-rot-1 oxide-rot-1 1.0.35  6edf9b5e6aa5c928a5462bda1f7a4c6f3caa40ab 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 
        RotSlotB   oxide-rot-1 oxide-rot-1 1.0.35  6edf9b5e6aa5c928a5462bda1f7a4c6f3caa40ab 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 
        Stage0     oxide-rot-1 oxide-rot-1 1.4.0   bdf56dd950b934360df596ed5b2d8b8813c92168 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 
        Stage0Next oxide-rot-1 oxide-rot-1 1.4.0   bdf56dd950b934360df596ed5b2d8b8813c92168 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 
    RoT pages:
        SLOT         DATA_BASE64                         
        Cmpa         c2lkZWNhci1jbXBhAAAAAAAAAAAAAAAA... 
        CfpaActive   c2lkZWNhci1jZnBhLWFjdGl2ZQAAAAAA... 
        CfpaInactive c2lkZWNhci1jZnBhLWluYWN0aXZlAAAA... 
        CfpaScratch  c2lkZWNhci1jZnBhLXNjcmF0Y2gAAAAA... 
    RoT: active slot: slot A
    RoT: persistent boot preference: slot A
    RoT: pending persistent boot preference: -
    RoT: transient boot preference: -
    RoT: slot A SHA3-256: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    RoT: slot B SHA3-256: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb

Switch SimSidecar1
    part number: FAKE_SIM_SIDECAR
    power:    A2
    revision: 0
    MGS slot: Switch 1
    found at: 2025-08-15T05:48:05.233Z from http://[::1]:58847
    cabooses:
        SLOT       BOARD        NAME          VERSION GIT_COMMIT SIGN                                                             
        SpSlot0    SimSidecarSp SimSidecar    0.0.2   ffffffff   n/a                                                              
        SpSlot1    SimSidecarSp SimSidecar    0.0.1   fefefefe   n/a                                                              
        RotSlotA   SimRot       SimSidecarRot 0.0.4   eeeeeeee   11594bb5548a757e918e6fe056e2ad9e084297c9555417a025d8788eacf55daf 
        RotSlotB   SimRot       SimSidecarRot 0.0.3   edededed   11594bb5548a757e918e6fe056e2ad9e084297c9555417a025d8788eacf55daf 
        Stage0     SimRotStage0 SimSidecarRot 0.0.200 ddddddddd  11594bb5548a757e918e6fe056e2ad9e084297c9555417a025d8788eacf55daf 
        Stage0Next SimRotStage0 SimSidecarRot 0.0.200 dadadadad  11594bb5548a757e918e6fe056e2ad9e084297c9555417a025d8788eacf55daf 
    RoT pages:
        SLOT         DATA_BASE64                         
        Cmpa         c2lkZWNhci1jbXBhAAAAAAAAAAAAAAAA... 
        CfpaActive   c2lkZWNhci1jZnBhLWFjdGl2ZQAAAAAA... 
        CfpaInactive c2lkZWNhci1jZnBhLWluYWN0aXZlAAAA... 
        CfpaScratch  c2lkZWNhci1jZnBhLXNjcmF0Y2gAAAAA... 
    RoT: active slot: slot A
    RoT: persistent boot preference: slot A
    RoT: pending persistent boot preference: -
    RoT: transient boot preference: -
    RoT: slot A SHA3-256: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    RoT: slot B SHA3-256: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb

Closes: #7913

@karencfv karencfv marked this pull request as ready for review August 15, 2025 06:07
Copy link
Contributor

@jgallagher jgallagher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some nits but in general this looks fine to me. In terms of #7913 - did you try updating an sp-sim running with a real board with a real artifact from a TUF repo?

@@ -62,7 +64,9 @@ impl GatewayTestContext {
}
}

pub fn load_test_config() -> (omicron_gateway::Config, sp_sim::Config) {
pub fn load_test_config(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems a little unfortunate this touches a bunch of callers and most of them are just passing DEFAULT_SP_SIM_CONFIG. I wonder if instead this should be a separate function? I.e., keep load_test_config() as it was, and add something like load_test_config_from(path) for the handful of callers that want to be able to pass a specific config?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, changing that specific function only removes the need for DEFAULT_SP_SIM_CONFIG in like two tests 🙃 . I was looking to avoid using Option<T> , but I've had to add a few, specifically in setup_with_config_impl<N: NexusServer>(), which kinda bubbles up to a few more methods.

I don't love it? But I guess if you're writing tests for the console API you really don't need to know where the configuration for the SP sim is coming from. This whole test and simulated component set-up probably needs a clean up, but I'm not sure this PR is the place to do it. It seems like it's not a trivial endeavour.

Copy link
Contributor Author

@karencfv karencfv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review! I think I've addressed your comments. It's still a little bit odd, but going down the rabbit hole of cleaning everything up was going to take a while. Let me know what you think!

@@ -62,7 +64,9 @@ impl GatewayTestContext {
}
}

pub fn load_test_config() -> (omicron_gateway::Config, sp_sim::Config) {
pub fn load_test_config(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, changing that specific function only removes the need for DEFAULT_SP_SIM_CONFIG in like two tests 🙃 . I was looking to avoid using Option<T> , but I've had to add a few, specifically in setup_with_config_impl<N: NexusServer>(), which kinda bubbles up to a few more methods.

I don't love it? But I guess if you're writing tests for the console API you really don't need to know where the configuration for the SP sim is coming from. This whole test and simulated component set-up probably needs a clean up, but I'm not sure this PR is the place to do it. It seems like it's not a trivial endeavour.

@karencfv
Copy link
Contributor Author

In terms of #7913 - did you try updating an sp-sim running with a real board with a real artifact from a TUF repo?

@jgallagher yep!

$ ./target/debug/faux-mgs --sp-sim-addr [::1]:51033 update sp 0 /var/tmp/R13/repo/targets/47266ede81e13f5f1e36623ea8dd963842606b783397e4809a9a5f0bda0f8170.gimlet_sp-gimlet-e-1.0.34.tar.gz
Aug 18 08:45:04.212 INFO creating SP handle on to talk to SP simulator at [::1]:51033, component: faux-mgs
Aug 18 08:45:04.213 INFO initial discovery complete, addr: [::1]:51033, component: faux-mgs
Aug 18 08:45:04.215 INFO generated update ID, id: e62d3282-b45e-4ffd-8f3c-7a09aca446fb, component: faux-mgs
Aug 18 08:45:04.353 INFO starting SP update, sp_image_size: 655616, aux_flash_size: 0, aux_flash_chck: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], id: e62d3282-b45e-4ffd-8f3c-7a09aca446fb, component: faux-mgs
Aug 18 08:45:04.354 INFO update in progress, total_size: 655616, bytes_received: 0, component: faux-mgs
Aug 18 08:45:04.354 INFO update preparation complete, update_id: e62d3282-b45e-4ffd-8f3c-7a09aca446fb, component: faux-mgs
Aug 18 08:45:04.434 INFO update complete, id: e62d3282-b45e-4ffd-8f3c-7a09aca446fb, component: faux-mgs
update complete

@@ -673,10 +675,14 @@ impl<'a, N: NexusServer> ControlPlaneTestContextBuilder<'a, N> {
&mut self,
switch_location: SwitchLocation,
port: Option<u16>,
sp_sim_config_file: Option<Utf8PathBuf>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ug, yeah, I don't love callers passing None here. I think I'd prefer either:

  • two methods (start_gateway that doesn't take this arg, start_gateway_with_sp_sim_config that takes this arg non-optionally)
  • go back to making this required, and make the callers pass the DEFAULT... constant

but in general I agree that all of this test setup is pretty rough and this isn't the PR to do major cleanup.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • two methods (start_gateway that doesn't take this arg, start_gateway_with_sp_sim_config that takes this arg non-optionally)
  • go back to making this required, and make the callers pass the DEFAULT... constant

I think I'd rather go back to making this required. Having two methods for start_gateway feels like it would add even more complexity to an already complex and convoluted testing setup. WDYT?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sp-sim could more faithfully simulate updates and cabooses
2 participants