[do-not-merge] Virtio-mem rebase #5435
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update the Kconfig to enable support for virtio-mem in the guest. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
virtio-mem requires a new Kconfig, so I've rebuilt the artifacts and now I'm making the devctr point to those. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Automatically generate bindings for virtio-mem. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Create the new module for the virtio-mem device. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Allow to configure the virtio-mem device from the VmmConfig and the PUT API to /hotplug/memory. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Test the freshly added PUT API to /hotplug/memory. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Add a dummy virtio-mem device that is detected by the guest driver. The device is configured with total_size, block_size, and slot_size, and uses a fixed address after the MMIO64 memory zone. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Check that the driver correctly detects the virtio-mem device, with the correct parameters. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Add support for GET /hotplug/memory that returns the current status of the virtio-mem device. This API can only be called after boot. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Add new API to swagger and device-api.md Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Avoid multiple conversions back and forth from MiB to bytes by just storing as MiB. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
We should match the exact reason for the RuntimeError to ensure we're failing for what we're expecting. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
It's better to be explicit on the conversion we're doing as u64 to usize is always safe in our platforms. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
This type annotation is redoundant, so we can remove it. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Implements basic snapshot/restore functionality for the dummy virtio-mem device. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Wire support for virtio-mem metrics, adding a few basic metrics: queue events, queue event fails, activation fails. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
commit 8208ee8 Author: Patrick Roy <roypat@amazon.co.uk> Date: Mon Sep 8 17:33:38 2025 +0100 fix(test): correct operator precedence bitshift shift has lower precedence than division, meaning in MemoryUsageExceededError.__init__ we end up trying to bitshift a float, which is nonsense (as the expression in the f-string gets interpreted as (threshold / 1) << 20, and the result of all divisions in python is a float). Add some parenthesis. We cannot just write this as `threshold >> 20`, as we care about the fractional part. Fixes: 7d1549f ("tests: fix MMIO gaps in memory monitor tool") Signed-off-by: Patrick Roy <roypat@amazon.co.uk> commit 0cd25f0 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Sep 8 16:11:14 2025 +0000 chore(deps): Bump the firecracker group across 1 directory with 19 updates Bumps the firecracker group with 11 updates in the / directory: | Package | From | To | | --- | --- | --- | | [zerocopy](https://github.com/google/zerocopy) | `0.8.26` | `0.8.27` | | [clap](https://github.com/clap-rs/clap) | `4.5.45` | `4.5.47` | | [uuid](https://github.com/uuid-rs/uuid) | `1.18.0` | `1.18.1` | | [micro_http](https://github.com/firecracker-microvm/micro-http) | ``98d8567`` | ``3248cee`` | | [log](https://github.com/rust-lang/log) | `0.4.27` | `0.4.28` | | [bitflags](https://github.com/bitflags/bitflags) | `2.9.3` | `2.9.4` | | [memfd](https://github.com/lucab/memfd-rs) | `0.6.4` | `0.6.5` | | [cc](https://github.com/rust-lang/cc-rs) | `1.2.34` | `1.2.36` | | [js-sys](https://github.com/wasm-bindgen/wasm-bindgen) | `0.3.77` | `0.3.78` | | [wasi](https://github.com/bytecodealliance/wasi-rs) | `0.14.2+wasi-0.2.4` | `0.14.4+wasi-0.2.4` | | [winapi-util](https://github.com/BurntSushi/winapi-util) | `0.1.10` | `0.1.11` | Updates `zerocopy` from 0.8.26 to 0.8.27 - [Release notes](https://github.com/google/zerocopy/releases) - [Changelog](https://github.com/google/zerocopy/blob/main/CHANGELOG.md) - [Commits](google/zerocopy@v0.8.26...v0.8.27) Updates `clap` from 4.5.45 to 4.5.47 - [Release notes](https://github.com/clap-rs/clap/releases) - [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md) - [Commits](clap-rs/clap@clap_complete-v4.5.45...clap_complete-v4.5.47) Updates `uuid` from 1.18.0 to 1.18.1 - [Release notes](https://github.com/uuid-rs/uuid/releases) - [Commits](uuid-rs/uuid@v1.18.0...v1.18.1) Updates `micro_http` from `98d8567` to `3248cee` - [Commits](firecracker-microvm/micro-http@98d8567...3248cee) Updates `log` from 0.4.27 to 0.4.28 - [Release notes](https://github.com/rust-lang/log/releases) - [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md) - [Commits](rust-lang/log@0.4.27...0.4.28) Updates `bitflags` from 2.9.3 to 2.9.4 - [Release notes](https://github.com/bitflags/bitflags/releases) - [Changelog](https://github.com/bitflags/bitflags/blob/main/CHANGELOG.md) - [Commits](bitflags/bitflags@2.9.3...2.9.4) Updates `memfd` from 0.6.4 to 0.6.5 - [Release notes](https://github.com/lucab/memfd-rs/releases) - [Commits](lucab/memfd-rs@v0.6.4...v0.6.5) Updates `cc` from 1.2.34 to 1.2.36 - [Release notes](https://github.com/rust-lang/cc-rs/releases) - [Changelog](https://github.com/rust-lang/cc-rs/blob/main/CHANGELOG.md) - [Commits](rust-lang/cc-rs@cc-v1.2.34...cc-v1.2.36) Updates `clap_builder` from 4.5.44 to 4.5.47 - [Release notes](https://github.com/clap-rs/clap/releases) - [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md) - [Commits](clap-rs/clap@v4.5.44...v4.5.47) Updates `clap_derive` from 4.5.45 to 4.5.47 - [Release notes](https://github.com/clap-rs/clap/releases) - [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md) - [Commits](clap-rs/clap@v4.5.45...v4.5.47) Updates `js-sys` from 0.3.77 to 0.3.78 - [Release notes](https://github.com/wasm-bindgen/wasm-bindgen/releases) - [Changelog](https://github.com/wasm-bindgen/wasm-bindgen/blob/main/CHANGELOG.md) - [Commits](https://github.com/wasm-bindgen/wasm-bindgen/commits) Updates `wasi` from 0.14.2+wasi-0.2.4 to 0.14.4+wasi-0.2.4 - [Commits](bytecodealliance/wasi-rs@0.14.2...0.14.4) Updates `wasm-bindgen` from 0.2.100 to 0.2.101 - [Release notes](https://github.com/wasm-bindgen/wasm-bindgen/releases) - [Changelog](https://github.com/wasm-bindgen/wasm-bindgen/blob/main/CHANGELOG.md) - [Commits](wasm-bindgen/wasm-bindgen@0.2.100...0.2.101) Updates `wasm-bindgen-backend` from 0.2.100 to 0.2.101 - [Release notes](https://github.com/wasm-bindgen/wasm-bindgen/releases) - [Changelog](https://github.com/wasm-bindgen/wasm-bindgen/blob/main/CHANGELOG.md) - [Commits](wasm-bindgen/wasm-bindgen@0.2.100...0.2.101) Updates `wasm-bindgen-macro` from 0.2.100 to 0.2.101 - [Release notes](https://github.com/wasm-bindgen/wasm-bindgen/releases) - [Changelog](https://github.com/wasm-bindgen/wasm-bindgen/blob/main/CHANGELOG.md) - [Commits](wasm-bindgen/wasm-bindgen@0.2.100...0.2.101) Updates `wasm-bindgen-macro-support` from 0.2.100 to 0.2.101 - [Release notes](https://github.com/wasm-bindgen/wasm-bindgen/releases) - [Changelog](https://github.com/wasm-bindgen/wasm-bindgen/blob/main/CHANGELOG.md) - [Commits](wasm-bindgen/wasm-bindgen@0.2.100...0.2.101) Updates `wasm-bindgen-shared` from 0.2.100 to 0.2.101 - [Release notes](https://github.com/wasm-bindgen/wasm-bindgen/releases) - [Changelog](https://github.com/wasm-bindgen/wasm-bindgen/blob/main/CHANGELOG.md) - [Commits](wasm-bindgen/wasm-bindgen@0.2.100...0.2.101) Updates `winapi-util` from 0.1.10 to 0.1.11 - [Commits](BurntSushi/winapi-util@0.1.10...0.1.11) Updates `zerocopy-derive` from 0.8.26 to 0.8.27 - [Release notes](https://github.com/google/zerocopy/releases) - [Changelog](https://github.com/google/zerocopy/blob/main/CHANGELOG.md) - [Commits](google/zerocopy@v0.8.26...v0.8.27) --- updated-dependencies: - dependency-name: zerocopy dependency-version: 0.8.27 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: clap dependency-version: 4.5.47 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: uuid dependency-version: 1.18.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: micro_http dependency-version: 3248ceeae41461d034624b582d5d358cd6e6f89f dependency-type: direct:production dependency-group: firecracker - dependency-name: log dependency-version: 0.4.28 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: bitflags dependency-version: 2.9.4 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: memfd dependency-version: 0.6.5 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: cc dependency-version: 1.2.36 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: clap_builder dependency-version: 4.5.47 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: clap_derive dependency-version: 4.5.47 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: js-sys dependency-version: 0.3.78 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: wasi dependency-version: 0.14.4+wasi-0.2.4 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: wasm-bindgen dependency-version: 0.2.101 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: wasm-bindgen-backend dependency-version: 0.2.101 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: wasm-bindgen-macro dependency-version: 0.2.101 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: wasm-bindgen-macro-support dependency-version: 0.2.101 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: wasm-bindgen-shared dependency-version: 0.2.101 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: winapi-util dependency-version: 0.1.11 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: zerocopy-derive dependency-version: 0.8.27 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker ... Signed-off-by: dependabot[bot] <support@github.com> commit 1bd63b2 Author: Nikita Kalyazin <kalyazin@amazon.com> Date: Wed Sep 3 15:15:29 2025 +0000 test(perf/vsock): copy iperf binary only once Problem: Currently we copy the iperf3-vsock binary into the guest in spawn_iperf3_client that is called by every client (we have 2). If by the time the second binary is being copied that first one is already executing, we get the "Text file busy" error. Solution: Move the copy into the test construction to copy the binary only once. Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com> commit 286aa17 Author: Riccardo Mancini <mancio@amazon.com> Date: Tue Sep 2 11:15:15 2025 +0100 chore(buildkite): add pipeline definition for release QA Adds a definition for release QA pipeline to also define it as code in our repository. This also adds a test mode when VERSION=dev where "make_release" is used to generate the artifacts instead of downloading them from S3. Signed-off-by: Riccardo Mancini <mancio@amazon.com> commit 5dde18c Author: Riccardo Mancini <mancio@amazon.com> Date: Wed Aug 27 15:46:38 2025 +0100 chore(release-policy): add 1.13 to the release policy table Add the new version to the table! Signed-off-by: Riccardo Mancini <mancio@amazon.com> commit 0656d09 Author: Riccardo Mancini <mancio@amazon.com> Date: Thu Aug 28 12:04:40 2025 +0100 chore: update CREDITS.md Update the credits to thank all our awesome contributors! Signed-off-by: Riccardo Mancini <mancio@amazon.com> commit ed27e22 Author: Riccardo Mancini <mancio@amazon.com> Date: Mon Sep 1 14:28:42 2025 +0100 chore(changelog): add missing entry for Swagger fix In firecracker-microvm#5418 I fixed the swagger definition but didn't add it to the changelog. For consistency with the release branch, let's keep it here as well. Signed-off-by: Riccardo Mancini <mancio@amazon.com> commit 4a21dc6 Author: Riccardo Mancini <mancio@amazon.com> Date: Mon Sep 1 15:07:29 2025 +0100 fix(getting-started): increase ext4 size and verify artifacts We increased the ext4 size from 400M to 500M in our devtool scripts, but not in the getting started. Let's bump it to 1G to avoid issues like this in the future. In the longer term, we may look into simplifying the getting started guide, but for now it's quicker to just fix the issue. Fixes: firecracker-microvm#5349 Signed-off-by: Riccardo Mancini <mancio@amazon.com> commit 1ba85a7 Author: Riccardo Mancini <mancio@amazon.com> Date: Mon Sep 1 13:44:32 2025 +0100 fix(swagger): fix typos in descriptions Fix another bunch of typos in the swagger spec descriptions. As these are only in the description, this doesn't have any functional change to code generation. - `s/snaphot/snapshot/` - `s/reponds/responds/` Signed-off-by: Riccardo Mancini <mancio@amazon.com> commit 35745b6 Author: Riccardo Mancini <mancio@amazon.com> Date: Mon Sep 1 13:43:33 2025 +0100 fix(swagger): typo in imds_compat parameter Replace `imds_comat` with the correct `imds_compat`. Fixes: firecracker-microvm#5417 Signed-off-by: Riccardo Mancini <mancio@amazon.com> commit 52eefd8 Author: Riccardo Mancini <mancio@amazon.com> Date: Wed Aug 27 12:15:29 2025 +0100 chore: pythonify BK definitions for docker and coverage We used to manage these definitions in the BK with no versioning or source control, let's move them to python as all the others so that they stay in sync with the latest and greatest updates to the supported platforms and instance types. Signed-off-by: Riccardo Mancini <mancio@amazon.com> commit 0b2fd57 Author: Patrick Roy <roypat@amazon.co.uk> Date: Wed Aug 27 14:38:25 2025 +0100 devctr: pin kani to 0.64.0 Kani 0.65.0 crashes with ICE when trying to verify firecracker, see also model-checking/kani#4322 Signed-off-by: Patrick Roy <roypat@amazon.co.uk> commit 7243f99 Author: Patrick Roy <roypat@amazon.co.uk> Date: Wed Aug 27 14:36:54 2025 +0100 fix(test): pass --workspace for kani invocation It seems that with the newer kani versions, we need to explicitly pass --workspace to get it to pick up harnesses from within workspace crates. Signed-off-by: Patrick Roy <roypat@amazon.co.uk> commit d1b9a5a Author: Patrick Roy <roypat@amazon.co.uk> Date: Wed Aug 27 13:49:46 2025 +0100 chore: bump devctr version we go from v83 to v84. Ubuntu version of the base image changed during the container rebuild, so update that one python doctest. Signed-off-by: Patrick Roy <roypat@amazon.co.uk> commit be1bb3b Author: Patrick Roy <roypat@amazon.co.uk> Date: Wed Aug 27 13:14:31 2025 +0100 chore: appease rustfmt, clippy and rustc Lots of new warnings and lints Signed-off-by: Patrick Roy <roypat@amazon.co.uk> commit 584c52c Author: Patrick Roy <roypat@amazon.co.uk> Date: Wed Aug 27 12:20:10 2025 +0100 chore: update rust toolchain version Current newest is 1.89.0 Signed-off-by: Patrick Roy <roypat@amazon.co.uk> commit 55abaa8 Author: Riccardo Mancini <mancio@amazon.com> Date: Wed Aug 27 10:15:12 2025 +0100 chore: drop support for Skylake instances We're dropping official support for our oldest supported instance type. There is no functional change, meaning Firecracker will continue to work on that instance type, but we will drop all automated testing on it. Signed-off-by: Riccardo Mancini <mancio@amazon.com> commit f101299 Author: Babis Chalios <bchalios@amazon.es> Date: Wed Aug 27 09:38:01 2025 +0200 fix: typo in Getting started guide `--enable-pci` flag was added within quotes that were enclosing the name for the API socket path, leading to invalid socket path (and no PCI support). Signed-off-by: Babis Chalios <bchalios@amazon.es> commit 6169ec3 Author: Babis Chalios <bchalios@amazon.es> Date: Mon Aug 25 17:39:29 2025 +0200 pci: document potential contention on PCI bus lock We have a single lock for all devices on the PCI bus that serializes reads and writes on the devices' PCI configuration space and BARs. This should not be a problem at the moment. It should be out of any hot path and only up until we are setting up devices. However, add a comment that mentions the existence of the contention so that we keep it in mind for the futuer (and maybe perform some profiling). Signed-off-by: Babis Chalios <bchalios@amazon.es> commit 431d77e Author: Babis Chalios <bchalios@amazon.es> Date: Mon Aug 25 17:29:29 2025 +0200 doc: add documentation on usage of PCI support Add information in our Documentation regarding how users can enable PCI support for Firecracker microVMs and mention requirements for building the guest kernel, as well as the requirements for kernel command line parameters. Also, add an entry in the CHANGELOG mentioning the addition of PCI support. Signed-off-by: Babis Chalios <bchalios@amazon.es> Signed-off-by: Riccardo Mancini <mancio@amazon.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## feature/virtio-mem #5435 +/- ##
======================================================
- Coverage 83.18% 82.68% -0.51%
======================================================
Files 272 269 -3
Lines 31009 27935 -3074
======================================================
- Hits 25796 23097 -2699
+ Misses 5213 4838 -375
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Manciukic
force-pushed
the
feature/virtio-mem
branch
from
397f1d2 to
62ce1ab
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Reason
Run integration tests.
License Acceptance
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.PR Checklist
tools/devtool checkbuild --allto verify that the PR passesbuild checks on all supported architectures.
tools/devtool checkstyleto verify that the PR passes theautomated style checks.
how they are solving the problem in a clear and encompassing way.
in the PR.
CHANGELOG.md.Runbook for Firecracker API changes.
integration tests.
TODO.rust-vmm.