DPDK attempt 2 #1072

daniel-noland · 2025-11-22T03:46:53Z

Partially addresses #1051

This is somewhat messy due to the integration push. I intend to clean it up, but I think it should be merged as soon as possible in order to avoid bit rot on the dpdk integration which follows this.

Copilot

Pull request overview

This PR implements an "init handoff" mechanism to transfer configuration from the dataplane-init process to the dataplane worker process using sealed memory file descriptors (memfd). The changes introduce comprehensive argument parsing, hardware device scanning, and zero-copy configuration serialization using rkyv.

Key changes:

Complete rewrite of init/src/main.rs to handle device detection, VFIO binding, and configuration handoff
Extensive documentation and type refinement in args/src/lib.rs including NetworkDeviceDescription, LaunchConfiguration, and FinalizedMemFile
Addition of ProfilingConfigSection for Pyroscope integration

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
init/src/main.rs	Implements hardware scanning, device validation, VFIO binding, and configuration serialization/handoff to dataplane worker
init/Cargo.toml	Adds dependencies for args parsing, command-fds, miette, rkyv, serde, and tracing
args/src/lib.rs	Adds extensive documentation, renames `PortArg` to `NetworkDeviceDescription`, adds `ProfilingConfigSection`, refines configuration types
dataplane/Cargo.toml	Adds rkyv dependency for configuration deserialization
Dockerfile	Changes entrypoint from `/dataplane` to `/dataplane-init` and copies artifact to both paths
Cargo.lock	Updates various dependency versions (clap, cc, hashbrown, indexmap, etc.)

Copilot · 2025-11-22T06:23:22Z

args/src/lib.rs

+)]
+#[rkyv(attr(derive(PartialEq, Eq, Debug)))]
+pub struct ProfilingConfigSection {
+    /// The URL of the pryroscope url


Corrected spelling of 'pryroscope' to 'pyroscope'.

Suggested change

/// The URL of the pryroscope url

/// The URL of the pyroscope server

init/src/main.rs

Copilot · 2025-11-22T06:23:23Z

args/src/lib.rs

    pub tracing: TracingConfigSection,
+    /// Metrics collection configuration
    pub metrics: MetricsConfigSection,
+    /// Profileing collection configuration


Corrected spelling of 'Profileing' to 'Profiling'.

Suggested change

/// Profileing collection configuration

/// Profiling collection configuration

Copilot · 2025-11-22T06:23:23Z

args/src/lib.rs

+#[derive(
+    Debug, Clone, PartialEq, Eq, serde::Serialize, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive,
+)]


ProfilingConfigSection is missing the CheckBytes derive macro. Since it's included in LaunchConfiguration (line 749) which derives CheckBytes, all nested types must also derive CheckBytes for rkyv validation to work correctly. Add CheckBytes to the derive list and add the appropriate use bytecheck::CheckBytes; if not already present.

Copilot · 2025-11-22T06:23:23Z

Dockerfile

 COPY --link --chown=0:0 "${ARTIFACT}" /dataplane
+COPY --link --chown=0:0 "${ARTIFACT}" /dataplane-init


The Dockerfile copies the same ${ARTIFACT} to both /dataplane (line 5) and /dataplane-init (line 6). Based on the context, it appears /dataplane-init should be a separate binary built from the init crate. The build argument should likely be ${ARTIFACT_INIT} or similar to distinguish between the init and dataplane binaries.

Suggested change

COPY --link --chown=0:0 "${ARTIFACT}" /dataplane

COPY --link --chown=0:0 "${ARTIFACT}" /dataplane-init

ARG ARTIFACT_INIT

COPY --link --chown=0:0 "${ARTIFACT}" /dataplane

COPY --link --chown=0:0 "${ARTIFACT_INIT}" /dataplane-init

this needed to be punted to the next PR (where it will be fixed)

args/src/lib.rs

Fredi-raspall · 2025-11-22T19:17:01Z

init/src/main.rs

+
+    let launch_config = launch_config.to_owned_fd();
+
+    let io_err = std::process::Command::new("/bin/dataplane")


I wonder if the path to the dataplane should be configurable via cmd line of init itself.

Why? It is fixed in the container?

The location of dataplane will be fix in image, yes. Both things (init and dataplane) will be part of the image, so it would be okay to hardcode it. OTH, making it configurable just gives you more flexibility and makes things less implicit. But we can address this later if needed. We just need to make sure that the image builder ensures that dataplane is at /bin/dataplane (or change the init to just "dataplane" and make sure /bin is in the PATH?)

Fredi-raspall

Looks good to me Daniel. Thanks!
Please address the small typos detected by copilot.

qmonnet

Please split your first commit. Description says “update documentation”, but it does also a number of other modifications to the code. They look minor, but the size of the commit makes it hard to follow what it does.

I think we should also consider mentioning when some code or docs is AI-generated (it looks like some of you docs are).

args/src/lib.rs

qmonnet · 2025-11-24T11:39:59Z

args/src/lib.rs

+    /// Number of dataplane worker threads / cores
+    pub dataplane_workers: usize,


This doesn't seem to be used in this commit.

The goal is to fill out the file. Not consume the file. That is for another PR

OK, in that case it would be worth mentioning it in the commit description, in my opinion. At the moment the title says docs(args): basic docs for LaunchConfiguration, which implies we only add docs in that commit, and it's not clear at all why we also add new fields to the struct.

qmonnet · 2025-11-24T11:41:45Z

args/src/lib.rs

+    /// Metrics collection configuration
    pub metrics: MetricsConfigSection,
+    /// Profileing collection configuration
+    pub profiling: ProfilingConfigSection,


Same, I see where you set the field but I can't find where you read it. I may have missed it, which is also a good indicator this commit contains too many things.

I don't read it in this PR. The goal is to fill out the file, not consume it

Same discussion as above #1072 (comment)

qmonnet · 2025-11-24T14:46:42Z

Now that we've done GW R1, I updated target milestone to GW R2.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

New config file section to set up and manage pyroscope. Signed-off-by: Daniel Noland <daniel@githedgehog.com>

This allows a child process to inherit configuration from a parent process. Signed-off-by: Daniel Noland <daniel@githedgehog.com>

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

Fixes for clippy pedantic and general style. Signed-off-by: Daniel Noland <daniel@githedgehog.com>

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

This commit series introduces new functionality which broke the prior tests. This commit amends the tests to account for that. Signed-off-by: Daniel Noland <daniel@githedgehog.com>

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

qmonnet · 2025-11-25T10:01:39Z

args/src/lib.rs

    pub fn kernel_num_workers(&self) -> usize {
        self.num_workers.into()
    }
-    // backwards-compatible, to deprecate


We should probably keep that comment along with the new one?

qmonnet · 2025-11-25T10:02:45Z

args/src/lib.rs

-pub type KiB = NonZero<u64>;
-
-#[derive(
-    Debug,
-    Default,
-    serde::Serialize,
-    serde::Deserialize,
-    rkyv::Serialize,
-    rkyv::Deserialize,
-    rkyv::Archive,
-)]
-pub enum WorkerStackSize {
-    #[default]
-    Default,
-    Size(KiB),
-}


Commit docs(args): basic docs for DpdkDriverConfigSection - it's not clear at all why this is going away.

qmonnet · 2025-11-25T10:11:23Z

Thank you for splitting the first commit. I didn't ask for so many commits, it feels a bit like we went from one extreme to the other 😅. I think most docs commits could remain together, as long as they only add docs. But OK, given the choice between the two options I much prefer the 30+ commits than the big one with all the clean-up mixed together. Thanks for this work!

IMPOSSIBLE TO BREAK DPDK INTO TWO STEPS

Noted. Can we come back to a meaningful PR name, maybe? 🙂

daniel-noland force-pushed the pr/daniel-noland/init-handoff branch 4 times, most recently from 2357024 to 67620ec Compare November 22, 2025 06:09

daniel-noland marked this pull request as ready for review November 22, 2025 06:16

daniel-noland requested a review from a team as a code owner November 22, 2025 06:16

daniel-noland requested review from Fredi-raspall and removed request for a team November 22, 2025 06:16

daniel-noland self-assigned this Nov 22, 2025

daniel-noland added Integration Branch that should NOT be merged but kept stable as other branches depend on it. and removed Integration Branch that should NOT be merged but kept stable as other branches depend on it. labels Nov 22, 2025

daniel-noland added this to the GW R1 milestone Nov 22, 2025

daniel-noland linked an issue Nov 22, 2025 that may be closed by this pull request

Ordered recovery of dpdk integration work #1051

Open

daniel-noland added the enhancement New feature or request label Nov 22, 2025

daniel-noland requested a review from Copilot November 22, 2025 06:20

Copilot started reviewing on behalf of daniel-noland November 22, 2025 06:21 View session

Copilot finished reviewing on behalf of daniel-noland November 22, 2025 06:22

Copilot AI reviewed Nov 22, 2025

View reviewed changes

daniel-noland force-pushed the pr/daniel-noland/init-handoff branch 3 times, most recently from f6a5cf8 to 33ede97 Compare November 22, 2025 07:25

Fredi-raspall reviewed Nov 22, 2025

View reviewed changes

args/src/lib.rs Show resolved Hide resolved

Fredi-raspall reviewed Nov 22, 2025

View reviewed changes

Fredi-raspall approved these changes Nov 22, 2025

View reviewed changes

qmonnet requested changes Nov 24, 2025

View reviewed changes

qmonnet modified the milestones: GW R1, GW R2 Nov 24, 2025

daniel-noland force-pushed the pr/daniel-noland/init-handoff branch from 33ede97 to 4d1cab8 Compare November 25, 2025 01:25

daniel-noland enabled auto-merge November 25, 2025 02:00

daniel-noland added 23 commits November 25, 2025 04:27

docs(args): basic docs for MetricsConfigSection

f5e277e

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

docs(args): basic docs for TracingConfigSection

94e8c69

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

docs(args): basic docs for TracingDisplayOption

acf5e8a

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

docs(args): basic docs for TracingShowSection

7aa3037

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

docs(args): basic docs for RoutingConfigSection

5550224

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

docs(args): basic docs for ConfigServerSection

d39d8d0

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

feat(args): intro ProfilingConfigSection

409c80e

New config file section to set up and manage pyroscope. Signed-off-by: Daniel Noland <daniel@githedgehog.com>

feat(args): introduce inherit method

970978f

This allows a child process to inherit configuration from a parent process. Signed-off-by: Daniel Noland <daniel@githedgehog.com>

feat(args): add defensive annotations on inherit

a0984f1

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

feat(args): intro AsFinalizedMemFd trait (REWORD)

ef6def9

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

feat(args): add must_use annotation on to_owned_fd (REWORD)

f8abfff

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

chore(args): style cleanup for from_fd (REWORD)

99a88b8

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

docs(args): basic docs for FinalizedMemFd::validate

79f2b5c

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

docs(args): basic docs for IntegrityCheckError

172e511

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

docs(args): basic docs for IntegrityCheck

558e2b1

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

docs(args): basic docs for InvalidCmdArguments

e2e6164

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

docs(args): basic docs for LaunchConfiguration

5333e8c

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

docs(args): notes for LaunchConfiguration file descriptor numbers

01c04c0

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

chore(args): cleanup LaunchConfiguration

7175cc4

Fixes for clippy pedantic and general style. Signed-off-by: Daniel Noland <daniel@githedgehog.com>

docs(args): basic docs for CmdArgs

3dfbbf4

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

docs(args): must_use annotations for CmdArgs

84e6114

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

test(args): cleanup tests post rework

2319285

This commit series introduces new functionality which broke the prior tests. This commit amends the tests to account for that. Signed-off-by: Daniel Noland <daniel@githedgehog.com>

chore(args): remove unused import

3018f63

Signed-off-by: Daniel Noland <daniel@githedgehog.com>

daniel-noland marked this pull request as draft November 25, 2025 04:30

auto-merge was automatically disabled November 25, 2025 04:30
Pull request was converted to draft

daniel-noland force-pushed the pr/daniel-noland/init-handoff branch from f3ecb35 to 3018f63 Compare November 25, 2025 04:36

daniel-noland changed the title ~~init handoff~~ IMPOSSIBLE TO BREAK DPDK INTO TWO STEPS Nov 25, 2025

qmonnet reviewed Nov 25, 2025

View reviewed changes

daniel-noland changed the title ~~IMPOSSIBLE TO BREAK DPDK INTO TWO STEPS~~ DPDK attempt 2 Nov 25, 2025

	/// The URL of the pryroscope url
	/// The URL of the pyroscope server

	/// Profileing collection configuration
	/// Profiling collection configuration

		COPY --link --chown=0:0 "${ARTIFACT}" /dataplane
		COPY --link --chown=0:0 "${ARTIFACT}" /dataplane-init


		let launch_config = launch_config.to_owned_fd();

		let io_err = std::process::Command::new("/bin/dataplane")

		/// Number of dataplane worker threads / cores
		pub dataplane_workers: usize,

DPDK attempt 2 #1072

Are you sure you want to change the base?

DPDK attempt 2 #1072

Uh oh!

Conversation

daniel-noland commented Nov 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Nov 22, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Nov 22, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 22, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Fredi-raspall left a comment

Choose a reason for hiding this comment

Uh oh!

qmonnet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

qmonnet commented Nov 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

qmonnet commented Nov 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

daniel-noland commented Nov 22, 2025 •

edited

Loading