feat(k8s-intf): Generate Rust types for gateway-agent CRD #1058
Conversation
mvachhar
force-pushed
the
pr/mvachhar/k8s-direct
branch
4 times, most recently
from
9ffda18 to
1fb7102
Compare
mvachhar
requested review from
Frostman,
Copilot and
qmonnet
and removed request for
sergeymatov
There was a problem hiding this comment.
Pull Request Overview
This PR introduces the k8s-direct crate to enable the dataplane to communicate directly with Kubernetes, eliminating the need for the gateway agent and the gateway-proto repository. The key changes include:
- Adding build infrastructure that uses kopium to generate Rust types from the gateway-agent CRD
- Setting up automated CRD fetching and code generation during the build process
- Creating the foundation for future direct Kubernetes integration
Reviewed Changes
Copilot reviewed 6 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/k8s-crd.env | Defines the gateway reference and CRD URL for fetching the gateway-agent CRD definition |
| k8s-direct/src/lib.rs | Main library file exposing the generated gateway-agent CRD types |
| k8s-direct/build.rs | Build script that fetches the CRD and generates Rust types using kopium |
| k8s-direct/Cargo.toml | Package configuration with dependencies for Kubernetes interaction and code generation |
| k8s-direct/.gitignore | Excludes the kopium binary installation directory from version control |
| Cargo.toml | Workspace configuration updates adding the k8s-direct crate and required dependencies |
mvachhar
force-pushed
the
pr/mvachhar/k8s-direct
branch
from
1fb7102 to
a456785
Compare
mvachhar
force-pushed
the
pr/mvachhar/k8s-direct
branch
from
a456785 to
566e7ef
Compare
qmonnet
left a comment
qmonnet
left a comment
There was a problem hiding this comment.
We miss a step in CI workflow to make sure that there's no git diff on the generated file after rebuilding the project, or we risk missing updates.
Actually, I think we should commit the source file for generating gateway_agent_crd.rs, rather than the generated file. It means more churn when we update the file (although we could automate bump PR creation), but would feel safer and more deterministic - we'd be sure to use the source in the repo, not something that might diverge in time.
mvachhar
force-pushed
the
pr/mvachhar/k8s-direct
branch
3 times, most recently
from
8a13a73 to
7f5846b
Compare
scripts/k8s-crd.env
Outdated
| @@ -0,0 +1,2 @@ | |||
| K8S_GATEWAY_AGENT_REF=master | |||
There was a problem hiding this comment.
I think it's a bad idea to use master as you'll get random rebuilds and dirty repo just by running the build if gateway CRD will change in the background. You can use latest tag v0.28.0 instead and we can later add auto-bump job to the gateway repo, but I'm okay if you want to keep that behavior
There was a problem hiding this comment.
Actually no, I'm not okay with that as it may cause dirty state in CI, which e.g. will cause tagged releases to be published with -dirty suffix in the version
There was a problem hiding this comment.
I'll use a sha or tag here.
mvachhar
changed the title
Frostman
left a comment
Frostman
left a comment
There was a problem hiding this comment.
See comment about using master for getting CRD
This is in support of githedgehog/dataplane#1058 Signed-off-by: Daniel Noland <daniel@githedgehog.com>
This is in support of githedgehog/dataplane#1058 Signed-off-by: Daniel Noland <daniel@githedgehog.com>
This is in support of githedgehog/dataplane#1058 Signed-off-by: Daniel Noland <daniel@githedgehog.com>
mvachhar
force-pushed
the
pr/mvachhar/k8s-direct
branch
from
7f5846b to
da240ba
Compare
mvachhar
force-pushed
the
pr/mvachhar/k8s-direct
branch
from
da240ba to
ce34f27
Compare
Frostman
left a comment
Frostman
left a comment
There was a problem hiding this comment.
Small nit on the package name, but overall looks reasonable
Will rename |
mvachhar
force-pushed
the
pr/mvachhar/k8s-direct
branch
from
ce34f27 to
af2bdf8
Compare
qmonnet
left a comment
qmonnet
left a comment
There was a problem hiding this comment.
We're also still missing a CI step to check that there's no diff for the generated file after re-building the package.
mvachhar
force-pushed
the
pr/mvachhar/k8s-direct
branch
2 times, most recently
from
439bde0 to
b4d362a
Compare
rustls uses webpki which has the CDLA-Permissive-2.0 license. Allow this license so we can add build.rs code to fetch the gateway-agent CRD in a future release. Signed-off-by: Manish Vachharajani <manish@githedgehog.com>
This is the first step to getting rid of the gateway agent and having dataplane talk directly to k8s. This should greatly simplify the code, eliminate the gRPC conversion and eliminate the entire gateway-proto repository. This commit adds the k8s-intf crate with build infrastructure that uses kopium to generate the Rust types for the CRD for the gateway-agent that is in the gateway repository. The next steps will use the CRD to fetch and parse data from k8s, and then translate it into the inner config format of the dataplane. Signed-off-by: Manish Vachharajani <manish@githedgehog.com>
Build should not generate any diffs or untracked files. Signed-off-by: Manish Vachharajani <manish@githedgehog.com>
mvachhar
force-pushed
the
pr/mvachhar/k8s-direct
branch
from
b4d362a to
a9e4a84
Compare
Done. |
This is the first step to getting rid of the gateway agent and having dataplane talk directly to k8s. This should greatly simplify the code, eliminate the gRPC conversion and eliminate the entire gateway-proto repository.
This commit adds the k8s-direct crate with build
inrastructure that uses kopium to generate the Rust types for the CRD for the gateway-agent that is in the gateway repository.
The next steps will use the CRD to fetch and parse data from k8s, and then translate it into the inner config format of the dataplane.
Fixes #1054