Update docker image p4lang/third-party in to support proto editions

[verios-google]

I would like to migrate p4runtime.proto from proto3 to proto editions in order to add proto extensions to parts of the proto (this will be a future PR). Extensions are only supported by proto2 and proto editions

In order to support editions, protobuf dependencies have to be updated to their latest versions. I know how to handle the bazel updates, but I have no idea how to update the docker base image p4lang/third-party used in codegen/Dockerfile.

@antoninbas do you by any chance know how to update the docker image?

[antoninbas]

The image is built from this source code: https://github.com/p4lang/third-party

Maybe check with @jafingerhut as he was working on bumping up the Ubuntu version in the image to 24.04.

[jafingerhut]

I have done a few hours of experiments working on this, but have not yet found changes that work across all of the repos p4runtime, PI, behavioral-model, and p4c. If someone else wants to take a crack at it, they are likely to beat me to it if they start soon.

I am happy to share my false start at it if they think it might help them, but I'm not sure that it would.

[verios-google]

Please do share the false start @jafingerhut. Even if it's wrong, it may give me insight into how to change a docker image (I have no idea what I need to touch).

[verios-google]

Also, what are the chances we can just move away from Docker support completely @smolkaj, @jafingerhut, @antoninbas, @chrispsommers. Given that the docker base image hasn't been updated in 3 years, it seems like it has been forgotten

[jafingerhut]

If I understand what we have now, it was created by Antonin Bas, and has the following "flow":

• This Dockerfile starts with a base ubuntu20.04 Docker image, and adds some packages like grpc, thrift, and a few others developed outside of the p4lang organization, and are major dependencies used by some p4lang projects.
  • Dockerfile: https://github.com/p4lang/third-party/blob/main/Dockerfile
  • CI script where this Dockerfile is used: https://github.com/p4lang/third-party/blob/main/.github/workflows/build.yml
  • output Docker image name: p4lang/third-party
• This Dockerfile starts from p4lang/third-party image, and adds built code from the p4lang/PI repo:
  • Dockerfile: https://github.com/p4lang/PI/blob/main/Dockerfile
  • CI script: https://github.com/p4lang/PI/blob/main/.github/workflows/build.yml
  • output Docker image name: p4lang/pi
• This Dockerfile starts from p4lang/pi, and adds built code from the p4lang/behavioral-model repo:
  • Dockerfile: https://github.com/p4lang/behavioral-model/blob/main/Dockerfile
  • CI script: https://github.com/p4lang/behavioral-model/blob/main/.github/workflows/build.yml
  • output Docker image name: p4ang/behavioral-model

At one time, the p4lang/p4c repo used that p4lang/behavioral-model Docker image to run p4c CI tests, but I think it has been a while since it switched to using Ubuntu deb packages installed via the apt-get command. Unfortunately those deb packages have not been updated since August 2023. The person who was generating them appears to have moved on to other things, and no one else has chosen to update them.

Recently I have propopsed to the P4 Governance Board an idea from Fabian Ruffy: hire someone part time whose main job will be to maintain the CI infrastructure of multiple p4lang projects, including at least all of the ones mentioned above. Sure, if volunteers are willing to do it, great, or if companies are willing to pay people to do this part-time, excellent, but that has been a very short list of people over the last 5 years (mostly Antonin Bas and Fabian Ruffy). The P4 Governance Board sounds open to the idea of paying some money for a qualified person to work on this, but selecting a qualified person has only barely started, and I do not know how long it will take.

All of that said, if there is some specific flow of compute jobs that you are interested in creating automation for, in a way that you believe others will be able to maintain and enhance later, let us know. There are many choices besides Docker, I am sure, but I don't have good enough knowledge of the alernatives to compare and contrast those choices.

[chrispsommers]

Andy, thanks for the analysis. I agree that this needs some dedicated resource, since the volunteer pool has shrunken/moved on. A comprehensive analysis across multiple projects seems worthwhile, followed by a set of choices of how to proceed. Then the TST can make some choices. Hopefully it won't require a major overhaul, simple a refresh. I believe the Docker-based approach still makes sense (it's still mainstream and widely understood), but another perspective is always welcome.

[jafingerhut]

I copied my comment above, and added some links to work-in-progress, but not yet working, modified Dockerfiles that attempt to start with Ubuntu 24.04 instead of Ubuntu 20.04 as the current p4lang DOckerfiles do: p4lang/project-ideas#38

[chrispsommers]

Also, what are the chances we can just move away from Docker support completely

Bear in mind there may be 100's or 1000's of users and many OSS projects which have baked the Docker approach into their workflows, so I suggest we simply uplift it, not forklift it. The blast radius is huge.