Rework the protobuf ecosystem

[suhailskhan]

Description

This is a follow-up to #28098.

Work in progress; see comment below for description of what this PR is intended to do.

Cc @mascguy

Type(s)

• bugfix
• enhancement
• security fix

Tested on

macOS 15.4.1 24E263 arm64
Xcode 16.3 16E140

Verification

Have you

• followed our Commit Message Guidelines?
• squashed and minimized your commits?
• checked that there aren't other open pull requests for the same change?
• checked your Portfile with port lint?
• tried existing tests with sudo port test?
• tried a full install with sudo port -vst install?
• tested basic functionality of all binary files?
• checked that the Portfile's most important variants haven't been broken?

[skip notification]

[suhailskhan]

The intent of this PR is as follows:

• Move protobuf-cpp to protobuf2-cpp and segregate it; update dependent ports to accommodate for the change
• Segregate protobuf3-cpp
• Create a new, unsegregated protobuf-cpp port that tracks the latest stable release, replacing the segregated protobuf3-cpp-upstream port
• Ports that currently depend on protobuf3-cpp most likely should work with the the new protobuf-cpp port, so modify these ports to use it instead of protobuf3-cpp

So far, this PR only implements the first bullet point. The remaining ones are only planned, and have not been done yet. This is only a proposal for one possible approach, I am open to scrapping this and taking a different approach based on feedback.

[mascguy]

The intent of this PR is as follows:

• Move protobuf-cpp to protobuf2-cpp and segregate it; update dependent ports to accommodate for the change
• Segregate protobuf3-cpp
• Create a new, unsegregated protobuf-cpp port that tracks the latest stable release, replacing the segregated protobuf3-cpp-upstream port
• Ports that currently depend on protobuf3-cpp most likely should work with the the new protobuf-cpp port, so modify these ports to use it instead of protobuf3-cpp

So far, this PR only implements the first bullet point. The remaining ones are only planned, and have not been done yet. This is only a proposal for one possible approach, I am open to scrapping this and taking a different approach based on feedback.

I definitely support this.

One of our older tickets has plenty of details regarding the challenges for some of our older ports:

https://trac.macports.org/ticket/57117

But as long as we take our time, and work through the details, this would be a nice improvement.

I'm going to try and pull these changes down over the next few days, and review how everything fits together.

And finally, thank you so much for taking the initiative on this!

[suhailskhan]

So far, this PR only implements the first bullet point.

I’ll share some details on the status of this.

protobuf-cpp/protobuf2-cpp has three dependent ports: protobuf-java, py27-protobuf, and shogun-devel. As of opening the PR yesterday, I verified the successful build of protobuf-java, but py27-protobuf and shogun-devel were still broken.

Just now, I got py27-protobuf to build, and I force pushed a few minutes ago. The test fails, but it appears this port’s test may have been broken for some time now.

I cannot test shogun-devel as it has many dependencies that are “known to fail.” Neither are any of the Buildbots able to build this port as is, anyway.

Overall, I think whatever worked before, still works, and what doesn’t work, continues to not work. So I believe this phase is successful, unless protobuf-java and/or py27-protobuf fail to function as they did before despite successful builds.

[suhailskhan]

I have begun the process of updating protobuf3-cpp’s dependents to use protobuf-cpp, starting with atuin.

I am looking at apache-arrow next, which depends on grpc, another dependent of protobuf3-cpp.

grpc is outdated, so its version will need to be bumped in addition to updating its dependencies.

[suhailskhan]

protobuf-cpp will need to be updated to 6.30.2, but for that the patches will need to be updated as well.

Once that is done, we may want to add a segregated protobuf6-cpp port.

[mascguy]

protobuf-cpp will need to be updated to 6.30.2, but for that the patches will need to be updated as well.

Once that is done, we may want to add a segregated protobuf6-cpp port.

Yep, for sure!

[suhailskhan]

perhaps this is an opportunity to have a new Python protobuf port that tracks the latest release, like protobuf-cpp will be doing.

Though for now, I suppose the focus should remain on updating protobuf3-cpp’s dependents.

I have done this now. I moved py-protobuf to py-protobuf2, py-protobuf3 stays as is, and py-protobuf tracks the distribution on PyPI. For the most part it seems like the new py-protobuf works as a drop-in replacement for py-protobuf3 for dependent ports, at least so far.

[suhailskhan]

There is only one PR left to get merged, so I will let the CI checks run now. astroid is the expected failure until its PR is merged.

[mascguy]

There is only one PR left to get merged, so I will let the CI checks run now. astroid is the expected failure until its PR is merged.

Sounds good, we'll be able to merge the last PR at the 72-hour mark. (Or sooner, if the maintainer approves.)