"linux: clarify pids cgroup settings" is causing widespread difficulty in adopting runtime-spec 1.3.0

[samuelkarp]

(Note: this issue is a personal perspective and does not represent a stance of the OCI TOB)

869b2d5 changed LinuxPid from int64 to *int64. While the description in the commit is logical, the effect is that this creates a cascade across ecosystem packages that depend on the runtime-spec's Go types. Here is a sampling of issues:

• client: clarify and rename WithRootFS containerd/containerd#1249 (this was some copy-paste error, not sure what the link was I had intended to include)
• deps: bump runtime-spec to v1.3.0. containerd/nri#243
• runtime-spec: bump to v1.3.0, fixup for backward incompatible linux PIDs limit type change . runtime-tools#795
• build(deps): bump github.com/containerd/cgroups/v3 from 3.1.0 to 3.1.1 containerd/containerd#12465
• Bump github.com/opencontainers/runtime-spec from 1.2.1 to 1.3.0 NVIDIA/nvidia-container-toolkit#1423
• fix(deps): update module github.com/opencontainers/runtime-spec to v1.3.0 containers/container-libs#437
• Update module github.com/opencontainers/runtime-spec to v1.3.0 containers/podman#27440

It may be too late to unwind the change in 1.3.0 (and revert in 1.3.1?), but this was certainly a breaking change from a Go library perspective even if it wasn't from an actual spec perspective. Since this repo serves as both the home for the spec (and versions the spec according to semver) and the Go library, it's possible to introduce breaking changes in one but not the other.

I think it would be helpful to either (a) create a policy regarding the level of acceptability for breaking changes in the Go library, or (b) separate the Go library from the spec so it can be versioned independently, or (c) discourage the use of this official Go library.

[utam0k]

I think it's healthier to maintain compatibility with specifications and libraries separately. I prefer (a) or (b).

[utam0k]

@opencontainers/runtime-tools-maintainers What does everyone think?

[cyphar]

I think that having it separate would better express that SemVer only applies to our specifications, not the Go bindings.

However, I'm not quite sure how just having a separate repo would've helped here -- would the idea be we would've tagged a new major version (to do /v2 imports) to avoid build breakages? Would that not cause a different kind of confusion (/v12 of the bindings is actually runtime-spec v1.8.0)? Or would we do something like /vN is runtime-spec v1.N.0?

My probably-quite-controversial opinion is that build breakages which occur when you do an update are not as big of a deal as other kinds of subtle breakages that SemVer is primarily concerned with (i.e., breaking behaviour changes that still compile). The way that Go selects versions can lead to confusing breakages (updating a package could cause a different package to break and this could even be triggered by a third library changing their minimum version requirement) that I think are more problematic, but they all require you to update go.sum for the breakage to occur and it is detectable at build time.

[tianon]

I think that roughly maps to (a), but with the policy being "best effort avoiding breaking changes in Go but really no promises at all because the version is the spec version" and I think that matches the reality of how we've maintained this (it's really a shame that Go decided to be so dogmatic here and doesn't have a way for us to say "no, these often preexisting tags aren't actually semantically meaningful for Go").

If we'd had a separate repo (or module) it would only defer the breaking change, not prevent it, and be more overhead on making sure the spec and the Go stay in sync.

[samuelkarp]

However, I'm not quite sure how just having a separate repo would've helped here

Separate repo or separate tags; you can have multiple distinct modules in the same git repo with their own versions. But that's beside the point a bit...

would the idea be we would've tagged a new major version (to do /v2 imports) to avoid build breakages? Would that not cause a different kind of confusion (/v12 of the bindings is actually runtime-spec v1.8.0)? Or would we do something like /vN is runtime-spec v1.N.0?

I'm not really proposing a specific solution here. /v2 or /vN could be a possibility, though the downsides you describe around confusion are definitely real. I think the issue for this particular change is that it wasn't really even a spec change at all, it was just an incompatible Go change.