lib.makeOverridable: observe <result>.__overriders

[ShamrockLee]

This PR solves lib.makeOverridable's not handling existing overriders beside overrideAttrs by introducing the __overriders attribute.

Such attribute contains the names of existing overriders of the attribute set.
When a lib.makeOverridable-like function adds a new overrider, it can check result.__overriders, decorates existing overriders, and add the name of the overriders it provide to the __overriders list.

This is an extensible solution/workaround to issues like <set>.override lost after <set>.overrideScope overriding (#447012) without substantial infrastructural changes (like what #273815 proposes).

This PR helps unblock improve PR #445668. Cc: @LunNova @bb010g

Things done

• Built on platform:
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• Tested, as applicable:
  • NixOS tests in nixos/tests.
  • Package tests at passthru.tests.
  • Tests in lib/tests or pkgs/test for functions and "core" functionality.
• Ran nixpkgs-review on this PR. See nixpkgs-review usage.
• Tested basic functionality of all binary files, usually in ./result/bin/.
• Nixpkgs Release Notes
  • Package update: when the change is major or breaking.
• NixOS Release Notes
  • Module addition: when adding a new NixOS module.
  • Module update: when the change is significant.
• Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

---

Add a 👍 reaction to pull requests you find important.

[ShamrockLee]

I added a test case in lib/tests/misc.nix.

[emilazy]

Thanks for looking into this!

FWIW, I think this shouldn’t block #445668 because llvmPackages.overrideScope didn’t previously exist – so its result not having .override is not a critical issue, just an unfortunate one. My review comments there were only about the need to ensure the Darwin stdenv’s use of it don’t clobber it, and to ensure that we can provide an .override that actually works there, but it seems that can be achieved by hand for now.

However, solving this problem in general is of course important. I hope you don’t mind me wanting to take some time to think about this approach, as this area of things is relevant to the by-name work I’m doing with @wolfgangwalther, and I worry a little about adding this to 25.11 and then potentially wanting to break it shortly after.

(To be clear, though, I don’t want to punt on this indefinitely, and although #273815 has been relevant to my thinking, I would not want to couple the kinds of solutions I’m thinking about to a meaningful breaking change in the stdenv.mkDerivation interface – I am hoping we can achieve something relatively general that preserves good backwards compatibility with our existing APIs, but I don’t object to smaller changes that work well.)

[emilazy]

An interesting option here would be to explicitly allow override here, which could potentially be used to allow migrating foo = bar.override { … }; definitions into by-name without breaking override interfaces. However, there are other complications there, with regards to the mirrorArgs stuff. (I got something to allow composing override interfaces working locally, but wasn’t entirely happy with the trade-offs.)

[ShamrockLee]

An interesting option here would be to explicitly allow override here

Could you please elaborate?

We cannot implement the override attribute with overrideResult in lib.makeOverridable. It will always get overwritten.

As for overrideAttrs, we could do it here instead of defining explicitly above, but I'm worried about the potential performance implication.

[emilazy]

I can’t directly recall what I was thinking at the time, but I believe that what I meant was that we could allow packages to expose their own .override field rather than always overriding it, which lets you callPackage/.override some other package in by-name without having the usual callPackage override interface clobber the one you’d like to inherit. (Hence why we have some overrides directly in all-packages.nix still.) Though in further thinking about this whole class of problem I decided it was more complicated than that, because sometimes you don’t want to clobber the callPackage override interface in that way.

[ShamrockLee]

FWIW, I think this shouldn’t block #445668 because llvmPackages.overrideScope didn’t previously exist – so its result not having .override is not a critical issue, just an unfortunate one. My review comments there were only about the need to ensure the Darwin stdenv’s use of it don’t clobber it, and to ensure that we can provide an .override that actually works there, but it seems that can be achieved by hand for now.

Thank you for explaining and clarifying it! The discussion thread is quite long under that PR, and I only have a vague impression about it.

However, solving this problem in general is of course important. I hope you don’t mind me wanting to take some time to think about this approach, as this area of things is relevant to the by-name work I’m doing with @wolfgangwalther, and I worry a little about adding this to 25.11 and then potentially wanting to break it shortly after.

This change affects the whole pkgs, and changes like this inherently require more consensus and review effort. It's sensible to defer its merge after the 25.11 release.