fixDarwinDylibNames breaks code signing when building cdylib crates

[mrene]

We have some rust cdylibs that get linked into another application. On aarch64-darwin, these would crash at runtime due to a code signing verification failure.

Since the dylibs id (LC_ID_DYLIB) needs to match with their on-disk location, it's common to use the fixDarwinDylibNames build hook in order to update it to the nix store path, so it doesn't reference the cargo target folder. This takes place in the fixup phase, since the hook is added to fixupOutputHooks.

However, because removeReferencesToVendoredSources added a code signature in the postInstall phase, the LC_ID_DYLIB change invalidates the signature because it changes mach-o headers.

I would propose moving reference removal and code signing into the fixup phase, and conditionally adding fixDarwinDylibNames whenever the stdenv host platform is aarch64-darwin. This way the LC_ID_DYLIB fixup would always happen before code signing.

What do you think?

[ipetkov]

Hi @mrene thanks for the report!

removeReferencesToVendoredSources intentionally runs as a postInstall hook so it can be somewhat agnostic to how/what binaries get installed (e.g. buildPackage uses the cargo build log to determine what to install, but its intentionally configurable to allow a derivation to install things however it pleases yet not have to manually invoke removeReferencesToVendoredSources).

I'm not too familiar with darwin signing and fixups. If fixDarwinDylibNames ends up invalidating code signatures, should it be re-signing things itself (via signIfRequired ...)?