From 125d73139f2bd9745bfecb8a476c205302537cbf Mon Sep 17 00:00:00 2001 From: purepani Date: Sun, 26 May 2024 21:29:37 -0500 Subject: [PATCH 1/5] buildDotnetModule: allow fetching nuget dependencies at compile time This allows user to specify a hash for the dependencies using the "nugetSha256" attribute, instead of having to manually generate a lockfile. This is done by generating a lockfile with `dotnet restore`, and then parsing the requested version ranges to see if anything is floating. Afterwards we generate a nix-based lockfile containing hashes and stable downloads for each dependency, which we can IFD. The checksum of this lockfile is specified with "nugetSha256". We want to use the checksum of the nix-based lockfile instead of hashing the entire nuget source so that we can independantly fetch all dependencies, and re-use them across derivations. Co-authored-by: Valentin Gagarin --- doc/languages-frameworks/dotnet.section.md | 29 +++++--- .../dotnet/build-dotnet-module/default.nix | 46 ++++++++---- .../dotnet/build-dotnet-module/fetch-deps.nix | 73 +++++++++++++++++++ .../build-dotnet-module/hooks/default.nix | 11 ++- .../hooks/dotnet-configure-hook.sh | 12 +++ .../hooks/dotnet-validate-lockfile.sh | 67 +++++++++++++++++ 6 files changed, 214 insertions(+), 24 deletions(-) create mode 100644 pkgs/build-support/dotnet/build-dotnet-module/fetch-deps.nix create mode 100644 pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-validate-lockfile.sh diff --git a/doc/languages-frameworks/dotnet.section.md b/doc/languages-frameworks/dotnet.section.md index 36c20a9e9c507..34f84bc177a3d 100644 --- a/doc/languages-frameworks/dotnet.section.md +++ b/doc/languages-frameworks/dotnet.section.md @@ -2,7 +2,7 @@ ## Local Development Workflow {#local-development-workflow} -For local development, it's recommended to use nix-shell to create a dotnet environment: +For local development, it's recommended to use `nix-shell` to create a dotnet environment: ```nix # shell.nix @@ -93,11 +93,11 @@ The `dotnetCorePackages.sdk` contains both a runtime and the full sdk of a given To package Dotnet applications, you can use `buildDotnetModule`. This has similar arguments to `stdenv.mkDerivation`, with the following additions: * `projectFile` is used for specifying the dotnet project file, relative to the source root. These have `.sln` (entire solution) or `.csproj` (single project) file extensions. This can be a list of multiple projects as well. When omitted, will attempt to find and build the solution (`.sln`). If running into problems, make sure to set it to a file (or a list of files) with the `.csproj` extension - building applications as entire solutions is not fully supported by the .NET CLI. -* `nugetDeps` takes either a path to a `deps.nix` file, or a derivation. The `deps.nix` file can be generated using the script attached to `passthru.fetch-deps`. If the argument is a derivation, it will be used directly and assume it has the same output as `mkNugetDeps`. +* `nugetDeps` takes either a path to a `deps.nix` file, or a derivation, in case we cannot deterministically do so. The `deps.nix` file can be generated using the script attached to `passthru.fetch-deps`. If the argument is a derivation, it will be used directly and assume it has the same output as `mkNugetDeps`. This option is mutually exclusive with the `nugetSha256` attribute. ::: {.note} For more detail about managing the `deps.nix` file, see [Generating and updating NuGet dependencies](#generating-and-updating-nuget-dependencies) ::: - +* `nugetSha256` is used to specify the hash of the generated Nix-compatible lockfile containing all NuGet dependencies. * `packNupkg` is used to pack project as a `nupkg`, and installs it to `$out/share`. If set to `true`, the derivation can be used as a dependency for another dotnet project by adding it to `projectReferences`. * `projectReferences` can be used to resolve `ProjectReference` project items. Referenced projects can be packed with `buildDotnetModule` by setting the `packNupkg = true` attribute and passing a list of derivations to `projectReferences`. Since we are sharing referenced projects as NuGets they must be added to csproj/fsproj files as `PackageReference` as well. For example, your project has a local dependency: @@ -126,7 +126,15 @@ For more detail about managing the `deps.nix` file, see [Generating and updating * `dotnetPackFlags` can be used to pass flags to `dotnet pack`. Used only if `packNupkg` is set to `true`. * `dotnetFlags` can be used to pass flags to all of the above phases. -When packaging a new application, you need to fetch its dependencies. Create an empty `deps.nix`, set `nugetDeps = ./deps.nix`, then run `nix-build -A package.fetch-deps` to generate a script that will build the lockfile for you. +When packaging a new application, you need to fetch and lock its Nuget dependencies. `buildDotnetModule` has two ways of handling this: +* Using `nugetSha256`. We try to generate a Nix-compatible lockfile at build time, of which you can specify the hash with `nugetSha256`. When you try to build the derivation, everything gets fetched automatically and the expected hash gets printed out. We try this by default when neither `nugetSha256` nor `nugetDeps` are set. This is the recommended way to resolve dependencies. The passthru attribute `nuget-lockfile` is exposed which builds this lockfile for you. The downside of this method is that it only works if no floating versions are requested upstream – otherwise an error will be thrown and you will need to use `nugetDeps` instead. +* Using `nugetDeps`. This method requires you to manually generate a Nix-compatible lockfile, which has to be checked into source control. You can run `nix-build -A package.fetch-deps` to build a script which generates the lockfile. After running said script you can set the `nugetDeps` attribute to the path of the generated lockfile. Note that if you change any flags affecting the restore process, you will need to regenerate this script. This takes more work to maintain, so you should use `nugetSha256` instead when possible. + +Note that both of these methods use the flags and attributes from your derivation, and attempt to mirror the restore process from the package itself. That means that if you for example set `dotnetRestoreFlags`, it will be used both for fetching dependencies and for building the derivation. + +If you change the `dotnet-sdk` attribute, or set a flag effecting `dotnet restore`, you will most likely need to regenerate the lockfile. + +When updating an existing application, you should *always* regenerate its lockfile. Here is an example `default.nix`, using some of the previously discussed arguments: ```nix @@ -141,21 +149,22 @@ in buildDotnetModule rec { src = ./.; projectFile = "src/project.sln"; - # File generated with `nix-build -A package.passthru.fetch-deps`. - # To run fetch-deps when this file does not yet exist, set nugetDeps to null - nugetDeps = ./deps.nix; + + nugetSha256 = "sha256-8FBfEhXmnfk/Y2Cd5jWoKgSS/HbEJyBXV5jOTyR+ft8="; # Generated automatically. + nugetDeps = ./deps.nix; # File generated with `nix-build -A package.passthru.fetch-deps`, this is *not* recommanded. projectReferences = [ referencedProject ]; # `referencedProject` must contain `nupkg` in the folder structure. - dotnet-sdk = dotnetCorePackages.sdk_6_0; - dotnet-runtime = dotnetCorePackages.runtime_6_0; + dotnet-sdk = dotnetCorePackages.sdk_7_0; + dotnet-runtime = dotnetCorePackages.runtime_7_0; + dotnetFlags = [ "--runtime linux-x64" ]; # Note: this usually is not required. executables = [ "foo" ]; # This wraps "$out/lib/$pname/foo" to `$out/bin/foo`. executables = []; # Don't install any executables. packNupkg = true; # This packs the project as "foo-0.1.nupkg" at `$out/share`. - runtimeDeps = [ ffmpeg ]; # This will wrap ffmpeg's library path into `LD_LIBRARY_PATH`. + runtimeDeps = [ ffmpeg ]; # This will wrap ffmpeg into `LD_LIBRARY_PATH`. } ``` diff --git a/pkgs/build-support/dotnet/build-dotnet-module/default.nix b/pkgs/build-support/dotnet/build-dotnet-module/default.nix index 6f5df2d34b882..00e02acb69f25 100644 --- a/pkgs/build-support/dotnet/build-dotnet-module/default.nix +++ b/pkgs/build-support/dotnet/build-dotnet-module/default.nix @@ -5,11 +5,12 @@ , srcOnly , linkFarmFromDrvs , symlinkJoin +, nuget-to-nix +, writeText , makeWrapper , dotnetCorePackages , mkNugetSource , mkNugetDeps -, nuget-to-nix , cacert , coreutils , runtimeShellPackage @@ -17,8 +18,11 @@ { name ? "${args.pname}-${args.version}" , pname ? name +, meta ? { platforms = dotnet-sdk.meta.platforms; maintainers = []; } + , enableParallelBuilding ? true , doCheck ? false + # Flags to pass to `makeWrapper`. This is done to avoid double wrapping. , makeWrapperArgs ? [ ] @@ -64,6 +68,13 @@ # platforms in meta.platforms which are supported by the sdk. , runtimeId ? null +# The hash of the generated nix-based lockfile containing all Nuget dependencies. This depends on no floating version ranges being requested upstream, as we cannot deterministically fetch those. +# If any floating version is requested, an error will be thrown. Note that this option is mutually exclusive with the `nugetDeps` attribute. +, nugetSha256 ? null +# Don't expose any Nuget source to `dotnet restore`. +, dontSetNugetSource ? false + + # Tests to disable. This gets passed to `dotnet test --filter "FullyQualifiedName!={}"`, to ensure compatibility with all frameworks. # See https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test#filter-option-details for more details. , disabledTests ? [ ] @@ -84,22 +95,32 @@ , dotnet-sdk ? dotnetCorePackages.sdk_6_0 # The dotnet runtime to use. , dotnet-runtime ? dotnetCorePackages.runtime_6_0 -, ... -} @ args: + +, ... } @ args: + +assert projectFile == null -> throw "Defining the `projectFile` attribute is required. This is usually an `.csproj`, or `.sln` file."; +assert (nugetDeps != null && nugetSha256 != null) -> throw "The attributes `nugetDeps` and 'nugetSha256' are mutually exclusive!"; let platforms = if args ? meta.platforms then lib.intersectLists args.meta.platforms dotnet-sdk.meta.platforms else dotnet-sdk.meta.platforms; + # Nuget packages provided by the SDK, these get excluded from the generated lockfile + sdkExclusions = writeText "${dotnet-sdk.name}-exclusions" (lib.concatStringsSep "\n" (dotnet-sdk.passthru.packages { fetchNuGet = attrs: attrs.pname; })); inherit (callPackage ./hooks { - inherit dotnet-sdk disabledTests nuget-source dotnet-runtime runtimeDeps buildType; + inherit dotnet-sdk disabledTests nuget-source dotnet-runtime runtimeDeps buildType dontSetNugetSource; runtimeId = if runtimeId != null then runtimeId else dotnetCorePackages.systemToDotnetRid stdenvNoCC.targetPlatform.system; - }) dotnetConfigureHook dotnetBuildHook dotnetCheckHook dotnetInstallHook dotnetFixupHook; + }) dotnetConfigureHook dotnetValidateLockfileHook dotnetBuildHook dotnetCheckHook dotnetInstallHook dotnetFixupHook; + + fetchImpureDeps = callPackage ./fetch-deps.nix { + inherit name meta dotnet-sdk nugetSha256 projectFile testProjectFile dotnetFlags dotnetRestoreFlags enableParallelBuilding dotnetValidateLockfileHook sdkExclusions; + src = srcOnly args; + }; localDeps = if (projectReferences != [ ]) @@ -107,14 +128,13 @@ let else null; _nugetDeps = - if (nugetDeps != null) then - if lib.isDerivation nugetDeps + let + importedDeps = if (nugetDeps == null) + then fetchImpureDeps + else nugetDeps; + in if lib.isDerivation nugetDeps then nugetDeps - else mkNugetDeps { - inherit name; - sourceFile = nugetDeps; - } - else throw "Defining the `nugetDeps` attribute is required, as to lock the NuGet dependencies. This file can be generated by running the `passthru.fetch-deps` script."; + else mkNugetDeps { inherit name; nugetDeps = import importedDeps; }; # contains the actual package dependencies dependenciesSource = mkNugetSource { @@ -188,6 +208,7 @@ stdenvNoCC.mkDerivation (args // { propagatedSandboxProfile = toString dotnet-runtime.__propagatedSandboxProfile; passthru = { + nuget-lockfile = fetchImpureDeps; inherit nuget-source; } // lib.optionalAttrs (!lib.isDerivation nugetDeps) { fetch-deps = @@ -209,7 +230,6 @@ stdenvNoCC.mkDerivation (args // { in writeShellScript "fetch-${pname}-deps" '' set -euo pipefail - export PATH="${lib.makeBinPath [ coreutils runtimeShellPackage dotnet-sdk (nuget-to-nix.override { inherit dotnet-sdk; }) ]}" for arg in "$@"; do diff --git a/pkgs/build-support/dotnet/build-dotnet-module/fetch-deps.nix b/pkgs/build-support/dotnet/build-dotnet-module/fetch-deps.nix new file mode 100644 index 0000000000000..b7a2822e5e45b --- /dev/null +++ b/pkgs/build-support/dotnet/build-dotnet-module/fetch-deps.nix @@ -0,0 +1,73 @@ +{ lib +, buildDotnetModule +, dotnetValidateLockfileHook +, nuget-to-nix +, nugetSha256 + +, src +, name +, meta +, dotnet-sdk +, projectFile +, testProjectFile +, dotnetFlags +, dotnetRestoreFlags +, enableParallelBuilding +, sdkExclusions +} @ args: + +buildDotnetModule rec { + name = "${args.name}-nuget-lockfile"; + + inherit src dotnet-sdk projectFile testProjectFile dotnetFlags dotnetRestoreFlags enableParallelBuilding; + + nativeBuildInputs = [ dotnetValidateLockfileHook ]; + + generateLockfile = true; + dontSetNugetSource = true; + dontDotnetFixup = true; + + impureEnvVars = lib.fetchers.proxyImpureEnvVars; + outputHashAlgo = "sha256"; + outputHashMode = "flat"; + outputHash = if (nugetSha256 != null) + then nugetSha256 + else ""; # This needs to be set for networking, an empty string prints the "empty hash found" warning + + preConfigure = '' + dotnetRestoreFlags+=( + --packages "$HOME/nuget-pkgs" + ) + + echo "Evaluating and fetching Nuget dependencies" + ''; + + buildPhase = '' + runHook preBuild + + NUGET_DEPS="$HOME/deps.nix" + echo "Writing lockfile..." + ${nuget-to-nix}/bin/nuget-to-nix "$HOME/nuget-pkgs" "${sdkExclusions}" > "$NUGET_DEPS" + + runHook postBuild + ''; + + installPhase = '' + runHook preInstall + + cp "$NUGET_DEPS" $out + echo "Installed lockfile to: $out" + + runHook postInstall + ''; + + # This is the last phase that runs before we error out about the hash being wrong + postFixup = lib.optionalString (nugetSha256 == null) '' + echo "Please set nugetSha256 to the hash below!" + ''; + + meta = { + description = "A lockfile containing the Nuget dependencies for ${name}"; + inherit (args.meta) maintainers platforms; + }; +} diff --git a/pkgs/build-support/dotnet/build-dotnet-module/hooks/default.nix b/pkgs/build-support/dotnet/build-dotnet-module/hooks/default.nix index 44091604f5c2c..eea5b1cc6f464 100644 --- a/pkgs/build-support/dotnet/build-dotnet-module/hooks/default.nix +++ b/pkgs/build-support/dotnet/build-dotnet-module/hooks/default.nix @@ -7,6 +7,7 @@ , callPackage , makeSetupHook , makeWrapper +, jq , dotnet-sdk , disabledTests , nuget-source @@ -14,6 +15,7 @@ , runtimeDeps , buildType , runtimeId +, dontSetNugetSource }: assert (builtins.isString runtimeId); @@ -24,8 +26,9 @@ in dotnetConfigureHook = makeSetupHook { name = "dotnet-configure-hook"; + deps = [ dotnet-sdk ]; substitutions = { - nugetSource = nuget-source; + nugetSource = lib.optional (!dontSetNugetSource) nuget-source; dynamicLinker = "${stdenv.cc}/nix-support/dynamic-linker"; libPath = lib.makeLibraryPath [ stdenv.cc.cc.lib @@ -39,6 +42,12 @@ in } ./dotnet-configure-hook.sh; + dotnetValidateLockfileHook = makeSetupHook + { + name = "dotnet-validate-lockfile-hook"; + deps = [ jq ]; + } ./dotnet-validate-lockfile.sh; + dotnetBuildHook = makeSetupHook { name = "dotnet-build-hook"; diff --git a/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-configure-hook.sh b/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-configure-hook.sh index 3eb0d4e1f2309..90822cb51e15f 100644 --- a/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-configure-hook.sh +++ b/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-configure-hook.sh @@ -13,6 +13,15 @@ dotnetConfigureHook() { local -r parallelFlag="--disable-parallel" fi + if [ -z "${dontSetNugetSource-}" ]; then + nugetSourceFlag="--source @nugetSource@/lib" + fi + + if [ "${generateLockfile-}" ]; then + lockfilePath="$HOME/lockfile.json" + lockfileFlag="--use-lock-file --lock-file-path ${lockfilePath}" + fi + dotnetRestore() { local -r project="${1-}" dotnet restore ${project-} \ @@ -20,9 +29,12 @@ dotnetConfigureHook() { -p:Deterministic=true \ --runtime "@runtimeId@" \ --source "@nugetSource@/lib" \ + ${nugetSourceFlag-} \ + ${lockfileFlag-} \ ${parallelFlag-} \ ${dotnetRestoreFlags[@]} \ ${dotnetFlags[@]} + } # Generate a NuGet.config file to make sure everything, diff --git a/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-validate-lockfile.sh b/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-validate-lockfile.sh new file mode 100644 index 0000000000000..cc28f90c2cb12 --- /dev/null +++ b/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-validate-lockfile.sh @@ -0,0 +1,67 @@ +# Validate a lockfile does not depend on any floating Nuget dependencies +# Note that all ranges other than floating resolve to the lowest available version by default +# See https://docs.microsoft.com/en-us/nuget/concepts/package-versioning#version-ranges for more information +dotnetValidateLockfileHook() { + echo "Executing dotnetValidateLockfileHook" + + if [[ ! -f "${lockfilePath-}" ]]; then + echo "Could not locate the lockfile! Consider setting \"dontDotnetValidateLockfile\"" + exit 1 + fi + + lockfileVersion="$(jq ".version" < "${lockfilePath}")" + + if (( "${lockfileVersion}" != 1 && "${lockfileVersion}" != 2 )); then + echo "error: unsupported lockfile version: ${lockfileVersion}" + exit 1 + fi + + # An array of most dependencies and their versions. + depsWithVersions="$(jq -rc ' + # Direct dependencies. This only includes entries with requested version ranges + [ .dependencies[] | with_entries(select(.value | has("requested"))) | to_entries[] | { + "name": .key, # Name of dependency + "version": .value.requested # Requested version range + # Dependencies of dependencies. Includes all of them, as requested ranges are not marked + }] + [ .dependencies[] | with_entries(select(.value | has("dependencies"))) | .[].dependencies | to_entries[] | { + "name": .key, # Name of dependency + "version": .value # Version, this can either be a range or an exact match + }] + ' < "${lockfilePath}")" + + depsLength="$(jq -rc 'length' <<< "${depsWithVersions}")" + lockfileErrors=() + + i=0 + while (( "$i" < "${depsLength}" )); do + depName="$(jq -rc --arg i "$i" '.[($i | tonumber)] | .name' <<< "${depsWithVersions}")" + depVersion="$(jq -rc --arg i "$i" '.[($i | tonumber)] | .version' <<< "${depsWithVersions}")" + + # Check if the version range is floating + if [[ "${depVersion}" = *"*"* ]]; then + lockfileErrors+=("Package \"${depName}\" requested floating version range \"${depVersion}\"") + fi + + i=$(( i + 1 )) + done + unset i + + if (( "${#lockfileErrors[@]}" > 0 )); then + echo "Attempted to fetch unstable NuGet dependency version(s):" + + for error in "${lockfileErrors[@]}"; do + echo " ${error}" + done + + echo "This project uses unstable version range(s) for its Nuget dependencies, which we cannot deterministically fetch." + echo "Please set the \"nugetDeps\" attribute to a lockfile generated with the \"passthru.fetch-deps\" script to ensure all dependencies are locked." + exit 1 + fi + + echo "succesfully validated lockfile" + echo "Finished dotnetValidateLockfileHook" +} + +if [[ -z "${dontDotnetValidateLockfile-}" && -z "${postConfigure-}" ]]; then + postConfigure=dotnetValidateLockfileHook +fi From 5a573fbe06caede87b6c13c3646c75ebcf33e202 Mon Sep 17 00:00:00 2001 From: purepani Date: Tue, 11 Jun 2024 22:29:17 -0500 Subject: [PATCH 2/5] buildDotnetModule: Makes build time lockfile generation functional --- pkgs/build-support/dotnet/build-dotnet-module/default.nix | 4 ++-- .../dotnet/build-dotnet-module/fetch-deps.nix | 7 +++++-- .../dotnet/build-dotnet-module/hooks/default.nix | 4 ++-- .../dotnet/build-dotnet-module/hooks/dotnet-build-hook.sh | 1 + .../build-dotnet-module/hooks/dotnet-configure-hook.sh | 3 +-- pkgs/development/compilers/dotnet/build-dotnet.nix | 1 + 6 files changed, 12 insertions(+), 8 deletions(-) diff --git a/pkgs/build-support/dotnet/build-dotnet-module/default.nix b/pkgs/build-support/dotnet/build-dotnet-module/default.nix index 00e02acb69f25..5f68f84bd3ef7 100644 --- a/pkgs/build-support/dotnet/build-dotnet-module/default.nix +++ b/pkgs/build-support/dotnet/build-dotnet-module/default.nix @@ -89,7 +89,7 @@ # Whether to use an alternative wrapper, that executes the application DLL using the dotnet runtime from the user environment. `dotnet-runtime` is provided as a default in case no .NET is installed # This is useful for .NET tools and applications that may need to run under different .NET runtimes , useDotnetFromEnv ? false - # Whether to explicitly enable UseAppHost when building. This is redundant if useDotnetFromEnv is enabledz + # Whether to explicitly enable UseAppHost when building. This is redundant if useDotnetFromEnv is enabled. , useAppHost ? true # The dotnet SDK to use. , dotnet-sdk ? dotnetCorePackages.sdk_6_0 @@ -107,7 +107,7 @@ let then lib.intersectLists args.meta.platforms dotnet-sdk.meta.platforms else dotnet-sdk.meta.platforms; # Nuget packages provided by the SDK, these get excluded from the generated lockfile - sdkExclusions = writeText "${dotnet-sdk.name}-exclusions" (lib.concatStringsSep "\n" (dotnet-sdk.passthru.packages { fetchNuGet = attrs: attrs.pname; })); + sdkExclusions = writeText "${dotnet-sdk.name}-exclusions" (lib.concatStringsSep "\n" (dotnet-sdk.passthru.packages_list { fetchNuGet = attrs: attrs.pname; })); inherit (callPackage ./hooks { inherit dotnet-sdk disabledTests nuget-source dotnet-runtime runtimeDeps buildType dontSetNugetSource; diff --git a/pkgs/build-support/dotnet/build-dotnet-module/fetch-deps.nix b/pkgs/build-support/dotnet/build-dotnet-module/fetch-deps.nix index b7a2822e5e45b..9cf0da750643c 100644 --- a/pkgs/build-support/dotnet/build-dotnet-module/fetch-deps.nix +++ b/pkgs/build-support/dotnet/build-dotnet-module/fetch-deps.nix @@ -21,7 +21,10 @@ buildDotnetModule rec { inherit src dotnet-sdk projectFile testProjectFile dotnetFlags dotnetRestoreFlags enableParallelBuilding; - nativeBuildInputs = [ dotnetValidateLockfileHook ]; + nativeBuildInputs = [ + dotnetValidateLockfileHook + (nuget-to-nix.override {inherit dotnet-sdk;}) + ]; generateLockfile = true; dontSetNugetSource = true; @@ -47,7 +50,7 @@ buildDotnetModule rec { NUGET_DEPS="$HOME/deps.nix" echo "Writing lockfile..." - ${nuget-to-nix}/bin/nuget-to-nix "$HOME/nuget-pkgs" "${sdkExclusions}" > "$NUGET_DEPS" + nuget-to-nix "$HOME/nuget-pkgs" "${sdkExclusions}" > "$NUGET_DEPS" runHook postBuild ''; diff --git a/pkgs/build-support/dotnet/build-dotnet-module/hooks/default.nix b/pkgs/build-support/dotnet/build-dotnet-module/hooks/default.nix index eea5b1cc6f464..fc2bd40c38b7f 100644 --- a/pkgs/build-support/dotnet/build-dotnet-module/hooks/default.nix +++ b/pkgs/build-support/dotnet/build-dotnet-module/hooks/default.nix @@ -26,7 +26,7 @@ in dotnetConfigureHook = makeSetupHook { name = "dotnet-configure-hook"; - deps = [ dotnet-sdk ]; + propagatedBuildInputs = [ dotnet-sdk ]; substitutions = { nugetSource = lib.optional (!dontSetNugetSource) nuget-source; dynamicLinker = "${stdenv.cc}/nix-support/dynamic-linker"; @@ -45,7 +45,7 @@ in dotnetValidateLockfileHook = makeSetupHook { name = "dotnet-validate-lockfile-hook"; - deps = [ jq ]; + propagatedBuildInputs = [ jq ]; } ./dotnet-validate-lockfile.sh; dotnetBuildHook = makeSetupHook diff --git a/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-build-hook.sh b/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-build-hook.sh index 798109291f92a..590ff683eaa67 100644 --- a/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-build-hook.sh +++ b/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-build-hook.sh @@ -46,6 +46,7 @@ dotnetBuildHook() { -p:BuildInParallel=$parallelBuildFlag \ -p:ContinuousIntegrationBuild=true \ -p:Deterministic=true \ + -p:UseAppHost=true \ --configuration "@buildType@" \ --no-restore \ ${versionFlags[@]} \ diff --git a/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-configure-hook.sh b/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-configure-hook.sh index 90822cb51e15f..e051add5be757 100644 --- a/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-configure-hook.sh +++ b/pkgs/build-support/dotnet/build-dotnet-module/hooks/dotnet-configure-hook.sh @@ -28,7 +28,6 @@ dotnetConfigureHook() { -p:ContinuousIntegrationBuild=true \ -p:Deterministic=true \ --runtime "@runtimeId@" \ - --source "@nugetSource@/lib" \ ${nugetSourceFlag-} \ ${lockfileFlag-} \ ${parallelFlag-} \ @@ -55,7 +54,7 @@ EOF find -name paket.dependencies -exec sed -i 's+source .*+source @nugetSource@/lib+g' {} \; find -name paket.lock -exec sed -i 's+remote:.*+remote: @nugetSource@/lib+g' {} \; - dotnet tool restore --add-source "@nugetSource@/lib" + #dotnet tool restore --add-source "@nugetSource@/lib" (( "${#projectFile[@]}" == 0 )) && dotnetRestore diff --git a/pkgs/development/compilers/dotnet/build-dotnet.nix b/pkgs/development/compilers/dotnet/build-dotnet.nix index 11ece53971662..5a3d6e6fa3026 100644 --- a/pkgs/development/compilers/dotnet/build-dotnet.nix +++ b/pkgs/development/compilers/dotnet/build-dotnet.nix @@ -136,6 +136,7 @@ mkCommon type rec { passthru = { inherit icu; + packages_list = packages; } // lib.optionalAttrs (type == "sdk") { packages = mkNugetDeps { name = "${pname}-${version}-deps"; From 5987390eff68c871d6304a36c914d654dcba24f4 Mon Sep 17 00:00:00 2001 From: purepani Date: Tue, 11 Jun 2024 22:30:17 -0500 Subject: [PATCH 3/5] jellyfin: Moves to build-time lockfile generation --- pkgs/by-name/je/jellyfin/package.nix | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pkgs/by-name/je/jellyfin/package.nix b/pkgs/by-name/je/jellyfin/package.nix index ecf1c6bda7923..129def611366c 100644 --- a/pkgs/by-name/je/jellyfin/package.nix +++ b/pkgs/by-name/je/jellyfin/package.nix @@ -25,8 +25,8 @@ buildDotnetModule rec { propagatedBuildInputs = [ sqlite ]; projectFile = "Jellyfin.Server/Jellyfin.Server.csproj"; + nugetSha256 = "sha256-riWeRMwcn6D8wFpgQD02Cs7/RrP6Y99lM8bH6hM7lco="; executables = [ "jellyfin" ]; - nugetDeps = ./nuget-deps.nix; runtimeDeps = [ jellyfin-ffmpeg fontconfig From 02e7e1c80cfeed3601e83ec329a3486f89456227 Mon Sep 17 00:00:00 2001 From: purepani Date: Tue, 11 Jun 2024 22:54:25 -0500 Subject: [PATCH 4/5] buildDotnetModule: Changes nugetSha256 in build-dotnet-module to lockfileSha256 --- .../dotnet/build-dotnet-module/default.nix | 6 +++--- .../dotnet/build-dotnet-module/fetch-deps.nix | 10 +++++----- pkgs/by-name/je/jellyfin/package.nix | 2 +- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/pkgs/build-support/dotnet/build-dotnet-module/default.nix b/pkgs/build-support/dotnet/build-dotnet-module/default.nix index 5f68f84bd3ef7..c016604b0c3e3 100644 --- a/pkgs/build-support/dotnet/build-dotnet-module/default.nix +++ b/pkgs/build-support/dotnet/build-dotnet-module/default.nix @@ -70,7 +70,7 @@ # The hash of the generated nix-based lockfile containing all Nuget dependencies. This depends on no floating version ranges being requested upstream, as we cannot deterministically fetch those. # If any floating version is requested, an error will be thrown. Note that this option is mutually exclusive with the `nugetDeps` attribute. -, nugetSha256 ? null +, lockfileSha256 ? null # Don't expose any Nuget source to `dotnet restore`. , dontSetNugetSource ? false @@ -99,7 +99,7 @@ , ... } @ args: assert projectFile == null -> throw "Defining the `projectFile` attribute is required. This is usually an `.csproj`, or `.sln` file."; -assert (nugetDeps != null && nugetSha256 != null) -> throw "The attributes `nugetDeps` and 'nugetSha256' are mutually exclusive!"; +assert (nugetDeps != null && lockfileSha256 != null) -> throw "The attributes `nugetDeps` and 'lockfileSha256' are mutually exclusive!"; let platforms = @@ -118,7 +118,7 @@ let }) dotnetConfigureHook dotnetValidateLockfileHook dotnetBuildHook dotnetCheckHook dotnetInstallHook dotnetFixupHook; fetchImpureDeps = callPackage ./fetch-deps.nix { - inherit name meta dotnet-sdk nugetSha256 projectFile testProjectFile dotnetFlags dotnetRestoreFlags enableParallelBuilding dotnetValidateLockfileHook sdkExclusions; + inherit name meta dotnet-sdk lockfileSha256 projectFile testProjectFile dotnetFlags dotnetRestoreFlags enableParallelBuilding dotnetValidateLockfileHook sdkExclusions; src = srcOnly args; }; diff --git a/pkgs/build-support/dotnet/build-dotnet-module/fetch-deps.nix b/pkgs/build-support/dotnet/build-dotnet-module/fetch-deps.nix index 9cf0da750643c..1e4cb661ac119 100644 --- a/pkgs/build-support/dotnet/build-dotnet-module/fetch-deps.nix +++ b/pkgs/build-support/dotnet/build-dotnet-module/fetch-deps.nix @@ -2,7 +2,7 @@ , buildDotnetModule , dotnetValidateLockfileHook , nuget-to-nix -, nugetSha256 +, lockfileSha256 , src , name @@ -33,8 +33,8 @@ buildDotnetModule rec { impureEnvVars = lib.fetchers.proxyImpureEnvVars; outputHashAlgo = "sha256"; outputHashMode = "flat"; - outputHash = if (nugetSha256 != null) - then nugetSha256 + outputHash = if (lockfileSha256 != null) + then lockfileSha256 else ""; # This needs to be set for networking, an empty string prints the "empty hash found" warning preConfigure = '' @@ -65,8 +65,8 @@ buildDotnetModule rec { ''; # This is the last phase that runs before we error out about the hash being wrong - postFixup = lib.optionalString (nugetSha256 == null) '' - echo "Please set nugetSha256 to the hash below!" + postFixup = lib.optionalString (lockfileSha256 == null) '' + echo "Please set lockfileSha256 to the hash below!" ''; meta = { diff --git a/pkgs/by-name/je/jellyfin/package.nix b/pkgs/by-name/je/jellyfin/package.nix index 129def611366c..3a1b96f81d34c 100644 --- a/pkgs/by-name/je/jellyfin/package.nix +++ b/pkgs/by-name/je/jellyfin/package.nix @@ -25,7 +25,7 @@ buildDotnetModule rec { propagatedBuildInputs = [ sqlite ]; projectFile = "Jellyfin.Server/Jellyfin.Server.csproj"; - nugetSha256 = "sha256-riWeRMwcn6D8wFpgQD02Cs7/RrP6Y99lM8bH6hM7lco="; + lockfileSha256 = "sha256-riWeRMwcn6D8wFpgQD02Cs7/RrP6Y99lM8bH6hM7lco="; executables = [ "jellyfin" ]; runtimeDeps = [ jellyfin-ffmpeg From c4ea389051a9f9f3a242ad680635bcb91cb6e938 Mon Sep 17 00:00:00 2001 From: purepani Date: Tue, 11 Jun 2024 22:59:30 -0500 Subject: [PATCH 5/5] buildDotnetModules: Removes assertion for project file --- pkgs/build-support/dotnet/build-dotnet-module/default.nix | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pkgs/build-support/dotnet/build-dotnet-module/default.nix b/pkgs/build-support/dotnet/build-dotnet-module/default.nix index c016604b0c3e3..8fab93890a5fb 100644 --- a/pkgs/build-support/dotnet/build-dotnet-module/default.nix +++ b/pkgs/build-support/dotnet/build-dotnet-module/default.nix @@ -98,7 +98,7 @@ , ... } @ args: -assert projectFile == null -> throw "Defining the `projectFile` attribute is required. This is usually an `.csproj`, or `.sln` file."; +#assert projectFile == null -> throw "Defining the `projectFile` attribute is required. This is usually an `.csproj`, or `.sln` file."; assert (nugetDeps != null && lockfileSha256 != null) -> throw "The attributes `nugetDeps` and 'lockfileSha256' are mutually exclusive!"; let