From 68f6a9f2c86a87c5df96717fb2f08952e981407f Mon Sep 17 00:00:00 2001 From: CEbbinghaus Date: Mon, 4 Mar 2024 12:04:10 +1100 Subject: [PATCH 1/6] Added first iteration of RFC for Central Lock Files --- accepted/2024/repo-level-lock-file.md | 83 +++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 accepted/2024/repo-level-lock-file.md diff --git a/accepted/2024/repo-level-lock-file.md b/accepted/2024/repo-level-lock-file.md new file mode 100644 index 000000000..632f2bd06 --- /dev/null +++ b/accepted/2024/repo-level-lock-file.md @@ -0,0 +1,83 @@ +# ***Repository Level Lock File*** + +- Author: [@CEbbinghaus](https://github.com/CEbbinghaus) +- Issue: [Nuget/Home#12409](https://github.com/NuGet/Home/issues/12409) + +## Summary + +Applications consisting of multiple solutions (using assembly references between projects) Cannot control their transitive dependencies by using Nuget. The current lock file is on a per-project basis rather than an application/repository basis. Given a `Directory.packages.props` file, each project could be resolved with different versions of transitive dependencies, which a single central lock file would solve, by allowing consistency between package versions across multiple solutions. + + +## Motivation + +Monorepositories with multiple solutions utilizing assembly references that deploy all their code together as a single application are unable to use Nuget (with or without CPM) due to transitive version conflicts. This is due to nuget being able to resolve different versions of transitive dependencies for each individual build, as they occur in isolation from one another. A single central lock file allows all transitive dependencies to have locked versions defined at the root, thereby eliminating third-party dependency management software such as Paket. + + +## Explanation + +### Functional explanation + + + + +The `Directory.packages.props` now uses the`[path]` property within a `` to specify its lock file path. It becomes the source of truth for all builds. Project level lock files are no longer required unless a package version is overridden. + +```xml + + + true + lockfile.json + + + + + +``` + +This lock file is generated during any restore operation during the evaluation of the `Directory.packages.props` file. This allows the central lock file to be generated by any project as all of the dependencies are isolated. + +## Drawbacks + + +It complicates Nuget restoring as it will require a separate dependency resolution step exclusive to the `Directory.packages.props` file. + +## Rationale and alternatives + + + + + +Some alternatives exist. Namely: +* **Solution-based lock files** + This has the upside of giving us a comprehensive list of all projects that make up the lockfile. As such, the lockfile can be optimally generated, taking every dependency of every project into account, and doesn't require project-specific dependencies. The downside is lock files are defined in any of the multitude of solutions within a monorepository. This leaves the problem of syncing transitive dependencies between multiple solutions within the same repository. It also requires all restores to happen at the solution level, preventing building singular projects simultaneously. + +* **Using Microsoft.Build.Traversal** + `Microsoft.Build.Traversal` could create a `.csproj` that references all projects under it. As such, it would have the context of every package as defined by the `ProjectReference`. This would then act as the restore project that, in turn, sorts out all dependencies and generates the most optimal lock file. The downside is that only this project can be used to restore packages since the lock file would only apply to it, making it impossible to restore single projects. + + +## Prior Art + + + + + + +The primary example of prior art for a centralized lock file is [Paket](https://fsprojects.github.io/Paket/) built around this exact concept. It uses a `paket.dependencies` file, similar to the `Directory.packages.props` file from Nuget. It allows for 'groups' to pull two identical dependencies in different versions as each group is resolved independently. It then uses a `paket.lock` to lock the dependencies at the root (right next to the paket.dependencies), which pins all primary and transitive package versions for all projects. A project then includes a package by listing its name in the `paket.references` file in the csproj directory. + +We can draw many parallels between Paket & Nuget (with a central lock file) function. For one, we define all the primary packages & versions at the root. Projects define which specific packages they depend on within the relative projects. Where paket would use groups to include dependencies at different versions, NuGet uses the `VersionOverride` property within the csproj. This is the primary difference in how locks are handled for version differences since Paket utilizes groups and writes the locked package versions into the `paket.lock` file. Nuget (with central locking) writes all the pinned versions into the local csproj lock file. + +NodeJS has the concept of [Workspaces](https://docs.npmjs.com/cli/v7/using-npm/workspaces#skip-to-content) which act more like Solutions since NodeJs only has the concept of Projects. However a Workspace does allow for all packages to be resolved to the same version, ensuring no version conflicts between different projects within the same workspace. + +## Unresolved Questions + + + + + +* How would the `Directory.packages.props` dependencies be separated out from the project dependencies to generate the lock file? +* How does the publish command handle this as it doesn't have full context of all packages. (Could be solved by the restore always running with --locked-mode) +* How will a project deleted from the repository get deleted from the lock file. + +## Future Possibilities + +Using the `TargetFrameworks` property within the `Directory.packages.props` file could narrow down the frameworks resolved for the provided packages. This can simplify the lock file and reduce some unnecessary dependency resolution. From 829f4887d7524800169fa5fc17558299256222b0 Mon Sep 17 00:00:00 2001 From: CEbbinghaus Date: Wed, 6 Mar 2024 16:20:24 +1100 Subject: [PATCH 2/6] Added extra unresolved question regarding nested files --- accepted/2024/repo-level-lock-file.md | 1 + 1 file changed, 1 insertion(+) diff --git a/accepted/2024/repo-level-lock-file.md b/accepted/2024/repo-level-lock-file.md index 632f2bd06..8741fcebd 100644 --- a/accepted/2024/repo-level-lock-file.md +++ b/accepted/2024/repo-level-lock-file.md @@ -77,6 +77,7 @@ NodeJS has the concept of [Workspaces](https://docs.npmjs.com/cli/v7/using-npm/w * How would the `Directory.packages.props` dependencies be separated out from the project dependencies to generate the lock file? * How does the publish command handle this as it doesn't have full context of all packages. (Could be solved by the restore always running with --locked-mode) * How will a project deleted from the repository get deleted from the lock file. +* How will Lock files be generated with nested `Directoy.packages.props` files. ## Future Possibilities From f991842db8e1c01b6780098500667e09b2c12226 Mon Sep 17 00:00:00 2001 From: CEbbinghaus Date: Wed, 20 Mar 2024 13:48:06 +1100 Subject: [PATCH 3/6] Applied Suggestions Co-authored-by: Yaakov --- accepted/2024/repo-level-lock-file.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/accepted/2024/repo-level-lock-file.md b/accepted/2024/repo-level-lock-file.md index 8741fcebd..fc306c68a 100644 --- a/accepted/2024/repo-level-lock-file.md +++ b/accepted/2024/repo-level-lock-file.md @@ -5,12 +5,12 @@ ## Summary -Applications consisting of multiple solutions (using assembly references between projects) Cannot control their transitive dependencies by using Nuget. The current lock file is on a per-project basis rather than an application/repository basis. Given a `Directory.packages.props` file, each project could be resolved with different versions of transitive dependencies, which a single central lock file would solve, by allowing consistency between package versions across multiple solutions. +Applications consisting of multiple solutions (using assembly references between projects) cannot control their transitive dependencies by using Nuget. The current lock file is on a per-project basis rather than an application/repository basis. Given a `Directory.packages.props` file, each project could resolve different versions of transitive dependencies, which a single central lock file would solve by allowing consistency between package versions across multiple solutions. ## Motivation -Monorepositories with multiple solutions utilizing assembly references that deploy all their code together as a single application are unable to use Nuget (with or without CPM) due to transitive version conflicts. This is due to nuget being able to resolve different versions of transitive dependencies for each individual build, as they occur in isolation from one another. A single central lock file allows all transitive dependencies to have locked versions defined at the root, thereby eliminating third-party dependency management software such as Paket. +Monorepositories with multiple solutions utilizing assembly references that deploy all their code together as a single application are unable to use Nuget (with or without CPM) due to transitive version conflicts. This is due to NuGet being able to resolve different versions of transitive dependencies for each individual build, as they occur in isolation from one another. A single central lock file allows all transitive dependencies to have locked versions defined at the root, thereby eliminating third-party dependency management software such as Paket. ## Explanation @@ -64,9 +64,9 @@ Some alternatives exist. Namely: The primary example of prior art for a centralized lock file is [Paket](https://fsprojects.github.io/Paket/) built around this exact concept. It uses a `paket.dependencies` file, similar to the `Directory.packages.props` file from Nuget. It allows for 'groups' to pull two identical dependencies in different versions as each group is resolved independently. It then uses a `paket.lock` to lock the dependencies at the root (right next to the paket.dependencies), which pins all primary and transitive package versions for all projects. A project then includes a package by listing its name in the `paket.references` file in the csproj directory. -We can draw many parallels between Paket & Nuget (with a central lock file) function. For one, we define all the primary packages & versions at the root. Projects define which specific packages they depend on within the relative projects. Where paket would use groups to include dependencies at different versions, NuGet uses the `VersionOverride` property within the csproj. This is the primary difference in how locks are handled for version differences since Paket utilizes groups and writes the locked package versions into the `paket.lock` file. Nuget (with central locking) writes all the pinned versions into the local csproj lock file. +We can draw many parallels between Paket & NuGet (with a central lock file) function. For one, we define all the primary packages & versions at the root. Projects define which specific packages they depend on within the relative projects. Where paket would use groups to include dependencies at different versions, NuGet uses the `VersionOverride` property within the csproj. This is the primary difference in how locks are handled for version differences since Paket utilizes groups and writes the locked package versions into the `paket.lock` file. Nuget (with central locking) writes all the pinned versions into the local csproj lock file. -NodeJS has the concept of [Workspaces](https://docs.npmjs.com/cli/v7/using-npm/workspaces#skip-to-content) which act more like Solutions since NodeJs only has the concept of Projects. However a Workspace does allow for all packages to be resolved to the same version, ensuring no version conflicts between different projects within the same workspace. +NodeJS has the concept of [Workspaces](https://docs.npmjs.com/cli/v7/using-npm/workspaces#skip-to-content) which act more like Solutions since NodeJS only has the concept of Projects. However a Workspace does allow for all packages to be resolved to the same version, ensuring no version conflicts between different projects within the same workspace. ## Unresolved Questions @@ -77,7 +77,7 @@ NodeJS has the concept of [Workspaces](https://docs.npmjs.com/cli/v7/using-npm/w * How would the `Directory.packages.props` dependencies be separated out from the project dependencies to generate the lock file? * How does the publish command handle this as it doesn't have full context of all packages. (Could be solved by the restore always running with --locked-mode) * How will a project deleted from the repository get deleted from the lock file. -* How will Lock files be generated with nested `Directoy.packages.props` files. +* How will Lock files be generated with nested `Directory.packages.props` files? ## Future Possibilities From 2e86b5012a96e5c6ed07a7b3725f0cc2ed0d3ad6 Mon Sep 17 00:00:00 2001 From: CEbbinghaus Date: Wed, 20 Mar 2024 13:58:38 +1100 Subject: [PATCH 4/6] Added Additional benefit of TFM Lock file limitations --- accepted/2024/repo-level-lock-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accepted/2024/repo-level-lock-file.md b/accepted/2024/repo-level-lock-file.md index fc306c68a..fa7e6d30d 100644 --- a/accepted/2024/repo-level-lock-file.md +++ b/accepted/2024/repo-level-lock-file.md @@ -81,4 +81,4 @@ NodeJS has the concept of [Workspaces](https://docs.npmjs.com/cli/v7/using-npm/w ## Future Possibilities -Using the `TargetFrameworks` property within the `Directory.packages.props` file could narrow down the frameworks resolved for the provided packages. This can simplify the lock file and reduce some unnecessary dependency resolution. +Using the `TargetFrameworks` property within the `Directory.packages.props` file could narrow down the frameworks resolved for the provided packages. This can simplify the lock file and reduce some unnecessary dependency resolution as well as reduce merge conflicts as there is less content changing less often From 187533b080b03f0fe768d0fcca274d84a0482a34 Mon Sep 17 00:00:00 2001 From: CEbbinghaus Date: Fri, 7 Mar 2025 15:52:04 +1100 Subject: [PATCH 5/6] Moved Lockfile into 2025 --- accepted/{2024 => 2025}/repo-level-lock-file.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename accepted/{2024 => 2025}/repo-level-lock-file.md (100%) diff --git a/accepted/2024/repo-level-lock-file.md b/accepted/2025/repo-level-lock-file.md similarity index 100% rename from accepted/2024/repo-level-lock-file.md rename to accepted/2025/repo-level-lock-file.md From cc63cd3be892b0f52adc25d2b077f88971d4fd1e Mon Sep 17 00:00:00 2001 From: CEbbinghaus Date: Mon, 12 Jan 2026 17:19:14 +1100 Subject: [PATCH 6/6] Moved design into 2026 --- accepted/{2025 => 2026}/repo-level-lock-file.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename accepted/{2025 => 2026}/repo-level-lock-file.md (100%) diff --git a/accepted/2025/repo-level-lock-file.md b/accepted/2026/repo-level-lock-file.md similarity index 100% rename from accepted/2025/repo-level-lock-file.md rename to accepted/2026/repo-level-lock-file.md