ADR Suggestion Freezing Dependency Versions on Library Release

[AndrewSazonov]

Context and Problem Statement

Currently, when we release a new version of Python library, we do not specify any version limits for our dependencies in the pyproject.toml file. As a result, when a user installs one of our packages from PyPI, the latest available versions of all dependencies are installed - whether they were available at the time of our release or published later.

This has several risks:

• A newer version of a dependency (released after our own release) may introduce breaking changes, deprecations, or bugs that break our library.
• Users can encounter unexpected errors or inconsistent behavior that were not present at the time of release.
• It becomes difficult to reproduce the exact environment that was tested and validated for the release.

By freezing dependency versions (to varying degrees), we can mitigate these risks, improve reproducibility, and offer a more stable experience to users.

Current Approach

• We do not specify any version constraints in pyproject.toml.
• This means dependencies are unrestricted, and the latest available versions are installed by default.

Example of pyproject.toml (Current Approach):

[project]
dependencies = [
    "numpy",
    "scipy",
    "matplotlib"
]

Considered Options

1. Minimal Version Constraints with Open Upper Bound (Flexible Updates)

• Approach:
  Specify minimum versions for all dependencies, without restricting the upper versions.
  This guarantees the library works with at least a specific version, but allows future versions to be installed.

• Pros:

  • Provides flexibility to users to benefit from updates in dependencies.
  • Requires less frequent updates by maintainers.
  • Easy to maintain, while offering some control over minimum compatibility.

• Cons:

  • Future versions of dependencies may introduce breaking changes.
  • Harder to reproduce the exact environment used during testing and release.
  • Risk of instability as dependency ecosystems evolve.

• Example of pyproject.toml:

[project]
dependencies = [
    "numpy>=1.24",
    "scipy>=1.10",
    "matplotlib>=3.7"
]

2. Frozen Dependencies in Documentation (Soft Freezing with Recommendations)

• Approach:
  Continue using minimal version constraints (as in Option 1), but provide a frozen list of tested versions in the documentation (e.g., README.md or a docs/installation.md file). Users can choose to follow these recommendations or use newer versions.

• Pros:

  • Reproducibility for users who follow the documentation.
  • Easy to maintain minimal version constraints in pyproject.toml.
  • No need to generate or maintain lock files.

• Cons:

  • Users must manually pin versions if they want to reproduce the tested environment.
  • No enforcement through packaging tools; relies on users reading and following instructions.

• Example of pyproject.toml:
  Same as Option 1.

• Example of documented recommended versions:

  Tested versions (Release v1.0.0):
  numpy==1.24.2
  scipy==1.10.0
  matplotlib==3.7.1
  

3. Strict Version Pinning at Release Time (Hard Freeze)

• Approach:
  Specify exact versions for all dependencies directly in pyproject.toml.
  Users will automatically install the exact versions tested at the time of release when they run:

  pip install easydiffraction

• Pros:

  • Complete reproducibility for the tested environment.
  • Guarantees stability and avoids upstream breakages.
  • Simple user experience: no additional steps required.

• Cons:

  • Users won’t get bug fixes or improvements in dependencies unless we release a new version with updated pins.
  • Maintenance overhead: requires frequent re-testing and updating of pinned versions.
  • Reduces flexibility for users who want to integrate the library into larger environments.

• Example of pyproject.toml:

[project]
dependencies = [
    "numpy==1.24.2",
    "scipy==1.10.0",
    "matplotlib==3.7.1"
]

4. Hybrid Approach (Minimal Constraints + Optional Hard Freeze via Lock Files)

• Approach:

  • Continue to specify minimum version constraints in pyproject.toml.
  • Provide an optional lock file (using a tool like pip-tools) to freeze versions for users who need strict reproducibility.
  • Users who want flexibility can install via:

    pip install easydiffraction

  • Users who want a tested environment must download and install with:

    pip install -r requirements-strict.txt

• Pros:

  • Balances flexibility with reproducibility.
  • Advanced users can control the level of strictness they want.
  • Easier maintenance since pyproject.toml stays clean, and lock files are updated at release.

• Cons:

  • Breaks the idea of simplicity for less experienced users. Users would need to download requirements-strict.txt manually or clone the repository to access it. This is far from ideal UX, especially compared to a simple pip install command.
  • Adds complexity to documentation and user installation instructions.
  • Requires maintaining lock files separately.

• Example of pyproject.toml:
  Same as Option 1.

• Example lock file (not part of pyproject.toml):

  numpy==1.24.2
  scipy==1.10.0
  matplotlib==3.7.1
  

Comparison Between Options 3 and 4

While Option 4 appears to offer the best of both worlds (flexibility for advanced users and reproducibility for others), it complicates the user experience:

• Less experienced users, who we expect to run a simple:

  pip install easydiffraction

  Would instead be required to manually obtain and install from requirements-strict.txt, which requires extra steps and understanding.

• This breaks our goal of keeping the installation process as simple and intuitive as possible for users who are not Python experts.

By contrast, Option 3 guarantees a simple UX:

• The pip install easydiffraction command just works, providing the tested, stable, and reproducible environment we verified during release.

For these reasons, we propose to adopt Option 3 and not pursue Option 4 further at this time.

Decision Proposal

For the reasons mentioned above, I propose adopting Option 3 (Strict Version Pinning at Release Time), ensuring maximum stability and reproducibility for our users.

• In pyproject.toml, we will specify exact versions of dependencies for each release.
• Users will automatically install the tested versions when they pip install easydiffraction or similar.
• We will review and update the pinned versions regularly (ideally at every release cycle) to ensure we stay current with the latest improvements and fixes in our dependencies.

Open Questions

Currently, we use GitHub Dependabot, which automatically creates pull requests when dependencies are updated (configuration is typically done via dependabot.yml, although security updates can work without it). Dependabot notifies us and offers automated version bumps.

With hard freeze (Option 3), Dependabot seems to remain valuable (this needs to be confirmed):

• It will alert us to new versions.
• It can trigger automated CI tests with newer dependencies.

This process helps us decide when to unfreeze and update the pinned versions for the next release.

[damskii9992]

I don't have much to add except that I'm in favor of Option 3, the Hard Freeze.
It might be a slightly higher maintenance burden to manually test and update pinned versions, but considering the issue and maintenance burden associated with dependencies suddenly breaking our library, I think it will prove to be less of a burden with this approach in the long run.
And most users will just install with the easy pip install easydiffraction and might never look in any readme files or documentation for lock files or version recommendations.

[rozyczko]

I am for option 3 - we really want to make sure the environment is correctly set to minimize potential issues post-release.
Regarding listed cons:

Maintenance overhead: requires frequent re-testing and updating of pinned versions.

Or, as discussed earlier - immediately delete pinning in develop after the release, so feature branches depend on latest. This postpones testing to the last stage of the subsequent release.

Reduces flexibility for users who want to integrate the library into larger environments.

This is a non-issue for the apps, obviously, but for the libs it might cause ocassional hiccups on user systems.
The best advice would be to ask (require?) the installation to be done in a clean virt env. This should be standard, anyway.

[AndrewSazonov]

Or, as discussed earlier - immediately delete pinning in develop after the release, so feature branches depend on latest. This postpones testing to the last stage of the subsequent release.

That makes sense, but the problem is it’s easy to forget this step. We have already seen it happen from time to time in different projects. To avoid this, we could automate it with a workflow that creates a PR from master to develop right after tagging a new release. This PR would include unpinning all the dependencies so develop always depends on the latest versions.

On top of that, we could add another workflow that blocks or fails pull requests if the source branch was created from master. That should help keep things clean and make sure we are working off the right branch.

This is a non-issue for the apps, obviously, but for the libs it might cause ocassional hiccups on user systems.
The best advice would be to ask (require?) the installation to be done in a clean virt env. This should be standard, anyway.

Right, this isn’t relevant for the apps since all dependencies are bundled when building them. For the libraries, we already recommend using a virtual environment at the very beginning of the Installation section in the docs. But of course, we can’t force users to follow that advice.

So, I think the advantages of option 3 outweigh its downsides, even if there’s a small risk of occasional hiccups on user systems.

[rozyczko]

we could add another workflow that blocks or fails pull requests if the source branch was created from master

This is an interesting suggestion. Apart from maybe confusion for a new developer, this sounds solid.

[rozyczko]

Then, apart from having this specified as an ADR, we should update the Release Checklist to include the step of stripping the pinning info in pyproject.toml on develop after the release. Or, in case we are able to automate it - make sure the stripping is done.