-
-
Notifications
You must be signed in to change notification settings - Fork 27
CFEP 21 - conda-forge / Bioconda Transfer Policy - DRAFT #52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
58cbefe
cea3e4b
1ffde6e
bc242a8
f949bb7
04051d8
f6bf5a7
20593cf
4b73c2a
8a845d5
2192ead
c0ba9db
64487ba
a45980d
7349546
b2714ce
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,76 @@ | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| <table> | ||||||||||||||||||||||||||||||||
| <tr><td> Title </td><td> conda-forge / Bioconda Transfer Policy </td> | ||||||||||||||||||||||||||||||||
| <tr><td> Status </td><td> Draft </td></tr> | ||||||||||||||||||||||||||||||||
| <tr><td> Author(s) </td><td> Mervin Fansler <mervin.fansler@bric.ku.dk></td></tr> | ||||||||||||||||||||||||||||||||
| <tr><td> Created </td><td> 10 July 2024</td></tr> | ||||||||||||||||||||||||||||||||
| <tr><td> Updated </td><td> 19 September 2024</td></tr> | ||||||||||||||||||||||||||||||||
| <tr><td> Discussion </td><td> https://github.com/conda-forge/cfep/pull/52 </td></tr> | ||||||||||||||||||||||||||||||||
| <tr><td> Implementation </td><td> NA </td></tr> | ||||||||||||||||||||||||||||||||
| </table> | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| ## Abstract | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| Bioconda (BC) and conda-forge (CF) have occasionally transferred package recipes between them. The `strict` channel | ||||||||||||||||||||||||||||||||
| priority recommended to BC users and used when building packages on the BC channel poses an | ||||||||||||||||||||||||||||||||
| issue when transfers occur, especially in the CF-to-BC direction. This proposal aims to | ||||||||||||||||||||||||||||||||
| define a policy for standard procedures when such transfers occur. Included in this is the introduction | ||||||||||||||||||||||||||||||||
| of a new label to be applied to all package builds that transfer CF-to-BC. | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| ## Specification | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| ### Bioconda to conda-forge (BC-to-CF) Transfers | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| 1. Packages on BC considered for transfer should be marked with a ["Move to conda-forge" label](https://github.com/bioconda/bioconda-recipes/labels/Move%20to%20Conda-Forge) in the `bioconda-recipes` repository. This label can be applied either in an Issue or a Pull Request. | ||||||||||||||||||||||||||||||||
| 2. A Pull Request should be create on CF's `staged-recipes` repository to add the recipe. A link to the labeled BC Issue/Pull Request should be included in the body of the CF Pull Request. | ||||||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This should only happen once the bioconda team has agreed to allow the transfer to minimize noise. The reference to the comment in the bioconda org allowing the transfer should be mandatory. |
||||||||||||||||||||||||||||||||
| 3. The linter will flag when adding a recipe already hosted by BC. This will trigger a message to [the **conda-forge/bioconda-recipes** team](https://github.com/orgs/conda-forge/teams/bioconda-recipes). | ||||||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is an implementation detail we don't need in the specification. Perhaps in the Example workflow section. |
||||||||||||||||||||||||||||||||
| 4. Upon approval by [the **conda-forge/bioconda-recipes** team](https://github.com/orgs/conda-forge/teams/bioconda-recipes), the Pull Request can then be reviewed and merged. | ||||||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is implicit if the suggestions to point 2 are applied. |
||||||||||||||||||||||||||||||||
| 5. A Pull Request should be created (or edited) on `bioconda-recipes` that removes the recipe. A link to the CF `staged-recipes` Pull Request should be documented in the Pull Request text. | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| ### conda-forge to Bioconda (CF-to-BC) Transfers | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| 1. An Issue should be created on the CF feedstock with the title "Move to Bioconda". [The **conda-forge/bioconda-recipes** team](https://github.com/orgs/conda-forge/teams/bioconda-recipes) can also be pinged for their input. | ||||||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same comment here about specifics, e.g. the title. |
||||||||||||||||||||||||||||||||
| 2. A Pull Request should be created on the `conda-forge/admin-requests` feedstock to label all previous builds of this feedstock with a `transferred-to-bioconda` label. It should also initiate a request from the Core team to archive the feedstock. [The **conda-forge/bioconda-recipes** team](https://github.com/orgs/conda-forge/teams/bioconda-recipes) should be pinged to review. | ||||||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This should only happen after explicit approval by the feedstock maintainers and the bioconda team. I think I'm inclined to mark them as broken instead of relabeling. Not sure what happens with the original URL once they are relabeled, but we want to avoid 404s in lockfiles. |
||||||||||||||||||||||||||||||||
| 3. A Pull Request on BC's `bioconda-recipes` should be created to add the recipe there. A link to this Pull Request should be added to the above Issue and Pull Request. | ||||||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This should happen before the admin-requests PR. Once approved (but before the merge), then the admin-requests PR should be submitted. Both PRs should be merged as simultaneously as possible. |
||||||||||||||||||||||||||||||||
| 4. Once a link is provided and a [**conda-forge/bioconda-recipes**](https://github.com/bioconda/bioconda-recipes) member has reviewed the Pull Request on [`conda-forge/admin-requests`](https://github.com/conda-forge/admin-requests), **conda-forge/core** should proceed to review and merge. | ||||||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We need an extra step here where all the conda-forge packages for that named are copied to bioconda. I think this can be done programmatically with little effort. |
||||||||||||||||||||||||||||||||
| 5. The BC `bioconda-recipes` Pull Request can be merged after [the **conda-forge/bioconda-recipes** team](https://github.com/orgs/conda-forge/teams/bioconda-recipes) has approved the `conda-forge/admin-requests` Pull Request. | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| ### Expected Implementation | ||||||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| 1. An new option `cfep21_label` will be added to [`conda-forge/admin-requests`](https://github.com/conda-forge/admin-requests). | ||||||||||||||||||||||||||||||||
| 2. Once implemented, the following information will be added to the conda-forge Documentation: | ||||||||||||||||||||||||||||||||
| - A section on BC-to-CF transfer will be added under "Maintainer Documentation > Contributing packages". | ||||||||||||||||||||||||||||||||
| - A section on CF-to-BC transfer will be added under "Maintainer Documentation > Maintaining packages". | ||||||||||||||||||||||||||||||||
| - A section on using the `label/transferred-to-bioconda` label to solve environments requiring the labelled artifacts will be added under "User Documentation > FAQ". | ||||||||||||||||||||||||||||||||
|
Comment on lines
+42
to
+44
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think we need to discuss the documentation sections that explicitly. |
||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| ## Motivation | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| BC package builds use a `strict` channel priority with the channels `conda-forge > bioconda > defaults`. This creates a situation that when a package transfers from CF to BC the presence of builds with `main` label in `conda-forge` masks the solver from considering builds in a lower priority channel. | ||||||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is conda-forge placed before
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Because previously, packages have only migrated from bioconda to conda-forge, especially CRAN packages that don't have any bioconda dependencies. To properly resolve them, the channel order is recommended in that way. At least I think that was the motivation, but I gladly stand corrected by anybody with more in-depth knowledge. |
||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| ## Rationale | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| This CFEP proposes to resolve CF-to-BC transfer issues by moving `conda-forge` builds off the `main` label and onto a dedicated label. This approach relies on a metadata update to hide the artifacts in `conda-forge` channel so that artifacts in `bioconda` channel become visible in `strict` channel priority. By using a dedicated label, end-users will also have a means of re-enabling search of these older packages by specifying the label when solving. | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| ## Alternatives | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| Transferring artifacts (e.g., as in [CFEP-3](https://github.com/conda-forge/cfep/blob/main/cfep-03.md), followed by removal) would also solve the end-user issues. However, this would can be regarded as a more drastic change compared to a labeling approach. | ||||||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I like this, tbh.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, this sounds like a viable alternative. |
||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| ## Other sections | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| Other relevant sections of the proposal. Common sections include: | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| * Specification -- The technical details of the proposed change. | ||||||||||||||||||||||||||||||||
| * Motivation -- Why the proposed change is needed. | ||||||||||||||||||||||||||||||||
| * Rationale -- Why particular decisions were made in the proposal. | ||||||||||||||||||||||||||||||||
| * Backwards Compatibility -- Will the proposed change break existing | ||||||||||||||||||||||||||||||||
| packages or workflows. | ||||||||||||||||||||||||||||||||
| * Alternatives -- Any alternatives considered during the design. | ||||||||||||||||||||||||||||||||
| * Sample Implementation -- Links to prototype or a sample implementation of | ||||||||||||||||||||||||||||||||
| the proposed change. | ||||||||||||||||||||||||||||||||
| * FAQ -- Frequently asked questions (and answers to them). | ||||||||||||||||||||||||||||||||
| * Resolution -- A short summary of the decision made by the community. | ||||||||||||||||||||||||||||||||
| * Reference -- Any references used in the design of the CFEP. | ||||||||||||||||||||||||||||||||
|
Comment on lines
+58
to
+72
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| ## Copyright | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| All CFEPs are explicitly [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/). | ||||||||||||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd suggest not using too specific terms or guidelines in the Specification, in case those specifics change over time. For example, I'd remove the label details here, and then add an "Example workflow" section where those details are mentioned. The Specification should only say "Create an issue in
bioconda/bioconda-recipesto notify the intent of transferring a package to conda-forge".