Skip to content

Conversation

@aguidirh
Copy link
Contributor

@aguidirh aguidirh commented Jan 13, 2026

Description

This PR fixes a critical bug where mirroring operator catalogs to certain registries (like Nexus) fails with the error: Manifest list must be converted to type "application/vnd.docker.distribution.manifest.list.v2+json" to be written to destination, but we cannot modify it: "Instructed to preserve digests"

The root cause was that when oc-mirror rebuilds/filters operator catalogs, it sets PreserveDigests=true for all images. However, rebuilt catalogs have already been modified (filtered), so their digests differ from the original. When copying these rebuilt catalogs to registries that require Docker v2 manifest format conversion, the conversion is blocked by the PreserveDigests flag, causing the mirror operation to fail.

Solution: Set PreserveDigests=false for rebuilt operator catalogs, allowing necessary manifest format conversions while maintaining digest preservation for unmodified images (bundles, related images, etc.).

Github / Jira issue: OCPBUGS-73760

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Code Improvements (Refactoring, Performance, CI upgrades, etc)
  • Internal repo assets (diagrams / docs on github repo)
  • This change requires a documentation update on openshift docs

How Has This Been Tested?

Test Configuration:

  • Nexus OSS 3.37.3 running in a container with Docker registry on port 8082
  • ImageSet configuration targeting registry.redhat.io/redhat/redhat-operator-index:v4.17
  • oc-mirror v2 mirroring to Nexus registry

Test Steps:

  1. Set up Nexus with a Docker hosted repository
  2. Create an ImageSetConfiguration with operator catalogs requiring filtering
  3. Run: ./bin/oc-mirror -c <config>.yaml --workspace file://<path> docker://127.0.0.1:8082 --dest-tls-verify=false --v2
  4. Verify all operator images (including the rebuilt catalog) mirror successfully

Expected Outcome

Before the fix:

  • Mirroring fails with error: Manifest list must be converted to type "application/vnd.docker.distribution.manifest.list.v2+json" to be written to destination, but we cannot modify it: "Instructed to preserve digests"
  • Operator catalog image fails to copy to the destination registry
  • Related operator bundles are skipped due to catalog failure

After the fix:

  • ✅ All 74 operator images (including the rebuilt catalog) mirror successfully
  • ✅ Catalog successfully copies from cache to destination registry (Nexus)
  • ✅ No manifest conversion errors
  • ✅ Digest preservation still works for non-rebuilt images (bundles, related images)

Logs showing success:

2026/01/09 19:16:01 [INFO] : Success copying docker://registry.redhat.io/redhat/redhat-operator-index:v4.17 ➡️ 127.0.0.1:8082/redhat/
2026/01/09 19:16:03 [INFO] : Success copying docker://registry.redhat.io/redhat/redhat-operator-index:v4.17 ➡️ cache
2026/01/09 19:16:03 [INFO] : === Results ===

Additional Notes

This fix specifically targets rebuilt/filtered operator catalogs (identified by img.Type.IsOperatorCatalog() && img.RebuiltTag != ""). The change:

  • Does NOT affect digest preservation for operator bundles and related images
  • Only applies when a catalog has been rebuilt/filtered by oc-mirror
  • Allows manifest format conversion when required by the destination registry
  • Is safe because rebuilt catalogs already have different digests than the original

This resolves the issue for registries like Nexus that require Docker v2 manifest format but was previously blocked by the digest preservation constraint.

This bug fix is to avoid the issue docker://registry.redhat.io/redhat/redhat-operator-index:v4.17 error: Manifest list must be converted to type "application/vnd.docker.distribution.manifest.list.v2+json" to be written to destination, but we cannot modify it: "Instructed to preserve digests"

When a catalog image is filtered the digest is already different from
the original catalog, so it is fine to allow convertions.

Signed-off-by: Alex Guidi <aguidi@redhat.com>
@openshift-ci-robot openshift-ci-robot added jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jan 13, 2026
@openshift-ci-robot
Copy link

@aguidirh: This pull request references Jira Issue OCPBUGS-53455, which is invalid:

  • expected the bug to be open, but it isn't
  • expected the bug to target the "4.22.0" version, but no target version was set
  • expected the bug to be in one of the following states: NEW, ASSIGNED, POST, but it is Closed (Not a Bug) instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Description

This PR fixes a critical bug where mirroring operator catalogs to certain registries (like Nexus) fails with the error: Manifest list must be converted to type "application/vnd.docker.distribution.manifest.list.v2+json" to be written to destination, but we cannot modify it: "Instructed to preserve digests"

The root cause was that when oc-mirror rebuilds/filters operator catalogs, it sets PreserveDigests=true for all images. However, rebuilt catalogs have already been modified (filtered), so their digests differ from the original. When copying these rebuilt catalogs to registries that require Docker v2 manifest format conversion, the conversion is blocked by the PreserveDigests flag, causing the mirror operation to fail.

Solution: Set PreserveDigests=false for rebuilt operator catalogs, allowing necessary manifest format conversions while maintaining digest preservation for unmodified images (bundles, related images, etc.).

Github / Jira issue: OCPBUGS-53455

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Code Improvements (Refactoring, Performance, CI upgrades, etc)
  • Internal repo assets (diagrams / docs on github repo)
  • This change requires a documentation update on openshift docs

How Has This Been Tested?

Test Configuration:

  • Nexus OSS 3.37.3 running in a container with Docker registry on port 8082
  • ImageSet configuration targeting registry.redhat.io/redhat/redhat-operator-index:v4.17
  • oc-mirror v2 mirroring to Nexus registry

Test Steps:

  1. Set up Nexus with a Docker hosted repository
  2. Create an ImageSetConfiguration with operator catalogs requiring filtering
  3. Run: ./bin/oc-mirror -c <config>.yaml --workspace file://<path> docker://127.0.0.1:8082 --dest-tls-verify=false --v2
  4. Verify all operator images (including the rebuilt catalog) mirror successfully

Expected Outcome

Before the fix:

  • Mirroring fails with error: Manifest list must be converted to type "application/vnd.docker.distribution.manifest.list.v2+json" to be written to destination, but we cannot modify it: "Instructed to preserve digests"
  • Operator catalog image fails to copy to the destination registry
  • Related operator bundles are skipped due to catalog failure

After the fix:

  • ✅ All 74 operator images (including the rebuilt catalog) mirror successfully
  • ✅ Catalog successfully copies from cache to destination registry (Nexus)
  • ✅ No manifest conversion errors
  • ✅ Digest preservation still works for non-rebuilt images (bundles, related images)

Logs showing success:

2026/01/09 19:16:01 [INFO] : Success copying docker://registry.redhat.io/redhat/redhat-operator-index:v4.17 ➡️ 127.0.0.1:8082/redhat/
2026/01/09 19:16:03 [INFO] : Success copying docker://registry.redhat.io/redhat/redhat-operator-index:v4.17 ➡️ cache
2026/01/09 19:16:03 [INFO] : === Results ===

Additional Notes

This fix specifically targets rebuilt/filtered operator catalogs (identified by img.Type.IsOperatorCatalog() && img.RebuiltTag != ""). The change:

  • Does NOT affect digest preservation for operator bundles and related images
  • Only applies when a catalog has been rebuilt/filtered by oc-mirror
  • Allows manifest format conversion when required by the destination registry
  • Is safe because rebuilt catalogs already have different digests than the original

This resolves the issue for registries like Nexus that require Docker v2 manifest format but was previously blocked by the digest preservation constraint.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested review from adolfo-ab and r4f4 January 13, 2026 12:10
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 13, 2026
Signed-off-by: Alex Guidi <aguidi@redhat.com>
@openshift-ci
Copy link

openshift-ci bot commented Jan 13, 2026

@aguidirh: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@aguidirh aguidirh changed the title OCPBUGS-53455: fix conversion issue for catalog OCPBUGS-73760: fix conversion issue for catalog Jan 13, 2026
@openshift-ci-robot openshift-ci-robot removed the jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. label Jan 13, 2026
@openshift-ci-robot
Copy link

@aguidirh: This pull request references Jira Issue OCPBUGS-73760, which is invalid:

  • expected the bug to target either version "4.22." or "openshift-4.22.", but it targets "4.21.0" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Description

This PR fixes a critical bug where mirroring operator catalogs to certain registries (like Nexus) fails with the error: Manifest list must be converted to type "application/vnd.docker.distribution.manifest.list.v2+json" to be written to destination, but we cannot modify it: "Instructed to preserve digests"

The root cause was that when oc-mirror rebuilds/filters operator catalogs, it sets PreserveDigests=true for all images. However, rebuilt catalogs have already been modified (filtered), so their digests differ from the original. When copying these rebuilt catalogs to registries that require Docker v2 manifest format conversion, the conversion is blocked by the PreserveDigests flag, causing the mirror operation to fail.

Solution: Set PreserveDigests=false for rebuilt operator catalogs, allowing necessary manifest format conversions while maintaining digest preservation for unmodified images (bundles, related images, etc.).

Github / Jira issue: OCPBUGS-53455

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Code Improvements (Refactoring, Performance, CI upgrades, etc)
  • Internal repo assets (diagrams / docs on github repo)
  • This change requires a documentation update on openshift docs

How Has This Been Tested?

Test Configuration:

  • Nexus OSS 3.37.3 running in a container with Docker registry on port 8082
  • ImageSet configuration targeting registry.redhat.io/redhat/redhat-operator-index:v4.17
  • oc-mirror v2 mirroring to Nexus registry

Test Steps:

  1. Set up Nexus with a Docker hosted repository
  2. Create an ImageSetConfiguration with operator catalogs requiring filtering
  3. Run: ./bin/oc-mirror -c <config>.yaml --workspace file://<path> docker://127.0.0.1:8082 --dest-tls-verify=false --v2
  4. Verify all operator images (including the rebuilt catalog) mirror successfully

Expected Outcome

Before the fix:

  • Mirroring fails with error: Manifest list must be converted to type "application/vnd.docker.distribution.manifest.list.v2+json" to be written to destination, but we cannot modify it: "Instructed to preserve digests"
  • Operator catalog image fails to copy to the destination registry
  • Related operator bundles are skipped due to catalog failure

After the fix:

  • ✅ All 74 operator images (including the rebuilt catalog) mirror successfully
  • ✅ Catalog successfully copies from cache to destination registry (Nexus)
  • ✅ No manifest conversion errors
  • ✅ Digest preservation still works for non-rebuilt images (bundles, related images)

Logs showing success:

2026/01/09 19:16:01 [INFO] : Success copying docker://registry.redhat.io/redhat/redhat-operator-index:v4.17 ➡️ 127.0.0.1:8082/redhat/
2026/01/09 19:16:03 [INFO] : Success copying docker://registry.redhat.io/redhat/redhat-operator-index:v4.17 ➡️ cache
2026/01/09 19:16:03 [INFO] : === Results ===

Additional Notes

This fix specifically targets rebuilt/filtered operator catalogs (identified by img.Type.IsOperatorCatalog() && img.RebuiltTag != ""). The change:

  • Does NOT affect digest preservation for operator bundles and related images
  • Only applies when a catalog has been rebuilt/filtered by oc-mirror
  • Allows manifest format conversion when required by the destination registry
  • Is safe because rebuilt catalogs already have different digests than the original

This resolves the issue for registries like Nexus that require Docker v2 manifest format but was previously blocked by the digest preservation constraint.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@aguidirh
Copy link
Contributor Author

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jan 13, 2026
@openshift-ci-robot
Copy link

@aguidirh: This pull request references Jira Issue OCPBUGS-73760, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @nidangavali

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested a review from nidangavali January 13, 2026 14:59
@openshift-ci
Copy link

openshift-ci bot commented Jan 14, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adolfo-ab, aguidirh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants