Skip to content

feat: unify Docker registry handling via oras and adjust image processing order#71

Open
k1shk1n wants to merge 18 commits intoNetcracker:mainfrom
k1shk1n:feature/docker-registry-api-v2
Open

feat: unify Docker registry handling via oras and adjust image processing order#71
k1shk1n wants to merge 18 commits intoNetcracker:mainfrom
k1shk1n:feature/docker-registry-api-v2

Conversation

@k1shk1n
Copy link
Copy Markdown
Contributor

@k1shk1n k1shk1n commented Dec 24, 2025

Unify docker registries handling by using oras: support OCI Distribution Specs and Docker Registry HTTP API v2 (with Registry Authentication) and more native usage similar to docker cli

TODO:

  • Unify docker registries handling
  • Adjust image processing order like docker cli: order and default registries, repositories, tags
  • Manual testing of registries that are already supported
    • dockerhub: anon, non-anon
    • artifactory: anon, non-anon - instances are needed for testing
    • nexus: anon, non-anon
    • aws ecr: non-anon - instances are needed for testing
  • Manual testing of new registry types
    • ghcr
      • anon
      • non-anon
  • Add auto tests
  • Update documentation

Breaking changes:

  • (?) Removed --registry-type cli parameter from download, install and uninstall cli commands. ps may be we need to obtain this parameter (but not use it) so as not to break the processes of users who have hardcoded this parameter when using the cli in their orchestration (right now, any unknown parameter = error)

  1. removed custom handlers for different docker registries (nexus, artifactory, aws ecr, dockerhub) -> switch to oras responsibilities to work with different registries specs (docker, oci). Also removed --registry-type cli parameter
  2. The order of processing gear types has been changed from image, tar.gz, source -> source, tar.gz, image. The image check (matches with pattern) has also been removed, since the image is always the last option; we will simply try to work with it as an image
  3. Docker-like processing of the input image has been implemented. If no registry is specified, the default is docker.io (this was already implemented, but not quite correctly. Fixed). If no repository is specified for docker.io, library will be substituted; if no ref is specified, latest will be substituted (this was already implemented, but not quite correctly. Fixed). Added tests for it
  4. Remove localstack from devbox because aws ecr in localstack for basic and higher levels is not free
  5. Log improvements

test registries:

qrex@arch-pc ~/projects/python/gsf-fork git:(feature/docker-registry-api-v2 ~1)                                                                                                                                                           
(0) > gsf download ghcr.io/netcracker/qubership-gsf-discovery:feature_ns-filter-support_20251225-054807 --debug                                                                                                                           
                                                                                                                                                                                                                                          
    .-,                                                                                                                                                                                                                                   
 .^.: :.^.   ┏┓╻┳ ┏┓╻╻┏┓┳┏┓┏┳┓ ┏┓┏┓╻ ╻ ┏┓┏ ┓┏┓┳┓                                                                                                                                                                                          
,-' .-. '-,  ┃┓┃┃ ┗┓┗┃┗┓┃┣ ┃┃┃ ┣ ┃┃┃ ┃ ┃┃┃┃┃┣ ┣┛                                                                                                                                                                                          
'-. '-' .-'  ┗┛╹╹ ┗┛┗┛┗┛╹┗┛╹ ╹ ╹ ┗┛┗┛┗┛┗┛┗┻┛┗┛┛┗                                                                                                                                                                                          
 '.`; ;`.'   git-system-follower v0.6.6
    `-`
╭═1. Common parameters═════════════════════════════════════════════════════════════════════════════╮
  gears     = ghcr.io/netcracker/qubership-gsf-discovery:feature_ns-filter-support_20251225-054807
  directory = /home/qrex/projects/python/gsf-fork
  debug     = True
╰═════════════════════════════════════════════════════════════════════════════════════════total: 3═╯
╭═2. Registry parameters═══════════════════════════════════════════════════════════════════════════╮
  registry-username = 
  registry-password = 
  insecure-registry = False
╰═════════════════════════════════════════════════════════════════════════════════════════total: 3═╯
16:17:35.860 INFO     | :: Downloading packages
16:17:35.860 INFO     | -> Downloading ghcr.io/netcracker/qubership-gsf-discovery:feature_ns-filter-support_20251225-054807
16:17:35.860 DEBUG    | Temporary directory was created at /tmp/gsf-package-manager-yw9shy97
16:17:36.393 DEBUG    | Requesting anon token with params: {'service': 'ghcr.io', 'scope': 'repository:netcracker/qubership-gsf-discovery:pull'}
16:17:36.631 DEBUG    | Successfully obtained anonymous token!
16:17:36.938 DEBUG    | Found manifest:
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
  "config": {
    "mediaType": "application/vnd.docker.container.image.v1+json",
    "digest": "sha256:5396e5deb2a5e37919501fd62a5049ccad3f8bb4b13fc9b9faac7b0e8653acd7",
    "size": 1199
  },
  "layers": [
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "digest": "sha256:c00d43fbed253ce8cff4916e63103757e9c6d9d2145e7db20c660ff71b4e8bfa",
      "size": 3459
    }
  ]
}
16:17:37.745 DEBUG    | Downloaded sha256:c00d43fbed253ce8cff4916e63103757e9c6d9d2145e7db20c660ff71b4e8bfa layer to /tmp/gsf-package-manager-yw9shy97/gear.tar.gz
16:17:37.750 DEBUG    | Renamed /tmp/gsf-package-manager-yw9shy97/gear.tar.gz to /tmp/gsf-package-manager-yw9shy97/envgene_discovery_project@1.13.2.tar.gz
16:17:37.750 DEBUG    | Moved /tmp/gsf-package-manager-yw9shy97/envgene_discovery_project@1.13.2.tar.gz to envgene_discovery_project@1.13.2.tar.gz
16:17:37.751 INFO     | envgene_discovery_project@1.13.2 package is provided as docker image (Image: ghcr.io/netcracker/qubership-gsf-discovery:feature_ns-filter-support_20251225-054807)
16:17:37.751 SUCCESS  | Downloaded package from ghcr.io/netcracker/qubership-gsf-discovery:feature_ns-filter-support_20251225-054807 to envgene_discovery_project@1.13.2.tar.gz
16:17:37.752 DEBUG    | Temporary directory was removed at /tmp/gsf-package-manager-yw9shy97
16:17:37.754 WARNING  | Mismatch found in version of gear (feature_ns-filter-support_20251225-054807)and package.yaml (1.13.2)
16:17:37.755 INFO     | envgene_discovery_project@1.13.2 package is of simple structure type
16:17:37.755 SUCCESS  | Download complete

qrex@arch-pc ~/projects/python/gsf-fork git:(feature/docker-registry-api-v2 ?1 ~1)
(0) > ls -la | grep envgene
-rw-r--r--  1 qrex qrex  3459 Dec 25 16:17 envgene_discovery_project@1.13.2.tar.gz

@k1shk1n k1shk1n self-assigned this Dec 24, 2025
@k1shk1n k1shk1n added the enhancement New feature or request label Dec 24, 2025
@k1shk1n k1shk1n changed the title feat: summarizing work with docker registries - full use of oras for different specs feat: unify Docker registry handling via oras and adjust image processing order Dec 24, 2025
@k1shk1n k1shk1n marked this pull request as draft December 25, 2025 11:41
@k1shk1n k1shk1n marked this pull request as ready for review January 7, 2026 01:13
@komo0225 komo0225 self-requested a review as a code owner March 4, 2026 07:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant