NO-ISSUE: OTA-1605 Create first e2e test OCP-42543 #1249

JianLi-RH · 2025-10-22T08:41:41Z

In this PR, I finished 2 things:

MustGetKubeClient() function helps us create an Clientset instance which can be used to connect to cluster.
Update the README.md file, we can follow it to create our e2e test cases.

/cc @wking @DavidHurta @PratikMahajan @hongkailiu @jhou1 @dis016 @jiajliu

DavidHurta · 2025-10-23T12:07:35Z

Please note that @petr-muller is no longer an active reviewer or approver in the CVO repository. You can refrain from directly pinging him on usual PRs where the expertise of the active folks should be sufficient. He is an "emeritus" approver.

What's an emeritus approver?

GitHub usernames listed under the emeritus_approvers key can no longer approve code (use the /approve command) and will be ignored by prow for assignment. However, it can still be referenced by a person looking at the OWNERS file for a possible second or more informed opinion.

openshift-ci-robot · 2025-10-24T09:20:53Z

@JianLi-RH: This pull request explicitly references no jira issue.

In response to this:

In this PR, finished first e2e test case. Related case can be found in: https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-42543

Tested it locally:
[jianl@jianl-thinkpadt14gen4 cvo]$ ginkgo --label-filter="42543"
Running Suite: CVO Suite - /home/jianl/1_code/cluster-version-operator/test/cvo
===============================================================================
Random Seed: 1761205561

Will run 1 of 1 specs
•

Ran 1 of 1 Specs in 1.585 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS

Ginkgo ran 1 suite in 3.310067262s
Test Suite Passed
[jianl@jianl-thinkpadt14gen4 cvo]$ 
Here is the old case in openshift-tests-private: https://github.com/openshift/openshift-tests-private/blob/f1f32ff1e9eeef4cb2cb4ae6549daa9306c6616a/test/extended/ota/cvo/cvo.go#L716-L759

There is a disadvantage here when I use client-go, it seems client-go does not support all Openshift features, for example it could not extract all manifest and it although support methods to get all resource types but it could not use these types to get resources. So in these PR I don't check all manifests but check some special objects.

Another work we need to do is improve the make update command to let it can embed all tests into binary.

/cc @petr-muller @DavidHurta @hongkailiu @wking @jhou1 @dis016 @jiajliu

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

JianLi-RH · 2025-10-24T09:46:35Z

Tested on my local machine:

[jianl@jianl-thinkpadt14gen4 cluster-version-operator]$ make update
hack/build-go.sh
Using version from git...
Building binaries into _output/linux/amd64
Building github.com/openshift/cluster-version-operator cluster-version-operator-tests binary (v1.0.0-1482-gd9440005-dirty)
Compressing the cluster-version-operator-tests binary
Building github.com/openshift/cluster-version-operator cluster-version-operator binary (v1.0.0-1482-gd9440005-dirty)
hack/update-test-metadata.sh
Using version from git...
successfully updated metadata
[jianl@jianl-thinkpadt14gen4 cluster-version-operator]$ _output/linux/amd64/cluster-version-operator-tests list
[
  {
    "name": "[Jira:Cluster Version Operator] cluster-version-operator-tests Author:jianl-High-42543-the removed resources are not created in a fresh installed cluster",
    "labels": {
      "42543": {},
      "High": {},
      "cvo": {}
    },
    "resources": {
      "isolation": {}
    },
    "source": "openshift:payload:cluster-version-operator",
    "lifecycle": "blocking",
    "environmentSelector": {}
  }
]
[jianl@jianl-thinkpadt14gen4 cluster-version-operator]$

Run the case:

[jianl@jianl-thinkpadt14gen4 cluster-version-operator]$ _output/linux/amd64/cluster-version-operator-tests list | jq '.[] | select(.name | contains("42543")) | .name' | xargs _output/linux/amd64/cluster-version-operator-tests run-test
  Running Suite:  - /home/jianl/1_code/cluster-version-operator
  =============================================================
  Random Seed: 1761292866 - will randomize all specs

  Will run 1 of 1 specs
  ------------------------------
  [Jira:Cluster Version Operator] cluster-version-operator-tests Author:jianl-High-42543-the removed resources are not created in a fresh installed cluster [cvo, High, 42543]
  /home/jianl/1_code/cluster-version-operator/test/cvo/cvo.go:22
    STEP: Validate resource with 'release.openshift.io/delete: "true"' annotation is not installed @ 10/24/25 16:01:06.862
  • [1.428 seconds]
  ------------------------------

  Ran 1 of 1 Specs in 1.428 seconds
  SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
  {
    "name": "[Jira:Cluster Version Operator] cluster-version-operator-tests Author:jianl-High-42543-the removed resources are not created in a fresh installed cluster",
    "lifecycle": "blocking",
    "duration": 1428,
    "startTime": "2025-10-24 08:01:06.858981 UTC",
    "endTime": "2025-10-24 08:01:08.287644 UTC",
    "result": "passed",
    "output": "  STEP: Validate resource with 'release.openshift.io/delete: \"true\"' annotation is not installed @ 10/24/25 16:01:06.862\n"
  }
]
[jianl@jianl-thinkpadt14gen4 cluster-version-operator]$

DavidHurta · 2025-10-27T11:40:58Z

/cc

hongkailiu · 2025-10-27T13:04:40Z

/cc

test/README.md

test/cvo/cvo.go

.gitignore

.openshift-tests-extension/openshift_payload_cluster-version-operator.json

test/cvo/cvo.go

JianLi-RH · 2025-10-29T04:09:09Z

After modification, the case is still valid:

[jianl@jianl-thinkpadt14gen4 cluster-version-operator]$ _output/linux/amd64/cluster-version-operator-tests run-test "[Jira:Cluster Version Operator] The cluster version operator the removed resources are not created in a fresh installed cluster"
  Running Suite:  - /home/jianl/1_code/cluster-version-operator
  =============================================================
  Random Seed: 1761710868 - will randomize all specs

  Will run 1 of 1 specs
  ------------------------------
  [Jira:Cluster Version Operator] The cluster version operator the removed resources are not created in a fresh installed cluster [cvo, High, 42543]
  /home/jianl/1_code/cluster-version-operator/test/cvo/cvo.go:24
    STEP: Service controller-manager-service should not be installed @ 10/29/25 12:07:48.862
    STEP: ClusterRoleBindings default-account-openshift-machine-config-operator should not be installed @ 10/29/25 12:07:49.759
    STEP: CronJobs machine-config-nodes-crd-cleanup should not be installed @ 10/29/25 12:07:50.024
  • [1.428 seconds]
  ------------------------------

  Ran 1 of 1 Specs in 1.429 seconds
  SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
  {
    "name": "[Jira:Cluster Version Operator] The cluster version operator the removed resources are not created in a fresh installed cluster",
    "lifecycle": "blocking",
    "duration": 1429,
    "startTime": "2025-10-29 04:07:48.858522 UTC",
    "endTime": "2025-10-29 04:07:50.287978 UTC",
    "result": "passed",
    "output": "  STEP: Service controller-manager-service should not be installed @ 10/29/25 12:07:48.862\n  STEP: ClusterRoleBindings default-account-openshift-machine-config-operator should not be installed @ 10/29/25 12:07:49.759\n  STEP: CronJobs machine-config-nodes-crd-cleanup should not be installed @ 10/29/25 12:07:50.024\n"
  }
]
[jianl@jianl-thinkpadt14gen4 cluster-version-operator]$

cmd/cluster-version-operator-tests/README.md

.openshift-tests-extension/openshift_payload_cluster-version-operator.json

test/utilities/connection.go

JianLi-RH · 2025-11-11T04:00:59Z

@hongkailiu @DavidHurta I just removed the case, right now the PR only finished one thing: MustGetKubeClient().
Then we can start creating test cases in new PRs.

JianLi-RH · 2025-11-11T07:53:38Z

/retest

JianLi-RH · 2025-11-11T09:25:31Z

/hold

I want to investigate more about client-go.

DavidHurta · 2025-11-11T18:47:58Z

/hold
I want to investigate more about client-go.

@JianLi-RH, in case the investigation is not done by the end of the day, you want simply more time to explore or are simply busy, I would like to ask you whether it would be fine for me to rename the sanity test in a separate PR. That specific change is blocking my openshift/origin#30316 PR. I am happy to rename the test personally in a separate PR to unblock my PR while we are working on this one.

JianLi-RH · 2025-11-12T01:11:13Z

@DavidHurta of course, go ahead, please. I am not sure when I can finish the investigation.

JianLi-RH · 2025-11-13T09:11:37Z

/unhold

Clientset instance can be used to connect to cluster, manage resources. With clientset instance we can evaluate the posibility of moving other tests to OTE. The reference to OTE framework: https://docs.google.com/document/d/1cFZj9QdzW8hbHc3H0Nce-2xrJMtpDJrwAse9H7hLiWk/edit?tab=t.0#heading=h.8cf3f4eii1q8

JianLi-RH · 2025-11-17T04:31:42Z

@hongkailiu @DavidHurta Just added back the case, please help review it. Here is the output:

$ ginkgo ./test/...
Running Suite: CVO Suite - /home/jianl/1_code/cluster-version-operator/test/cvo
===============================================================================
Random Seed: 1763353592

Will run 2 of 2 specs
•Test Authentication...
namespace: default
        - Test services...
        - Test RoleBinding...
        - Test CronJob...
success
namespace: kube-node-lease
        - Test services...
        - Test RoleBinding...
        - Test CronJob...
success
namespace: kube-public
        - Test services...
        - Test RoleBinding...
        - Test CronJob...
.....................
namespace: openshift-vsphere-infra
        - Test services...
        - Test RoleBinding...
        - Test CronJob...
success
•

Ran 2 of 2 Specs in 82.840 seconds
SUCCESS! -- 2 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS

Ginkgo ran 1 suite in 1m24.112444203s
Test Suite Passed

If there is resource installed unexpected, it will fail as below:

$ ginkgo ./test/...
Running Suite: CVO Suite - /home/jianl/1_code/cluster-version-operator/test/cvo
===============================================================================
Random Seed: 1763353871

Will run 2 of 2 specs
•Test Authentication...

------------------------------
• [FAILED] [1.522 seconds]
[Jira:"Cluster Version Operator"] cluster-version-operator-tests [It] should not install resources annotated with release.openshift.io/delete=true [cvo, Conformance, High, 42543]
/home/jianl/1_code/cluster-version-operator/test/cvo/cvo.go:20

  [FAILED] Unexpectedly installed authentication cluster which has release.openshift.io/delete annotation
  Expected
      <bool>: true
  not to be true
  In [It] at: /home/jianl/1_code/cluster-version-operator/test/cvo/cvo.go:30 @ 11/17/25 12:31:14.175
------------------------------

Summarizing 1 Failure:
  [FAIL] [Jira:"Cluster Version Operator"] cluster-version-operator-tests [It] should not install resources annotated with release.openshift.io/delete=true [cvo, Conformance, High, 42543]
  /home/jianl/1_code/cluster-version-operator/test/cvo/cvo.go:30

Ran 2 of 2 Specs in 1.522 seconds
FAIL! -- 1 Passed | 1 Failed | 0 Pending | 0 Skipped
--- FAIL: TestGinkgo (1.52s)
FAIL

Ginkgo ran 1 suite in 2.725034617s

Test Suite Failed

…ient-go openshift/client-go gives us more efficient functions to access cluster

After adding new OTE test cases, we need to update test metadata

openshift-ci · 2025-11-17T12:52:40Z

@JianLi-RH: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-agnostic-ovn	`4dbdb2c`	link	true	`/test e2e-agnostic-ovn`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 22, 2025

JianLi-RH force-pushed the implement_client_oc branch 4 times, most recently from a6a8ada to 37caff4 Compare October 23, 2025 07:39

JianLi-RH changed the title ~~[WIP] Create packages for connecting to a cluster~~ [WIP] OTA-1605 Create e2e test in CVO repo Oct 23, 2025

JianLi-RH force-pushed the implement_client_oc branch 4 times, most recently from a79646a to d944000 Compare October 23, 2025 09:32

JianLi-RH force-pushed the implement_client_oc branch from d944000 to 067cd4b Compare October 24, 2025 07:45

JianLi-RH changed the title ~~[WIP] OTA-1605 Create e2e test in CVO repo~~ [WIP] NO-ISSUE: OTA-1605 Create e2e test in CVO repo Oct 24, 2025

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Oct 24, 2025

JianLi-RH changed the title ~~[WIP] NO-ISSUE: OTA-1605 Create e2e test in CVO repo~~ NO-ISSUE: OTA-1605 Create e2e test in CVO repo Oct 24, 2025

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 24, 2025

JianLi-RH force-pushed the implement_client_oc branch from 067cd4b to bac7793 Compare October 27, 2025 03:16

openshift-ci bot requested a review from DavidHurta October 27, 2025 11:41

openshift-ci bot requested a review from hongkailiu October 27, 2025 13:04

hongkailiu reviewed Oct 28, 2025

View reviewed changes

JianLi-RH force-pushed the implement_client_oc branch from bac7793 to 51facb3 Compare October 29, 2025 04:06

JianLi-RH force-pushed the implement_client_oc branch from 51facb3 to 8e9da36 Compare October 30, 2025 01:44

hongkailiu reviewed Oct 30, 2025

View reviewed changes

JianLi-RH force-pushed the implement_client_oc branch 2 times, most recently from 72354d1 to 3a8631b Compare November 4, 2025 01:46

openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Nov 7, 2025

JianLi-RH force-pushed the implement_client_oc branch 2 times, most recently from 80af1c8 to 56a29e0 Compare November 11, 2025 03:58

JianLi-RH changed the title ~~NO-ISSUE: OTA-1605 Create e2e test in CVO repo~~ NO-ISSUE: OTA-1605 Create MustGetKubeClient function for e2e test Nov 11, 2025

JianLi-RH force-pushed the implement_client_oc branch 6 times, most recently from d2300d9 to f98e03c Compare November 13, 2025 09:10

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 13, 2025

JianLi-RH force-pushed the implement_client_oc branch 4 times, most recently from 878cbc8 to 1102f9c Compare November 17, 2025 04:26

JianLi-RH force-pushed the implement_client_oc branch from 1102f9c to cc98b16 Compare November 17, 2025 06:23

JianLi-RH changed the title ~~NO-ISSUE: OTA-1605 Create MustGetKubeClient function for e2e test~~ NO-ISSUE: OTA-1605 Create first e2e test OCP-42543 Nov 17, 2025

JianLi-RH force-pushed the implement_client_oc branch from cc98b16 to c168996 Compare November 17, 2025 09:06

Finished first E2E test case by openshift/client-go and Kubernetes/cl…

e35f937

…ient-go openshift/client-go gives us more efficient functions to access cluster

JianLi-RH force-pushed the implement_client_oc branch from c168996 to e35f937 Compare November 17, 2025 09:45

test: Manually adding test metadata

4dbdb2c

After adding new OTE test cases, we need to update test metadata

NO-ISSUE: OTA-1605 Create first e2e test OCP-42543 #1249

Are you sure you want to change the base?

NO-ISSUE: OTA-1605 Create first e2e test OCP-42543 #1249

Conversation

JianLi-RH commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DavidHurta commented Oct 23, 2025

Uh oh!

openshift-ci-robot commented Oct 24, 2025

Uh oh!

JianLi-RH commented Oct 24, 2025

Uh oh!

DavidHurta commented Oct 27, 2025

Uh oh!

hongkailiu commented Oct 27, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JianLi-RH commented Oct 29, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JianLi-RH commented Nov 11, 2025

Uh oh!

JianLi-RH commented Nov 11, 2025

Uh oh!

JianLi-RH commented Nov 11, 2025

Uh oh!

DavidHurta commented Nov 11, 2025

Uh oh!

JianLi-RH commented Nov 12, 2025

Uh oh!

JianLi-RH commented Nov 13, 2025

Uh oh!

JianLi-RH commented Nov 17, 2025

Uh oh!

openshift-ci bot commented Nov 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

JianLi-RH commented Oct 22, 2025 •

edited

Loading