cks: assorted fixes, test refactoring by shwstppr · Pull Request #4269 · apache/cloudstack

shwstppr · 2020-08-17T03:34:55Z

Description

Fixes #4265
Fixes for:

template selection
network cleanup
capacity check
specific patch version during deployment

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):

How Has This Been Tested?

yadvr · 2020-08-17T10:44:51Z

+            "minmemory": 2048
        }
+    },
+    "cks_templates": {


@shwstppr you'll need to sync changes in test_data in trillian specific test_data as well

Have done that

yadvr · 2020-08-20T11:54:02Z

@shwstppr is this ready for testing?

blueorangutan · 2020-08-20T12:47:02Z

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-1793

blueorangutan · 2020-08-20T14:16:02Z

Trillian test result (tid-2524)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 1338 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4269-t2524-kvm-centos7.zip
Smoke tests completed. 0 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

blueorangutan · 2020-08-20T15:19:04Z

Trillian test result (tid-2526)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 1296 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4269-t2526-kvm-centos7.zip
Smoke tests completed. 0 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

yadvr · 2020-08-25T09:54:24Z

@blueorangutan package

blueorangutan · 2020-08-25T09:55:41Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-08-25T11:07:10Z

Packaging result: ✔centos7 ✖centos8 ✔debian. JID-1816

yadvr · 2020-08-25T11:23:16Z

@blueorangutan package

yadvr · 2020-08-28T06:38:16Z

@blueorangutan package

blueorangutan · 2020-08-28T06:38:49Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-08-28T07:29:04Z

Packaging result: ✔centos7 ✖centos8 ✔debian. JID-1834

yadvr · 2020-08-28T08:02:37Z

@blueorangutan test

blueorangutan · 2020-08-28T08:03:48Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2020-08-31T09:07:24Z

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-1853

blueorangutan · 2020-08-31T14:29:06Z

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-1854

blueorangutan · 2020-08-31T19:44:20Z

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-1856

shwstppr · 2020-09-01T10:01:24Z

@blueorangutan test

blueorangutan · 2020-09-01T10:02:07Z

@shwstppr a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2020-09-02T05:44:31Z

Trillian test result (tid-2633)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 68413 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4269-t2633-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_certauthority_root.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 81 look OK, 4 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_provision_certificate	`Failure`	48.28	test_certauthority_root.py
test_01_deploy_kubernetes_cluster	`Error`	0.05	test_kubernetes_clusters.py
test_02_deploy_kubernetes_ha_cluster	`Error`	33.15	test_kubernetes_clusters.py
test_05_deploy_and_upgrade_kubernetes_ha_cluster	`Error`	0.04	test_kubernetes_clusters.py
test_06_deploy_and_invalid_upgrade_kubernetes_cluster	`Error`	0.04	test_kubernetes_clusters.py
test_07_deploy_and_scale_kubernetes_cluster	`Error`	0.04	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	`Error`	41.16	test_kubernetes_clusters.py
test_04_rvpc_network_garbage_collector_nics	`Error`	3836.86	test_vpc_redundant.py
test_hostha_kvm_host_fencing	`Error`	167.92	test_hostha_kvm.py

blueorangutan · 2020-09-02T06:55:45Z

Packaging result: ✖centos7 ✖centos8 ✖debian. JID-1881

yadvr · 2020-09-02T07:12:04Z

@blueorangutan package

blueorangutan · 2020-09-02T07:13:13Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

shwstppr · 2020-09-15T13:50:05Z

@blueorangutan test

blueorangutan · 2020-09-15T13:51:18Z

@shwstppr a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2020-09-16T06:20:02Z

Trillian test result (tid-2751)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 56453 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4269-t2751-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_deploy_kubernetes_cluster	`Error`	3613.82	test_kubernetes_clusters.py
test_02_invalid_upgrade_kubernetes_cluster	`Error`	3616.58	test_kubernetes_clusters.py
test_03_deploy_and_upgrade_kubernetes_cluster	`Error`	0.05	test_kubernetes_clusters.py
test_04_deploy_and_scale_kubernetes_cluster	`Error`	0.06	test_kubernetes_clusters.py
test_05_delete_kubernetes_cluster	`Error`	0.05	test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster	`Error`	0.05	test_kubernetes_clusters.py
test_08_deploy_and_upgrade_kubernetes_ha_cluster	`Error`	0.06	test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster	`Error`	0.06	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	`Error`	130.63	test_kubernetes_clusters.py

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

blueorangutan · 2020-09-16T20:07:24Z

Packaging result: ✔centos7 ✖centos8 ✔debian. JID-2019

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

blueorangutan · 2020-09-17T07:43:22Z

Packaging result: ✔centos7 ✖centos8 ✔debian. JID-2025

yadvr · 2020-09-18T02:55:39Z

@blueorangutan test

blueorangutan · 2020-09-18T02:56:29Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2020-09-18T22:23:30Z

Trillian test result (tid-2777)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 67746 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4269-t2777-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_outofbandmanagement.py
Smoke tests completed. 81 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_deploy_kubernetes_cluster	`Failure`	3625.80	test_kubernetes_clusters.py
test_04_deploy_and_scale_kubernetes_cluster	`Failure`	31.88	test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster	`Failure`	54.85	test_kubernetes_clusters.py
test_08_deploy_and_upgrade_kubernetes_ha_cluster	`Failure`	0.07	test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster	`Failure`	0.06	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	`Error`	99.27	test_kubernetes_clusters.py
test_oobm_issue_power_cycle	`Error`	15.95	test_outofbandmanagement.py
test_oobm_issue_power_soft	`Error`	16.42	test_outofbandmanagement.py
test_oobm_zchange_password	`Error`	9.84	test_outofbandmanagement.py

yadvr · 2020-09-21T08:43:30Z

@shwstppr unfortunately, tests are still intermittently failing

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

blueorangutan · 2020-09-21T16:04:30Z

Packaging result: ✔centos7 ✖centos8 ✔debian. JID-2057

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

blueorangutan · 2020-09-22T18:29:40Z

Packaging result: ✔centos7 ✖centos8 ✔debian. JID-2067

blueorangutan · 2020-09-23T05:15:00Z

Trillian test result (tid-2838)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34704 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4269-t2838-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_pvlan.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_create_pvlan_network	`Error`	0.04	test_pvlan.py

yadvr · 2020-09-23T05:16:18Z

@davidjumani the pvlan test seems to be failing sometimes, can you help debug if it's env related or an intermittent failure?

davidjumani · 2020-09-23T06:17:20Z

@rhtyd Looks like it's because the same vlan id is used elsewhere. Changed in #4340

yadvr · 2020-09-24T07:40:34Z

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2020-09-24T07:41:19Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

blueorangutan · 2020-09-24T20:16:13Z

Trillian test result (tid-2848)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43270 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4269-t2848-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Smoke tests completed. 79 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

yadvr

LGTM, CKS tests passing

Pearl1594

LGTM
Did some basic tests:

deployed a k8s cluster
destroyed the k8s cluster
Deleted the k8s template and re-registered another one with the same name as defined earlier (and as defined in the global setting) and verified that it picks up the new one and DOES NOT throw an error of being unable to find the template

yadvr added this to the 4.15.0.0 milestone Aug 17, 2020

yadvr added the Severity:Critical Critical bug label Aug 17, 2020

yadvr reviewed Aug 17, 2020

View reviewed changes

yadvr closed this Aug 25, 2020

yadvr reopened this Aug 25, 2020

shwstppr force-pushed the cks-test-refactoring branch from d79d8af to 72b1691 Compare August 31, 2020 13:56

yadvr closed this Sep 1, 2020

yadvr reopened this Sep 1, 2020

improvements

4b4dcea

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

fix

d29d177

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

temp change to mention version

96aa16f

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

changes for specifying k8s version

7e90a05

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr marked this pull request as ready for review September 25, 2020 11:35

yadvr approved these changes Sep 25, 2020

View reviewed changes

yadvr requested a review from DaanHoogland September 25, 2020 12:31

yadvr modified the milestones: 4.15.0.0, 4.14.1.0 Sep 25, 2020

Pearl1594 approved these changes Sep 29, 2020

View reviewed changes

yadvr merged commit 9391fa9 into apache:4.14 Sep 29, 2020

DaanHoogland added type:bug type:cleanup labels Oct 21, 2020

Conversation

shwstppr commented Aug 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

yadvr Aug 17, 2020

Choose a reason for hiding this comment

Uh oh!

shwstppr Aug 21, 2020

Choose a reason for hiding this comment

Uh oh!

yadvr commented Aug 20, 2020

Uh oh!

blueorangutan commented Aug 20, 2020

Uh oh!

blueorangutan commented Aug 20, 2020

Uh oh!

blueorangutan commented Aug 20, 2020

Uh oh!

yadvr commented Aug 25, 2020

Uh oh!

blueorangutan commented Aug 25, 2020

Uh oh!

blueorangutan commented Aug 25, 2020

Uh oh!

yadvr commented Aug 25, 2020

Uh oh!

yadvr commented Aug 28, 2020

Uh oh!

blueorangutan commented Aug 28, 2020

Uh oh!

blueorangutan commented Aug 28, 2020

Uh oh!

yadvr commented Aug 28, 2020

Uh oh!

blueorangutan commented Aug 28, 2020

Uh oh!

blueorangutan commented Aug 31, 2020

Uh oh!

blueorangutan commented Aug 31, 2020

Uh oh!

blueorangutan commented Aug 31, 2020

Uh oh!

shwstppr commented Sep 1, 2020

Uh oh!

blueorangutan commented Sep 1, 2020

Uh oh!

blueorangutan commented Sep 2, 2020

Uh oh!

blueorangutan commented Sep 2, 2020

Uh oh!

yadvr commented Sep 2, 2020

Uh oh!

blueorangutan commented Sep 2, 2020

Uh oh!

shwstppr commented Sep 15, 2020

Uh oh!

blueorangutan commented Sep 15, 2020

Uh oh!

blueorangutan commented Sep 16, 2020

Uh oh!

blueorangutan commented Sep 16, 2020

Uh oh!

blueorangutan commented Sep 17, 2020

Uh oh!

yadvr commented Sep 18, 2020

Uh oh!

blueorangutan commented Sep 18, 2020

Uh oh!

blueorangutan commented Sep 18, 2020

Uh oh!

yadvr commented Sep 21, 2020

Uh oh!

blueorangutan commented Sep 21, 2020

Uh oh!

blueorangutan commented Sep 22, 2020

Uh oh!

shwstppr commented Aug 17, 2020 •

edited

Loading

davidjumani commented Sep 23, 2020 •

edited

Loading