Mshost stats by DaanHoogland · Pull Request #5588 · apache/cloudstack

DaanHoogland · 2021-10-19T12:40:43Z

Description

This PR implements statistics for ManagementServers. The code queries the local mxBeans and the filesystem /proc for information on the system. if then publishes this to all ManagementSevers that keep a list of the recent state of each other. Static data is stored in the database, so if this ManagementServer is down others can still query it's last boot/start/stop/shutdown data (and possibly more)

the listManagementServerMetrics API gathers the data from the internal list and augments it with any static data from the DB.

Doc PR: apache/cloudstack-documentation#256

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

so far only manual verification in a lab environment has been done.

DaanHoogland · 2022-04-20T13:09:02Z

@nvazquez @DaanHoogland the issues are following with simulator:

FAIL: test_list_management_server_metrics (integration.smoke.test_metrics_api.TestMetrics)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/rohit/lab/apache/cloudstack/test/integration/smoke/test_metrics_api.py", line 307, in test_list_management_server_metrics
    self.assertTrue(isinstance(metrics.lastserverstop, str))
AssertionError: False is not true

and

FAIL: test_list_usage_server_metrics (integration.smoke.test_metrics_api.TestMetrics)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/rohit/lab/apache/cloudstack/test/integration/smoke/test_metrics_api.py", line 326, in test_list_usage_server_metrics
    self.assertTrue(isinstance(metrics.lastheartbeat, str))
AssertionError: False is not true

so these are the same as in travis. but not in a nested env :( that is not a hopeful prospect.

yadvr · 2022-04-20T13:11:34Z

In a simulator based env, mgmt server is never stopped, perhaps the value is false/null? In case of trillian env mgmt server is restarted.

DaanHoogland · 2022-04-20T19:05:59Z

In a simulator based env, mgmt server is never stopped, perhaps the value is false/null? In case of trillian env mgmt server is restarted.

I think the other way around, but thanks for the clue. I think i fixed it now.

DaanHoogland · 2022-04-20T19:06:38Z

@blueorangutan package

acs-robot · 2022-04-20T19:08:06Z

Found UI changes, kicking a new UI QA build
@blueorangutan ui

blueorangutan · 2022-04-20T19:09:03Z

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

yadvr · 2022-04-20T19:14:15Z

@blueorangutan package

blueorangutan · 2022-04-20T19:15:03Z

@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with

SystemVM template(s). I'll keep you posted as I make progress.

blueorangutan · 2022-04-20T19:19:31Z

UI build: ✔️
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/5588 (SL-JID-1415)

acs-robot · 2022-04-20T19:20:28Z

Found UI changes, kicking a new UI QA build
@blueorangutan ui

blueorangutan · 2022-04-20T19:21:02Z

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan · 2022-04-20T19:32:02Z

UI build: ✔️
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/5588 (SL-JID-1416)

acs-robot · 2022-04-20T19:54:51Z

PR Coverage Report

CLASS	INSTRUCTION MISSED	INSTRUCTION COVERED	BRANCH MISSED	BRANCH COVERED	LINE MISSED	LINE COVERED





ConnectedAgentAttache	149	0	20	0	40	0

HostDaoImpl	4983	0	180	0	803	0









CloudStackContextLoaderListener	77	0	2	0	21	0
ListDbMetricsCmd	22	0	0	0	7	0
ListMgmtsMetricsCmd	35	0	0	0	9	0
ListUsageServerMetricsCmd	22	0	0	0	7	0


MetricsServiceImpl	2130	0	134	0	446	0
ClusterMetricsResponse	523	0	132	0	60	0
DbMetricsResponse	43	0	0	0	21	0

ManagementServerMetricsResponse	117	0	0	0	47	0
UsageServerMetricsResponse	23	0	0	0	11	0
ZoneMetricsResponse	501	0	126	0	56	0


ApiDBUtils	2374	0	210	0	592	0
ApiSessionListener	125	0	2	0	32	0
QueryManagerImpl	14177	0	1242	0	2395	0

ManagementServerJoinDaoImpl	3	0	0	0	1	0
ManagementServerJoinVO	57	0	0	0	19	0

ManagementServerHostStatsEntry	319	0	2	0	132	0
StatsCollector	2178	0	124	0	359	0

acs-robot · 2022-04-20T19:59:49Z

PR Coverage Report

CLASS	INSTRUCTION MISSED	INSTRUCTION COVERED	BRANCH MISSED	BRANCH COVERED	LINE MISSED	LINE COVERED





ConnectedAgentAttache	149	0	20	0	40	0

HostDaoImpl	4983	0	180	0	803	0









CloudStackContextLoaderListener	77	0	2	0	21	0
ListDbMetricsCmd	22	0	0	0	7	0
ListMgmtsMetricsCmd	35	0	0	0	9	0
ListUsageServerMetricsCmd	22	0	0	0	7	0


MetricsServiceImpl	2130	0	134	0	446	0
ClusterMetricsResponse	523	0	132	0	60	0
DbMetricsResponse	43	0	0	0	21	0

ManagementServerMetricsResponse	117	0	0	0	47	0
UsageServerMetricsResponse	23	0	0	0	11	0
ZoneMetricsResponse	501	0	126	0	56	0


ApiDBUtils	2374	0	210	0	592	0
ApiSessionListener	125	0	2	0	32	0
QueryManagerImpl	14177	0	1242	0	2395	0

ManagementServerJoinDaoImpl	3	0	0	0	1	0
ManagementServerJoinVO	57	0	0	0	19	0

ManagementServerHostStatsEntry	319	0	2	0	132	0
StatsCollector	2178	0	124	0	359	0

blueorangutan · 2022-04-20T21:07:43Z

Packaging result: ✔️ el7 ✔️ el8 ✖️ debian ✔️ suse15. SL-JID 3252

DaanHoogland · 2022-04-21T06:42:39Z

@blueorangutan test

blueorangutan · 2022-04-21T06:44:03Z

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2022-04-21T17:54:00Z

Trillian test result (tid-3966)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 38820 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5588-t3966-kvm-centos7.zip
Smoke tests completed. 93 look OK, 3 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_add_primary_storage_disabled_host	`Error`	0.74	test_primary_storage.py
test_01_primary_storage_nfs	`Error`	0.17	test_primary_storage.py
ContextSuite context=TestStorageTags>:setup	`Error`	0.31	test_primary_storage.py
test_03_deploy_and_scale_kubernetes_cluster	`Failure`	32.26	test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster	`Failure`	66.06	test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster	`Failure`	41.31	test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster	`Failure`	38.25	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	`Error`	127.84	test_kubernetes_clusters.py
test_hostha_enable_ha_when_host_in_maintenance	`Error`	302.97	test_hostha_kvm.py

github-actions · 2022-04-21T20:56:06Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

nvazquez · 2022-04-22T02:11:10Z

Hi @DaanHoogland can you please fix the conflict?

DaanHoogland · 2022-04-22T09:34:39Z

fixed @nvazquez

DaanHoogland · 2022-04-22T10:24:31Z

@blueorangutan package

blueorangutan · 2022-04-22T10:26:03Z

@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2022-04-22T11:30:11Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 3273

nvazquez · 2022-04-22T11:48:14Z

Merging based on approvals, tests results and ignoring intermittent known failures

DaanHoogland marked this pull request as draft October 19, 2021 12:40

GutoVeronezi reviewed Oct 20, 2021

View reviewed changes

apache deleted a comment from blueorangutan Oct 25, 2021

DaanHoogland requested a review from sureshanaparti October 26, 2021 06:29

sureshanaparti reviewed Oct 26, 2021

View reviewed changes

Comment thread api/src/main/java/org/apache/cloudstack/api/command/admin/management/ListMgmtsCmd.java Outdated

sureshanaparti reviewed Oct 26, 2021

View reviewed changes

Comment thread engine/schema/src/main/resources/META-INF/db/schema-41520to41600.sql Outdated

sureshanaparti reviewed Oct 26, 2021

View reviewed changes

Comment thread framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java Outdated

apache deleted a comment from blueorangutan Oct 26, 2021

GutoVeronezi reviewed Oct 26, 2021

View reviewed changes

DaanHoogland closed this Nov 5, 2021

DaanHoogland reopened this Nov 5, 2021

DaanHoogland force-pushed the mshostStats branch from be5685b to 0086e4f Compare November 5, 2021 08:01

apache deleted a comment from blueorangutan Nov 5, 2021

DaanHoogland added this to the 4.17.0.0 milestone Nov 15, 2021

DaanHoogland added complexity:normal component:management-server component:UI Severity:Minor type:new-feature labels Nov 15, 2021

DaanHoogland force-pushed the mshostStats branch from cb039ce to 0db1898 Compare November 15, 2021 12:20

apache deleted a comment from blueorangutan Nov 15, 2021

DaanHoogland force-pushed the mshostStats branch 2 times, most recently from c873397 to 3201158 Compare November 15, 2021 14:46

apache deleted a comment from blueorangutan Nov 15, 2021

apache deleted a comment from blueorangutan Nov 16, 2021

conditional assertion to deal with simulator restart

e2b91a4

Merge branch 'main' into mshostStats

164e6b2

weizhouapache mentioned this pull request Apr 22, 2022

DB: fix duplicated changes in schema-41610to41700.sql #6306

Merged

12 tasks

DaanHoogland mentioned this pull request Apr 25, 2022

remove superfluent counter and fix log message #6313

Merged

12 tasks

rajujith mentioned this pull request Dec 7, 2023

Management server metrics only shows on 1 GUI at a time #8235

Closed

Conversation

DaanHoogland commented Oct 19, 2021 • edited by sureshanaparti Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DaanHoogland commented Apr 20, 2022

Uh oh!

yadvr commented Apr 20, 2022

Uh oh!

DaanHoogland commented Apr 20, 2022

Uh oh!

DaanHoogland commented Apr 20, 2022

Uh oh!

acs-robot commented Apr 20, 2022

Uh oh!

blueorangutan commented Apr 20, 2022

Uh oh!

yadvr commented Apr 20, 2022

Uh oh!

blueorangutan commented Apr 20, 2022

Uh oh!

blueorangutan commented Apr 20, 2022

Uh oh!

acs-robot commented Apr 20, 2022

Uh oh!

blueorangutan commented Apr 20, 2022

Uh oh!

blueorangutan commented Apr 20, 2022

Uh oh!

acs-robot commented Apr 20, 2022

PR Coverage Report

Uh oh!

acs-robot commented Apr 20, 2022

PR Coverage Report

Uh oh!

blueorangutan commented Apr 20, 2022

Uh oh!

DaanHoogland commented Apr 21, 2022

Uh oh!

blueorangutan commented Apr 21, 2022

Uh oh!

blueorangutan commented Apr 21, 2022

Uh oh!

github-actions bot commented Apr 21, 2022

Uh oh!

nvazquez commented Apr 22, 2022

Uh oh!

DaanHoogland commented Apr 22, 2022

Uh oh!

DaanHoogland commented Apr 22, 2022

Uh oh!

blueorangutan commented Apr 22, 2022

Uh oh!

blueorangutan commented Apr 22, 2022

Uh oh!

nvazquez commented Apr 22, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

DaanHoogland commented Oct 19, 2021 •

edited by sureshanaparti

Loading