Simulator: Better VR Redundant Status Behaviour

[richardlawley]

Description

Before this change, when creating a HA network in the Simulator, both Routers would be in BACKUP mode. Many years ago in f0c3b4c, logic was added to allow the line "router_pr=100" to be added to the bootargs field of mockvm in the Simulator database to allow a VR to become MASTER. This doesn't work automatically (nowhere sets it) and actually fails because the field is too short, and no other statuses can be simulated.

With this PR, Redundant State now comes from the "redundate_state=MASTER" part in bootargs, which is already populated.

Additionally, the Start/Reboot/Stop VM code of the simulator now incldues logic to change the redundant state:

• When Starting a VR, if no other VRs are Running and MASTER on this network, become MASTER
• When Stopping a VR, if we are MASTER become BACKUP. If another VR is running and BACKUP, make it MASTER.
• When Rebooting a VR, combine this logic (Stop/Start)

This will allow integration tests to easily change the state of a VR.

With this PR, the simulator will change the redundant state of a VR when it is started or stopped. When a VR is started, if there are no other running VRs that are MASTER then

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):

How Has This Been Tested?

• Deploy Simulator with advanced config
• Create & Enable HA Network Offering
• Deploy Network and a VM, check Network > Virtual Appliances page and observe that 2 VRs are present, one MASTER and the other BACKUP
• Stop MASTER VR. Observe that other VR becomes MASTER
• Stop other VR. Both will internally become BACKUP, but this isn't displayed when VRs are stopped
• Start original VR. Observe that it becomes MASTER
• Start second VR. Observe that it becomes BACKUP
• Reboot MASTER VR. Observe that the states swap.

[yadvr]

Closed-reopend PR to rekick Travis
@blueorangutan package

[blueorangutan]

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2798

[yadvr]

Smoketests are not necessary as changes are purely in simulator, Travis tests should be enough to merge this.

[yadvr]

Pinging for review - @anuragaw @shwstppr

[yadvr]

Checked LGTM, no explicit smoketests needed as changes are in simulator code that is tested by Travis.

[shwstppr]

LGTM based on changes

[yadvr]

@richardlawley can you look at the failing marvin tests in Travis and fix or advise if they are related to this PR:
https://travis-ci.org/apache/cloudstack/builds/537778478

[anuragaw]

LGTM
(based on code)

[yadvr]

Closed reopen the PR to rekick Travis tests. I believe few failures seen in the travis run were caused by PR changes, can you test/confirm @richardlawley ?

[yadvr]

ping @richardlawley

[richardlawley]

I will get to this, but my other PR is higher on my priority list as this doesn't offer any functionality for 4.13 - it was there so that I could add tests for the password reset only working for one VR issue.

[yadvr]

@richardlawley Okay, I'll move this away from 4.13. If you manage to fix the issues before 4.13 RC1 ping me to change the milestone. Thanks.

[yadvr]

I'll close+reopen the PR to rekick Travis and see if it passes with the simulator related changes

[andrijapanicsb]

ping @richardlawley can you do any needed changes/fixes if you need this in 4.14 please - or shall we move it to 4.15 milestone?

[DaanHoogland]

@andrijapanicsb , this is just simulator code, no production code. I think we can merge this and forward fix if issues are revealed.

[andrijapanicsb]

You know better @DaanHoogland - if you feel so, then please move forward with it. (the reason I asked is due to some Travis tests failing, per other guys words due to those changes?)

[DaanHoogland]

ok, investigating quickly @andrijapanicsb

[DaanHoogland]

All jobs failed due to log length, try insanity to see if that helps.

[andrijapanicsb]

OK, merging based on 3 x LGTM

[yadvr]

@richardlawley can you have a look at it again and re-submit a new PR. It seem this PR breaks Travis/simulator for some of the tests you can see https://travis-ci.org/apache/cloudstack/builds/622114507?utm_source=github_status&utm_medium=notification
cc @andrijapanicsb @DaanHoogland