Start builds/tests over ssh/pywinrm in VagrantPlaybookCheck.sh

[Willsparker]

Currently, the scripts located in pbTestScripts used in the VagrantPlaybookCheck Jenkins job use ansible all ... as a way of starting the scripts to build / test a JDK. While this works, it doesn't output logs 'live'; It will only be output once the build / test has succeeded or failed, causing up to 2 hours of waiting, not knowing if the build / test is hung.
A way to fix this this would be to start builds and tests outside of ansible, using ssh for linux/freebsd and pywinrm for Windows.

[Willsparker]

Starting the scripts on the linux and freebsd vagrant VMs is relatively trivial as ssh keys are already put into the working directory ( https://github.com/AdoptOpenJDK/openjdk-infrastructure/blob/f6ba51beaa6db4f3585945ff046a5e41d5cc5753/ansible/pbTestScripts/vagrantPlaybookCheck.sh#L195 ) , and subsequently put into the VM's authorized_keys file on vagrant up.

Therefore, something such as

ssh -i id_rsa vagrant@$(cat playbooks/AdoptOpenJDK_Unix_Playbook/host.unx) "cd /vagrant/pbTestScripts && ./buildJDK.sh $buildURL $jdkToBuild $buildHotspot"

would do the job

[Willsparker]

Using pywinrm causes some other issues as some config will be required. It will already be installed and working on the machine as ansible uses it for communication.
Resources:
https://www.bloggingforlogging.com/2018/01/24/demystifying-winrm/
diyan/pywinrm#154

While it's fairly easy to setup the connection, following https://adamtheautomator.com/winrm-linux-remoting/ . However, that enables unencrypted communication on the Windows Host which is probably not a very good idea.

[sxa]

Also looks like you can't stream the output back in realtime using pywinrm, although someone has tried to implement it diyan/pywinrm#55 (But it was never merged) - we could try picking up that patch ...

[Willsparker]

https://github.com/Willsparker/pywinrm
I've put in the PR (whether it actually works or not... I don't know- I haven't touched python before), so we can see if that works. I'll see if I can install this to a VM to test it.

[Willsparker]

Using the above fork, I've been able to install the pywinrm module on a VM via:

git clone https://github.com/willsparker/pywinrm
sudo python setup.py install

To then send commands to a window VM, the following can be done:

$ python
>>> import winrm
>>> import sys
>>> s = winrm.Session('<windows_ip>', auth=('win_user', 'win_password'))
>>> s.run_ps("<Command>", sys.stdout, sys.stderr)

The PR @sxa linked above allowed the sys.stdout and sys.stderr to be passed into it. Some additional tweaking may be desirable possible to make the output slightly nicer/ more readable. i.e:

>>> s.run_ps("something_fake", sys.stdout, sys.stderr)
#< CLIXML
<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04"><Obj S="progress" RefId="0"><TN RefId="0"><T>System.Management.Automation.PSCustomObject</T><T>System.Object</T></TN><MS><I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj><Obj S="progress" RefId="1"><TNRef RefId="0" /><MS><I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj><S S="Error">something_fake : The term 'something_fake' is not recognized as the name of a _x000D__x000A_</S><S S="Error">cmdlet, function, script file, or operable program. Check the spelling of the _x000D__x000A_</S><S S="Error">name, or if a path was included, verify that the path is correct and try again._x000D__x000A_</S><S S="Error">At line:1 char:1_x000D__x000A_</S><S S="Error">+ something_fake_x000D__x000A_</S><S S="Error">+ ~~~~~~~~~~~~~~_x000D__x000A_</S><S S="Error">    + CategoryInfo          : ObjectNotFound: (something_fake:String) [], Comm _x000D__x000A_</S><S S="Error">   andNotFoundException_x000D__x000A_</S><S S="Error">    + FullyQualifiedErrorId : CommandNotFoundException_x000D__x000A_</S><S S="Error"> _x000D__x000A_</S></Objs><Response code 1, out "", err "something_fake : The">

However, I have been able to successfully to start the build script using this method, with it printing live stdout:

>>>  s.run_ps("cd C:/openjdk-infrastructure/ansible/pbTestScripts/ ; sh buildJDKWin.sh -v jdk11 ", sys.stdout, sys.stderr)
#< CLIXML
WORKSPACE not found, setting it to /tmp
...

[Willsparker]

While it's fairly easy to setup the connection, following https://adamtheautomator.com/winrm-linux-remoting/ . However, that enables unencrypted communication on the Windows Host which is probably not a very good idea.

Just realised that the unencrypted communication won't be an issue as the IP address of the Windows VM will only be available to the machine it's running on.

[Willsparker]

infra-vagrant-1 now has the patched version of pywinrm on it.
Attempting a VPC run with all the changes in it:
https://ci.adoptopenjdk.net/view/Tooling/job/VagrantPlaybookCheck/529/

[Willsparker]

Unfrotunately, installation wasn't as easy as previously thought -
python setup.py install seemed to only install for that ssh session, not permanently.
I ended up having to move the contents of pywinrm/build/lib to /usr/local/lib/python2.7/site-packages- allowing the Jenkins user to also use the patched version of pywinrm