Make ssh secure

[Panaetius]

Description

Currently all nodes use the same ssh key which is hardcoded. This is very insecure and should be replaced with something more robust.

[liehe]

I did not find much easier way to do it. For KubeFlow OpenMPI operator, they create a one-time ssh secret and use it for all workers. For Kube-openmpi, they also generate id_rsa/id_rsa.pub beforehand and append the values to values.yaml. hardcoding ssh keys might not be a problem as users can always overwrite them.

To be more secure, we can limit the roles of useraccount and have a standalone namespace for each jobs.

[Panaetius]

It would be cool if some keys were generated on the fly when installing with helm. But I don't think the helm template commands support enough scripting functionality.
Removing this from the 0.1.0 milestone