Tango on Docker architecture evolution proposal

Below is a list of points (from user/ developer perspective) that we should consider when defining the shape of the "official" Tango Docker images.

Images

• Use proper names for the containers
  • e.g. tango-libtango, tango-databaseds, tango-starter instead of tango-cpp
  • the user will immediately know what is inside the image
• Container (and image) per-application with proper ENTRYPOINT and CMD
  • e.g. this command should start DataBaseds on default port: docker run -e MYSQL_HOST=... tango-databaseds
  • images will be smaller (e.g. no Starter included in DataBaseds container)
  • images will simpler to use (no need to know executable name and path)
• Use upstream sources to build the images
  • if a bug is dedected and fixed in e.g. DataBaseds, user does not need to wait for a new Tango Source Distribution release
  • prerequisite for having "image-per-application" (source distribution is a single package with everything)
• Publish images with proper tags
  • FROM tango-libtango:9.3.3, ...
  • docker run tango-databaseds:5.14, docker run tango-databaseds:5.13, ...
  • the user will have control over the software that he is running
• Allow to specify different SW versions during build
  • docker build --build-arg ZMQ_VERSION=4.0.5 OMNIORB_VERSION=4.1.6 -t tango-libtango:like-in-debian-jessie .
  • docker build --build-arg VERSION=5.12 LIBTANGO_TAG=like-in-debian-jessie -t tango-databaseds:custom-dev .
  • may be useful for development and device server testing in CI on different configurations
• Remove sudo
  • having user with sudo access and no password is no more secure than running as root directly
• Remove APT proxy
  • proxy configuration is currently duplicated in each image in all steps calling apt-get
  • if possible, try to reorder layers to use Docker caching mechanism (and use --cache-from)
  • if not, try to configure it externally (or at least with --build-arg)
  • if not, try to move it to a base image (or external image and --copy from it) to reduce duplication
• Reduce size of Python (PyTango) image
  • runtime image shall only contain: libtango, pytango and pip (+ mandatory dependencies)
• Simplify Java images
  • remove tango-java image (which is jre + tango source distribution) (or change it to some minimal java image + JTango jar)
  • applications that need JTango library can specify dependency in their pom.xml (or whatever they use)
  • for jive, pogo, etc. there are fat jars on bintray. Those jars shall be put in some minimal java containers
• Use Alpine Linux
  • huge reduction in container size
  • important for CI and with low bandwidth connections

To assess the feasibility of running Tango on Alpine Linux, some PoC images were created (buildenv, libtango, databaseds): https://github.com/S2Innovation/tango-docker

• PoC: libtango container (cppTango + dependencies): 25 MB, Databaseds: another 25MB on top of that (50 MB in total)
• SKA's tango-cpp is 317MB (including 69MB debian base image)

CI infrastructure

• adopt the Docker Repo pattern:
  • https://medium.com/mobileforgood/patterns-for-continuous-integration-with-docker-on-travis-ci-71857fff14c5
  • dedicated repository for Dockerfiles
    • maybe: tango-controls/tango-docker
    • let's keep tango-controls/tango-cs-docker for old images
• all Dockerfiles stored in this repo
  • easier to maintain (single issue tracker, etc. ...) - especially that there are dependencies between images
  • easier to detect changes in base image (libtango) and rebuild dependent images
• build Dockerfiles in Travis CI and publish to Docker Hub
• after new application release/tag (e.g. TangoDatabase):
  • someone (developer) will have to manually update version in Dockerfile and trigger CI
  • alternative:
    • configure CI for each project (TangoDatabase, TangoTest, ...) to trigger build in the Dockerfile repo when new tag is created

Proposed dependency diagram