Enumerate test scenarios

[basil]

Describe your use-case which is not covered by existing documentation.

Currently the automated tests for this repository just install the .deb or .rpm package by filename, verify that the service started, verify that Jenkins is serving HTTP requests, and verify that the service stops. This is a good start, but it is not a comprehensive set of test scenarios.

This task covers enumerating all scenarios that need automated testing to prevent regressions. Off the top of my head I can think of the following:

• Installing from HTTP rather than by raw .deb or .rpm, including GPG key checks
• Configuring SSL/TLS

If there are additional scenarios to be tested, please enumerate them here.

CC @MarkEWaite

Reference any relevant documentation, other materials or issues/pull requests that can be used for inspiration.

No response

[timja]

Upgrading from a previous release?

[MarkEWaite]

Enumerating all scenarios is beyond my depth. Some of the scenarios that might be nice to consider in one place or another:

• Install current release from official repository servers (deb, Red Hat rpm, SUSE rpm) then upgrade from local file (deb or rpm) (single version upgrade)

• Install older release (not current) from official repository servers and upgrade from local file (version upgrade across more than one version)

• Create a local repository server (https?), copy the package to the local repository server, install from the local repository server

• Use the local repository server to install a previous version, then upgrade from previous version through local repository server

• Use Eclipse Temurin (8, 11) installed at a non-default location instead of the Java version delivered by the operating system provider (to better match how we deliver Docker images with Eclipse Temurin)

• Customize contents of /etc/default/jenkins or its equivalent and confirm the customizations are honored. Add additional JAVA_ARGS and confirm that it works. Change the webroot. Set the HTTPS port or the HTTP port and confirm the setting is honored.

• Install and configure Jenkins with HTTPS served by Jenkins (as described in "HTTPS with an existing certificate")

• Install and configure Jenkins with an nginx reverse proxy (as described in "Using NGINX")

• Install and configure the HTTP/2 additions as described in "Using HTTP/2" and confirm that it is working (I've seen bug reports that the instructions at that location are wrong)

• Use Amazon Corretto (8, 11) installed at a non-default location instead of the Java version delivered by the operating system provider

I consider all of those optional, but would love to have tests of a few of them if possible.

[basil]

I consider all of those optional, but would love to have tests of a few of them if possible.

Great, because I wasn't volunteering to do any of this work. What's the best way to onboard you to contributing tests to this repository, Mark?

[MarkEWaite]

Great, because I wasn't volunteering to do any of this work. What's the best way to onboard you to contributing tests to this repository, Mark?

I don't think that I'll be able to contribute much to the tests in this repository. I might be able to squeeze some time and interest into the upgrade scenarios (since I think they are a good safeguard to precede the systemd transition), but I doubt I'd be able to do much else.

[basil]

I do think it would be good to do some knowledge transfer since there is a fair bit of complexity to the Ansible setup and I'd like to not be the only one who understands it.

[MarkEWaite]

I like the idea of knowledge transfer. Would you be comfortable doing it in a recorded session that we could optionally share on the Jenkins YouTube channel and possibly embed in a page on www.jenkins.io?

If not, I'm happy to take good notes instead.

[basil]

I'm somewhat squeamish about being recorded in videos but I'm happy to do something informal. If you or anyone else wants to hop on a call (having done some preparation like installing Docker and Python 3 and pulling the huge Docker images used in the tests) I'd be happy to show you the ropes.

[dduportal]

Hello 👋 I'm also interested into learning and being able to help on this topic.

[lemeurherve]

Same as @dduportal 😃