Optionally disable caddy

[hexmode]

We're moving wikiapiary back to WMCloud, but this means we'll be sitting behind WMCloud's reverse proxy that provides SSL termination. As a result, we don't need Caddy and would like to disable it.

For now, I have gotten around this by adding the following to docker-compose.override.yml:

  varnish:
    ports:
      - 80:80

I've looked, but this is not a way to remove ports using the override file, so the startup looks like this:

$ sudo canasta start
2023/05/20 18:31:38 Using /etc/canasta for configuration...
Starting Canasta installation ''...
Creating network "wikiapiary_default" with the default driver
Creating wikiapiary_caddy_1 ... 
Creating wikiapiary_caddy_1         ... error
Creating wikiapiary_db_1      ... 
Creating wikiapiary_elasticsearch_1 ... 
Host is already in use by another container
Creating wikiapiary_varnish_1       ... done
Creating wikiapiary_db_1            ... done
Creating wikiapiary_elasticsearch_1 ... done
Creating wikiapiary_web_1           ... done

ERROR: for caddy  Cannot start service caddy: driver failed programming external connectivity on endpoint wikiapiary_caddy_1 (bc65595f04698b9e4f8c0bb671b9840b1a5dc8eab2de79bf0b8d7d6f62c46190): Bind for 0.0.0.0:80 failed: port is already allocated
Encountered errors while bringing up the project.

[jeffw16]

This is a good, oft-requested feature. We need to make changes in CanastaWiki/Canasta and CanastaWiki/Canasta-DockerCompose + CanastaWiki/Canasta-Kubernetes.

[freephile]

The right way to do this in Docker Compose is to use profiles. (Examples in StackOverflow thread). How should this be done in Kubernetes? Would we implement something using Charts and Helm? Is there a preferred best practice besides the "upstream" approach used in WMF operations? https://wikitech.wikimedia.org/wiki/Local-charts

[freephile]

One 'hack' to disable Caddy would be the following in docker-compose-override.yml (using the -r flag to canasta create):

services:
  caddy:
    image: alpine:latest
    restart: "no" 
    entrypoint: ["echo", "Caddy is disabled"]
  varnish:
    ports:
      - 80:80

_ hack provided by @cicalese _

[freephile]

Although a docker-compose-override.yml workaround was implemented and merged CanastaWiki/Canasta-DockerCompose#46 the Canasta CLI still assumes that Caddy is running. The CLI is the recommended way to use Canasta, so that assumption runs deep (meaning the CLI needs to change, and documentation needs to change; along with changes to the Compose repo to implement this issue).

Note: Just a little nuance from the comment above; the implemented workaround in pull 46 still assigns the web ports to Caddy (not varnish), but since it's in an override file, the system administrator can manually disable it without altering the Canasta compose file.

I'm sure there are many use cases where an existing infrastructure will have some other TLS termination. Or, the target host is either air-gapped or some virtual environment without public networking where Caddy fails to do its magic. This means that the Caddy service, although a valuable component of the Canasta Stack, should be optional.

Caddy can be made optional by using the Docker Compose feature called 'profiles', part of the compose file services element, like so:

version: '3.7'
### snip ####
  caddy:
    image: docker.io/library/caddy:2.4.6-alpine
    profiles: ["tls"]
    restart: unless-stopped
    entrypoint: caddy run --config /etc/caddy/Caddyfile --adapter caddyfile
    environment:
      - MW_SITE_FQDN=${MW_SITE_FQDN:-localhost}
    volumes:
      - caddy-data:/data
      - ./config/Caddyfile:/etc/caddy/Caddyfile
### snip ###

Then, either of the following would start the (optional) Caddy service.

docker compose --profile tls up -d
COMPOSE_PROFILES=tls docker compose up -d

Since all services without named profiles get started by default, maybe we could get fancy with a definition like
profiles: ["${CADDY_PROFILE}"]
and an .env.example with the following

# Uncomment the next line to disable the Caddy application server and TLS endpoint in favor of your own infrastructure
# CADDY_PROFILE=caddy_NOT_enabled

This way, Caddy would be default ON (because the environment variable is undefined and thus the parameter is empty), but easy to disable by uncommenting a line in the .env

[freephile]

It's worth mentioning that Caddy is extremely smart about providing HTTPS even for 'localhost' - by creating a self-signed certificate. So "my system is local" or "air gapped" isn't truly a problem for Caddy. The only true use case for making Caddy optional is that there will be people who already have some Application Distribution Controller (ADC) or Load Balancer product where they manage their TLS endpoint.