-
Notifications
You must be signed in to change notification settings - Fork 9
Description
Component(s): job-scheduler-ofelia, reverse-proxy-nginx
Environment: Docker Swarm (single node), prod mode, reverse-proxy in SECURE mode
Commit / Reference: 7a9bc0f2 (reverse-proxy conf)
Impact: Let’s Encrypt certificate renewal job does not run → certificates may not renew → potential downtime when certs expire.
Description
While bringing up the stack with the reverse proxy in SECURE mode, the Ofelia job scheduler starts but aborts with:
[Job Scheduler Ofelia] WARNING: config.ini file does not exist, Aborting...
This prevents the scheduled certificate renewal job from running. The reverse-proxy Nginx component appears to be prepared for ACME/renewal handling, but without Ofelia’s job the certs will not be renewed.
Relevant references in the repo:
- Nginx secure temp config referencing certificate handling / ACME paths:
location /.well-known/acme-challenge/ { - Ofelia README (notes around config file usage):
https://github.com/jembi/platform/blob/main/job-scheduler-ofelia/README.md - Example Ofelia config that defines the renewal job:
volume = renew-certbot-conf:/instant
The Ofelia README contains this caveat:
Ofelia does not support config.ini files when run in docker mode (which enables scheduling jobs with docker labels) thus we need to always use the config.ini file for creating jobs
Given the warning and behavior, the deployed container is expecting a config.ini, but it’s not present/mounted.
What I did
I discovered a remote-deployment-oriented file with template variables, adapted it, and brought up the stack. After that, I saw Ofelia warning that it was not configured.
Observed Logs (from bring-up)
🚀 Initializing package Job Scheduler Ofelia (job-scheduler-ofelia)...
🛠️ Config set for Job Scheduler Ofelia (job-scheduler-ofelia):
┌─────────┬──────────────────────────────┬──────────────────────────────────────────────────────┐
│ (index) │ Environment Variable │ Current Value │
├─────────┼──────────────────────────────┼──────────────────────────────────────────────────────┤
│ 0 │ 'JOB_SCHEDULER_OFELIA_IMAGE' │ 'mcuadros/ofelia:v0.3.6' │
│ 1 │ 'RENEWAL_EMAIL' │ 'dummy@jembi.org' │
│ 2 │ 'STAGING' │ 'false' │
│ 3 │ 'DOMAIN_NAME' │ 'jembi.fhirlab.net' │
│ 4 │ 'SUBDOMAINS' │ 'openhimcomms.jembi.fhirlab.net,openhimco...[trunc]' │
│ 5 │ 'OPENHIM_MONGO_URL' │ 'mongodb://mongo-1:27017/openhim' │
│ 6 │ 'REVERSE_PROXY_STACK_NAME' │ 'reverse-proxy' │
└─────────┴──────────────────────────────┴──────────────────────────────────────────────────────┘
ℹ️ [Message Bus Kafka] Running package in Single node mode
ℹ️ [Message Bus Kafka] Running package in PROD mode
ℹ️ [Interoperability Layer Openhim] Running package in Single node mode
ℹ️ [Interoperability Layer Openhim] Running package in PROD mode
ℹ️ [Reverse Proxy Nginx] Running package
ℹ️ [Message Bus Kafka] Deploy Kafka
ℹ️ [Database Postgres] Running package in Single node mode
ℹ️ [Reverse Proxy Nginx] Running package in SECURE mode
🔶 [Job Scheduler Ofelia] WARNING: config.ini file does not exist, Aborting...
Expected Behavior
- Ofelia should start with a valid
config.iniand schedule the renew-certs job. - Certificates should be renewed automatically per schedule (e.g., every 60 days), and reverse-proxy Nginx should serve/rotate to renewed certs without manual intervention.
Actual Behavior
- Ofelia aborts due to missing
config.ini. No renewal job is scheduled/executed. - Reverse proxy is running in SECURE mode, but renewal pipeline is inoperative.
Evidence in Repo (example job definition)
From job-scheduler-ofelia/config.example.ini (indicates that job-scheduler-ofelia is the component responsible for renewal):
[job-run "renew-certs"]
schedule = @every 1440h ;60 days
image = jembi/swarm-nginx-renewal:v1.0.0
volume = renew-certbot-conf:/instant
volume = /var/run/docker.sock:/var/run/docker.sock:ro
environment = RENEWAL_EMAIL=${RENEWAL_EMAIL}
environment = STAGING=${STAGING}
environment = DOMAIN_NAME=${DOMAIN_NAME}
environment = SUBDOMAINS=${SUBDOMAINS}
environment = REVERSE_PROXY_STACK_NAME=${REVERSE_PROXY_STACK_NAME}
delete = true- README context: https://github.com/jembi/platform/blob/main/job-scheduler-ofelia/README.md
- Example config context:
volume = renew-certbot-conf:/instant - Nginx secure temp config re ACME/renewal handling:
location /.well-known/acme-challenge/ {
Steps to Reproduce
- Deploy the stack with reverse-proxy Nginx in SECURE mode (single node, prod mode).
- Include the job-scheduler-ofelia service (e.g.,
mcuadros/ofelia:v0.3.6) with the environment variables as above (RENEWAL_EMAIL, STAGING, DOMAIN_NAME, SUBDOMAINS, REVERSE_PROXY_STACK_NAME). - Start the stack.
- Observe Ofelia logs:
WARNING: config.ini file does not exist, Aborting...
Why this is a problem
- The secure reverse-proxy configuration expects certificate management to be active (ACME challenge locations, etc.).
- Without the Ofelia job, certificates will not be renewed, causing eventual TLS expiry and outages.
Proposed Fix (one or more of the following)
-
Bundle & mount a default
config.ini:- Provide a
config.ini(derived fromconfig.example.ini) either:- As a file in the image at a known path (e.g.,
/etc/ofelia/config.ini), or - As a Swarm config/secret mounted into the Ofelia container at runtime.
- As a file in the image at a known path (e.g.,
- Ensure the Ofelia entrypoint passes
-config=/etc/ofelia/config.ini(or the correct path).
- Provide a
-
Template
config.inifrom env on startup:- Add a tiny entrypoint script that renders
config.inifrom environment variables (usingenvsubstor similar) if missing. - This keeps deployments simple and avoids manual config management.
- Add a tiny entrypoint script that renders
-
Clarify README & deployment docs:
- Make it explicit that the Docker-label mode is not used here and that
config.iniis required. - Provide a minimal Swarm stack example showing the mount of
config.iniand the required volumes:renew-certbot-confvolume (shared with reverse-proxy Nginx as needed)/var/run/docker.sock:rofor renewal job if necessary
- Make it explicit that the Docker-label mode is not used here and that
-
Optionally consider moving to label-based jobs if desired, but that would require re-architecting away from
config.iniand contradicts current README guidance.
Temporary Workaround
Create and mount a config.ini for Ofelia with the renew-certs job:
# /etc/ofelia/config.ini
[job-run "renew-certs"]
schedule = @every 1440h
image = jembi/swarm-nginx-renewal:v1.0.0
volume = renew-certbot-conf:/instant
volume = /var/run/docker.sock:/var/run/docker.sock:ro
environment = RENEWAL_EMAIL=${RENEWAL_EMAIL}
environment = STAGING=${STAGING}
environment = DOMAIN_NAME=${DOMAIN_NAME}
environment = SUBDOMAINS=${SUBDOMAINS}
environment = REVERSE_PROXY_STACK_NAME=${REVERSE_PROXY_STACK_NAME}
delete = trueMake sure the Ofelia container starts with:
ofelia -config=/etc/ofelia/config.ini…and that the renew-certbot-conf volume and other required volumes are available to both the renewal job and the reverse-proxy service.
Additional Context
- Reverse-proxy started with:
Running package in SECURE mode - Ofelia image:
mcuadros/ofelia:v0.3.6 - The environment variables are present, but without
config.inithey aren’t consumed by any job.
Labels
bug area: reverse-proxy-nginx component: job-scheduler-ofelia priority: high (due to potential cert expiry)