Skip to content

Break down rodan-python3-celery:nightly image in backend-django (previously rodan-main) #17

@notkaramel

Description

@notkaramel

Let's break down this line into a builder image

FROM ddmal/rodan-python3-celery:nightly


I'm not sure why this is the case. I'm assuming that Rodan resuses the same base everywhere but it's very confusing to know which one is an actual service and which one is just a base image.

I've found the reason! In settings.py, we have this part below. This explains why Rodan's rodan-main and celery (and probably python3-celery and gpu-celery in the past) all use the exact same base image. The environment variable CELERY_JOB_QUEUE is specified in docker-compose.yml file

# Job Packages
RODAN_JOB_QUEUE = os.getenv("CELERY_JOB_QUEUE")
RODAN_JOB_PACKAGES = []

BASE_JOB_PACKAGES = [base_jobs for base_jobs in allJobs['BASE_JOB_PACKAGES']]
RODAN_PYTHON3_JOBS = [py3_jobs for py3_jobs in allJobs['RODAN_PYTHON3_JOBS']]
RODAN_GPU_JOBS = [gpu_jobs for gpu_jobs in allJobs['RODAN_GPU_JOBS']]

if RODAN_JOB_QUEUE == "None" or RODAN_JOB_QUEUE == "celery":
    # All the jobs must be registered in the database, so all jobs must be here.
    RODAN_JOB_PACKAGES += BASE_JOB_PACKAGES
    RODAN_JOB_PACKAGES += RODAN_PYTHON3_JOBS
    RODAN_JOB_PACKAGES += RODAN_GPU_JOBS
elif RODAN_JOB_QUEUE == "Python3":
    RODAN_JOB_PACKAGES += BASE_JOB_PACKAGES
    RODAN_JOB_PACKAGES += RODAN_PYTHON3_JOBS
elif RODAN_JOB_QUEUE == "GPU":
    RODAN_JOB_PACKAGES += BASE_JOB_PACKAGES
    RODAN_JOB_PACKAGES += RODAN_GPU_JOBS
elif RODAN_JOB_QUEUE == "Docs":
    RODAN_JOB_PACKAGES = []
else:
    raise EnvironmentError(
        "An environment was not built for that specific rodan job-queue yet.\n"
        "\tBuild one and try again.\n"
        "\t{}\n".format(RODAN_JOB_QUEUE)
    )

Originally posted by @notkaramel in #11

Metadata

Metadata

Assignees

Labels

priorityAntoine is working on this at the moment!

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions