Added docker build and quick-start #101

mgaffigan · 2025-05-27T03:38:56Z

While this might be somewhat responsive to #40, it is primarily just to provide a quick-start way for a developer to build and test. While trying to address Issue 5 I had hours of challenges getting things building happily - and shifted to docker as an escape hatch.

I do not expect this PR is ready to merge as is, but wanted to get something posted publicly. Known issues include:

Not responsive to the acceptance criteria listed in Create OIE Docker Image #40
Uses a personal container registry namespace, rather than https://hub.docker.com/u/openintegrationengine
I'm not confident that I've implemented vmoptions handling correctly
Missing examples for docker-compose and playwithdocker

That said, it does work to the point of passing tests, running server, client, and simple message processing.

I'm new to the project, so if I've missed a step please let me know.

tonygermano · 2025-05-31T07:55:11Z

I updated #40 with how I think the "official" docker container should be implemented. However, I really like the idea of what is presented here to use docker as a developer tool for building and testing.

I've also responded to some of your questions here #2 (comment)

mgaffigan · 2025-06-11T05:10:44Z

@tonygermano, I'm no longer aware of issues outside of the registry name (presumably someone owns the official URL?).

Differences I'm aware of relative to the original repo:

Uses multi-stage build to allow forks and keep dev, test, and prod on the same build pipeline and artifacts
Still unclear on jar signing story
Omitted tests since they do not seem to run even in the original repo on any modern version of python and need to be reworked. I'm sure there is some version of python, dependencies, and docker that it works - but I've not found it.
Publish file is only pushing Temurin JDK on Ubuntu as ${tag}-jdk, and Temurin JRE on Alpine as ${tag} (e.g.: latest-jdk, latest-jre)

Happy to discuss as needed. Every compose file, kubernetes deployment, etc... that I've tested doesn't notice the difference between this and stock.

tonygermano · 2025-06-14T17:09:00Z

@mgaffigan I pulled your launcher script out of this PR, made significant changes, and put it in its own PR in #119 . I think it is useful outside of a docker context, and it can still be used with docker. Would you mind reviewing it and giving any feedback?

mgaffigan · 2025-06-14T17:48:39Z

@tonygermano, Excellent! I'd love to see that merged and agree it is useful outside of docker.

jonbartels · 2025-06-26T16:55:45Z

I think this overlaps with #126 but does not entirely conflict.

This PR creates a builder image and does a good job with doco and pulling in examples. #126 is a release/runtime image. They serve different purposes.

.gitattributes

.gitignore

DEVELOPERS.md

jonbartels · 2025-06-26T16:59:22Z

Dockerfile

+COPY .sdkmanrc .
+RUN apt-get update\
+    && apt-get install -y zip curl\
+    && curl -s "https://get.sdkman.io?ci=true" | bash \


Why do we need SDKMan and not just an ubuntu jdk image?

I think either is acceptable, and avoiding sdkman would certainly make the dockerfile simpler. I think I went this route because the zulu jdk image does not contain ant, so I had to pull additional packages regardless.

I'll see what that would look like.

See mgaffigan@123bbe2 for changes to avoid sdkman. Problems are:

Zulu nor Temurin seem to publish a JDK with FX included. Install from deb or tgz requires quite a few deps.

Ant requires separate installation

Docker revisions can get out of sync with .sdkmanrc

I don't love sdkman, but it does seem simpler to use it in the builder container.

I agree with the opinion that neither of the two options are perfect. Using a jdk image can easily drift with our preferred version from .sdkmanrc, and installing sdkman in a container is just nasty.

But in this case installing sdkman in the container is the better way to go to ensure as identical as possible envs.

Zulu nor Temurin seem to publish a JDK with FX included. Install from deb or tgz requires quite a few deps.

I'm not sure this is true. the 8.0.442.fx-zulu specified in .sdkmanrc is one example of that. Unless you're talking about Docker images.

I'm referring to docker images. Azul and Temurin publish tgz of JavaFX, but they do not provide pre-built images. See mgaffigan@123bbe2 for the changes to install from tgz rather than sdkman.

Dockerfile

docker/mirth-connect.sh

connectservice/.dockerignore

connectservice/Readme.md

mgaffigan · 2025-06-26T17:08:39Z

@jonbartels,

Thanks for the review. I think this PR got out of date and looks like it ended up with an accidental merge from #114 at some point. Let me clean it up to minimums. I'll pull the docker-compose examples out as well, assuming they fit better with #126.

…scripts) Signed-off-by: Mitch Gaffigan <mitch.gaffigan@comcast.net>

Signed-off-by: Mitch Gaffigan <mitch.gaffigan@comcast.net>

mgaffigan · 2025-06-26T18:46:02Z

I've updated the PR to depend upon #119 for oieserver.sh and match the conventions of #126 assuming that has more community input.

Currently blocked by merge of: #119 and #126

Process used for testing:

Check-out this branch
Merge Add oieserver launcher script #119, #40 Add initial Dockerfile #126
Edit docker/entrypoint.sh to remove cp oieserver_base.vmoptions oieserver.vmoptions
docker build -t oie-dev .
docker run --rm -p 8443:8443 oie-dev

kpalang · 2025-06-26T21:11:47Z

I've updated the PR to depend upon #119 for oieserver.sh and match the conventions of #126 assuming that has more community input.

Currently blocked by merge of: #119 and #126

Process used for testing:
1. Check-out this branch

2. Merge [Add oieserver launcher script #119](https://github.com/OpenIntegrationEngine/engine/pull/119), [#40 Add initial Dockerfile #126](https://github.com/OpenIntegrationEngine/engine/pull/126)

3. Edit `docker/entrypoint.sh` to remove `cp oieserver_base.vmoptions oieserver.vmoptions`

4. `docker build -t oie-dev .`

5. `docker run --rm -p 8443:8443 oie-dev`

Don't link to #126, it is still a draft and not ready for review.

kpalang · 2025-06-26T14:32:59Z

.dockerignore

suggestion: I think having an "ignore everything" and "unignore this" approach would be more readable here

* !setup/

Since we are building from source, I think an exclude list would be substantially more manageable than an include list.

When writing the previous comment, I was thinking of the release container.
But, if we're talking about a build container for development, I still think includelisting the 6 directories + possibly .sdkmanrc might be the cleaner way

Can you help me understand what you are intending? The primary goal of a .dockerignore is to avoid copying build artifacts, not to select things for the docker build. It should generally match .gitignore (since if you are not putting it in code-control, it probably should not be used for build).

The way I see it is that we have two options to use containers in the development cycle.

Compile OIE natively (ant build) from the host machine and package the build result into a container for runtime testing

Compile OIE inside a container that already packages required dependencies (and continue building runtime images from this)

With both approaches the build context should be as small as possible.
With approach 1 the context should only contain the setup/ directory and possibly some individual files. This is what I intended in the first comment.
With approach 2 things get a little more complicated, as we might need most of the dirs + files in the repo par some license or instructions files. But it is quite easy to find us with a bunch of little files that tweak some tooling or another, but are not necessary to compile OIE, which is why in my mind whitelisting only the files + dirs required for compilation might be the simpler way to go.

README.md

connectservice/Dockerfile

docker/docker-compose.yml

Dockerfile

kpalang · 2025-06-26T21:41:46Z

Dockerfile

@@ -0,0 +1,98 @@
+# syntax=docker/dockerfile:1.7-labs


suggestion: I think we should stay on docker/dockerfile:1

docker/dockerfile:1 doesn't support COPY --exclude=... and would thus result in increased image sizes.

question: Could the --exclude directives be run as rm-s in the builder stage?

You cannot run an rm in the same layer as a COPY, so the image size would still be inflated. I think the alternative here would be to add an option to mirth-server.xml to not copy them to setup.

The other options I'm disagreeing with are:

Copy each item from setup by name (verbose, fragile if new item added)

Copy to a third location (symlink?), then copy to final (ew for complexity)

Take the hit on layer-size (48 MB)

ADD with special tar (ew for time/space and general disgust)

I generally agree with the advice to stay on docker/dockerfile:1, but --exclude is fairly well supported by this point, and should get taken out of labs soon.

Out of curiosity since I don't currently have the means to test this, but when doing a multi-stage build with a COPY --from, will the layers from the first stage be included in the second stage or is there any squashing action going on?

DEVELOPERS.md

Signed-off-by: Mitch Gaffigan <mitch.gaffigan@comcast.net>

kpalang · 2025-07-01T17:08:47Z

Dockerfile

+COPY .sdkmanrc .
+RUN apt-get update\
+    && apt-get install -y zip curl\
+    && curl -s "https://get.sdkman.io?ci=true" | bash \


I agree with the opinion that neither of the two options are perfect. Using a jdk image can easily drift with our preferred version from .sdkmanrc, and installing sdkman in a container is just nasty.

But in this case installing sdkman in the container is the better way to go to ensure as identical as possible envs.

Zulu nor Temurin seem to publish a JDK with FX included. Install from deb or tgz requires quite a few deps.

I'm not sure this is true. the 8.0.442.fx-zulu specified in .sdkmanrc is one example of that. Unless you're talking about Docker images.

kpalang · 2025-07-01T17:13:38Z

Dockerfile

+##########################################
+#
+#     Ubuntu JDK Image
+#
+##########################################
+
+FROM eclipse-temurin:21.0.7_6-jdk-noble as jdk-run
+
+RUN groupadd engine \
+    && usermod -l engine ubuntu \
+    && adduser engine engine \
+    && mkdir -p /opt/engine/appdata \
+    && chown -R engine:engine /opt/engine
+
+WORKDIR /opt/engine
+COPY --chown=engine:engine --from=builder \
+    --exclude=cli-lib \
+    --exclude=mirth-cli-launcher.jar \
+    --exclude=mccommand \
+    --exclude=manager-lib \
+    --exclude=mirth-manager-launcher.jar \
+    --exclude=mcmanager \
+    /app/server/setup ./
+
+VOLUME /opt/engine/appdata
+VOLUME /opt/engine/custom-extensions
+EXPOSE 8443
+
+USER engine
+ENTRYPOINT ["./configure-from-env.sh"]
+CMD ["./oieserver"]
+
+##########################################
+#
+#     Alpine JRE Image
+#
+##########################################
+
+FROM eclipse-temurin:21.0.7_6-jre-alpine as jre-run
+
+# Alpine does not include bash by default, so we install it
+RUN apk add --no-cache bash
+# useradd and groupadd are not available in Alpine
+RUN addgroup -S engine \
+    && adduser -S -g engine engine \
+    && mkdir -p /opt/engine/appdata \
+    && chown -R engine:engine /opt/engine
+
+WORKDIR /opt/engine
+COPY --chown=engine:engine --from=builder \
+    --exclude=cli-lib \
+    --exclude=mirth-cli-launcher.jar \
+    --exclude=mccommand \
+    --exclude=manager-lib \
+    --exclude=mirth-manager-launcher.jar \
+    --exclude=mcmanager \
+    /app/server/setup ./
+
+VOLUME /opt/engine/appdata
+VOLUME /opt/engine/custom-extensions
+
+EXPOSE 8443
+
+USER engine
+ENTRYPOINT ["./configure-from-env.sh"]
+CMD ["./oieserver"]


question: These images have been moved to engine-docker and will be built with OpenIntegrationEngine/engine-docker#1. Would it be reasonable to edit the Dockerfile in this repo to be a build/local container only?

Can you clarify what you are suggesting be removed? I think it is useful to be able to do a local build of ubuntu or alpine, since those are the release environments and would thus both be needed for testing.

kpalang · 2025-07-01T17:15:28Z

server/basedir-includes/configure-from-env.sh

issue: @OpenIntegrationEngine/maintainers correct me if I'm wrong, but this file might live in engine-docker repo instead. I don't have a good suggestion on how to sync the two.

kpalang · 2025-07-01T17:23:59Z

Dockerfile

+COPY .sdkmanrc .
+RUN apt-get update\
+    && apt-get install -y zip curl\
+    && curl -s "https://get.sdkman.io?ci=true" | bash \


nitpick: Downloading and executing arbitrary scripts is not good practice. The correct way should be to download the binaries from GH and set them up ourselves, but it quite big hassle and I've never completed it, so might be good enough for now.

I'm not sure I see the difference between using sdkman and installing sdkman - since they both download and run untrusted scripts.

If we were aiming for zero-trust, then we would need to shift to installing from tgz and add hash verifications. sdkman is fundamentally opposed to that. Practically, I would recommend separately maintaining and publishing a javafx container then basing this file on that.

In my mind the difference is that we have no control over the script being downloaded from https://get.sdkman.io. It carries the risks of DNS poisoning (although highly unlikely), and malicious changes by sdkman maintainers themselves - also highly unlikely given their size. But then again, xz was even larger.

What I'm saying is this is something we should keep in mind. I don't have a viable alternative right now.

kpalang · 2025-07-01T17:26:34Z

Dockerfile

@@ -0,0 +1,98 @@
+# syntax=docker/dockerfile:1.7-labs


question: Could the --exclude directives be run as rm-s in the builder stage?

mgaffigan mentioned this pull request May 27, 2025

Create server launcher to replace mcserver #2

Open

3 tasks

mgaffigan mentioned this pull request Jun 11, 2025

Missing Documentation: Steps to Build and Run the Application from Source #118

Open

jonbartels requested review from kpalang and jonbartels June 26, 2025 14:04

jonbartels requested changes Jun 26, 2025

View reviewed changes

mgaffigan added 2 commits June 26, 2025 12:43

Added docker build and quick-start (without entrypoint and oieserver …

1491a35

…scripts) Signed-off-by: Mitch Gaffigan <mitch.gaffigan@comcast.net>

Updated MCAL language to addresss self-signed certs

6b7246f

Signed-off-by: Mitch Gaffigan <mitch.gaffigan@comcast.net>

mgaffigan force-pushed the feature/add-docker branch from a3709ea to 6b7246f Compare June 26, 2025 17:51

Match release dockerfile patterns in dev dockerfile

70e8e87

Signed-off-by: Mitch Gaffigan <mitch.gaffigan@comcast.net>

kpalang requested changes Jun 26, 2025

View reviewed changes

Add former entrypoint.sh script, fix name in developers.md

25d21a6

Signed-off-by: Mitch Gaffigan <mitch.gaffigan@comcast.net>

mgaffigan mentioned this pull request Jun 27, 2025

Sample statistics and notifications server #114

Open

kpalang self-requested a review July 1, 2025 17:00

kpalang reviewed Jul 1, 2025

View reviewed changes

Added docker build and quick-start #101

Are you sure you want to change the base?

Added docker build and quick-start #101

Uh oh!

Conversation

mgaffigan commented May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tonygermano commented May 31, 2025

Uh oh!

mgaffigan commented Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tonygermano commented Jun 14, 2025

Uh oh!

mgaffigan commented Jun 14, 2025

Uh oh!

jonbartels commented Jun 26, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mgaffigan commented Jun 26, 2025

Uh oh!

mgaffigan commented Jun 26, 2025

Uh oh!

kpalang commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kpalang Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mgaffigan commented May 27, 2025 •

edited

Loading

mgaffigan commented Jun 11, 2025 •

edited

Loading

kpalang commented Jun 26, 2025 •

edited

Loading

kpalang Jul 7, 2025 •

edited

Loading