Use of buildflag in general to circumvent missing shell on run-stack

[nicolasbender]

For the implementation of buildpack spec 0.9 shell processes should no longer implicitly be expected (RFC 93). During construction of build arguments, buildflag -H:+StaticExecutableWithDynamicLibC gets added which seems to be necessary for stacks without a shell.

Is there a reason why this buildflag should not be used in general?

Describe the Enhancement

Using -H:+StaticExecutableWithDynamicLibC in general would make this buildpack independent from the existence of a shell of the underlying run-stack.

Possible Solution

Remove the if-condition for a tiny-stack.

Motivation

Using stacks from noble onwards don't differentiate between different build-stacks and don't have necessarily a stack-id to rely on. This could lead to using this buildpack with a tiny-run-stack without a shell included and ending up with a failing startup at run time.

[c0d1ngm0nk3y]

@dmikusa Do you have any insights on that one?

[dmikusa]

Hmm, this will definitely be a problem for a handful of buildpacks, this one included (Tomcat, Tomee & Procfile are the others off the top of my head). This is also my main problem with RFC 96 - stack removal. The new approach provides no way for us to know what will actually be in the runtime image.

For this particular argument, -H:+StaticExecutableWithDynamicLibC, I believe it's there because of required dependency libraries. Without this argument, the binary that native image produces would require some system library that's not present in the tiny image (but is in base & full). When we pass this to native image it does a mostly static build, see here, which will then run OK on the tiny image.

The article I linked to says:

A mostly-static native executable is a binary that links all the shared libraries on which the native executable relies (zlib, JDK-shared static libraries) except the standard C library, libc. This is an alternative option to statically linking everything. Also, depending on the user’s code, it may link libstdc+ and libgcc. This approach is useful for deployment on a distroless container image.

so it seems like there are a couple of libraries it might need without this option.

---

I think this will be a hard problem to solve in general. I don't know of a way to know what's in the runtime image at build time. In theory, we should be able to know that if pack/lifecycle told us. The pack/lifecycle tools would know what the user specified or what the default is from the builder, but they would also have to take into account any buildpack extensions that run and might change the runtime image. It would need to know that prior to the build running and then pass that information into the stage of the builds where buildpacks run.

If there were a way to do this, we could know the runtime image at build time, which would put us in the same position that we are in now. We're looking at the image name to guess what's installed in it, which is OK, but it would be nice to know more about the runtime image to make better decisions. That's a larger conversation though.

---

The only option to work around this that comes to mind would be to push the decision back on the user. So do not set this flag at all, and then leave it up to the user to set the flag when it's required. That's a poor experience, though.

I think the implication above on the issue is that we should just always set this flag, and I'm not really sure what the side effects of doing that would be. Minimally, it seems like it would produce a larger native image since it's statically compiling a bunch of stuff into it. I don't know if there are limits to what native image can static compile into a binary, though, or if there are other drawbacks.

I also know that recent native image versions can do a fully static compilation and include glibc, too. That is more complicated, though, as it requires libmusl. If we can make that easy, then maybe we just always do a fully static build? That would run on anything. Again, I'm not sure what the trade-offs would be with doing this. Seems like there are always some trade-offs 🤔 .

@anthonydahanne or @sdeleuze - What do you think? Do you know of any drawbacks to switching the buildpack to always doing a mostly-static build or a fully static build?

[anthonydahanne]

What do you think? Do you know of any drawbacks to switching the buildpack to always doing a mostly-static build or a fully static build?

OK, so reading at that thread and linked docs:

• On the paper, I love the idea of "build static never worry about runtime base image"; it does sound like bigger images, I'd need to test it out, but anyway people stressed about this could configure their runtime image to use a minimal base image without a shell (nor libs!) to save space.
• That said, mostly-static seems to work with everything BUT musl-c based runtime image (Alpine comes to mind, and personally, I would not bother that much if Alpine based image stopped working) ; it could bother advanced users using Alpine though...

[dmikusa]

One thing that we do with this buildpack that we don't do with other buildpacks is have some arguments that are not overridable by users. We could switch and do things more like other buildpacks. So we would set an initial default list of arguments and then users can override that, but would need to provide a full set of arguments to override. Or possibly we can do that plus having an "append" option for users to append to the default set.

At any rate, this argument could be in the default list so it applies most of the time, but any advanced user that doesn't want this could supply their own set of args and override those defaults.

We'd probably need to trigger a new major version for this as it would change the behavior of things.

Does that sound reasonable to everyone?