Build environment and configuration

[LevT2]

Hello Andrey

Have you ever considered kind of a "dual" task?
I mean building the "strongly typed" local development environment with Infrastructure-as-Code approach.
It's correctness for a class of projects may be guaranteed by the build sources.

The practical need is that the build systems are normally working well in managed environments.
But individual developers have mutable environments in totally unknown state, for example with missing or misconfigured runtimes or build systems themselves.

[snowleopard]

@LevT2 Hi there!

The notion of a dual task is intriguing but I'm afraid I don't understand what exactly you suggest.

In our current model, tasks are pure functions: they simply combine the results of calling the fetch callback several times. The notion of environment suggests that the tasks will somehow be influenced by it. Will they simply get it as an additional env input? For example:

type Task env c k v = forall f. c f => env -> (k -> f v) -> f v

If yes, I don't see in what sense this is "dual", so I guess, you meant something else.

[LevT2]

@snowleopard Hi again,

I apologize for my poetic expression, it wasn't about your Tasks in any sense
I mean that your Build may happen to be only one side of the coin, and I suggest considering the other side.

The 'strongly typed' dev environment should drastically simplify dev tools, their confgurations and perhaps the build systems too

[snowleopard]

@LevT2 No problem at all, thanks for clarifying!

Can you elaborate on what aspects of build environment/configuration you think should be included in our models? It's always helpful to start with a couple of examples, to keep things simple but concrete.

[LevT2]

@snowleopard
Unfortunately, I am too rookie to help you elaborating your model, and my insight is totally driven by intuition.
What I see is a complimentary Infrastructure-as-Code model.

The both should meet in a kind of 'statically typed' dev environment, i.e. that is valid for a class of projects by construction

I consider building full-fledged IaC s producing local dev environments ready for project build, execution and debugging, leveraging complex composite runtimes such as Docker

[snowleopard]

OK, I think I see what you mean. Let's keep this issue open to collect ideas and examples.

[LevT2]

Hello @snowleopard

I would try to introduce some vocabulary that may or may not help.

On some Russian forum more than a decade ago, I have formulated the essence of software installation:
it is a mean to do software deployment in the form of adaptation == configuration == binding to the preexisting environment

Let's have a SUD (system under deployment), in this case it is a complex runtime environment as large as Operating System.

When we do our installation we adapt or bind our SUD to some concrete external environment.
In case of the OS installation it is the concrete hardware plus the externally (in BIOS/UEFI) preconfigured 'boot path'.

Let's look at following two examples, making no difference from this point of view:

1. regular OS installation (started from a CD/DVD static image with generic kernel and no optimizations)
2. installation of a prepared executable self-contained OS image file into some existing boot menu
   (I had taken my inspiration from the Windows Native boot feature.)

The implementation details differ:

• in the first case we are basically selecting the applicable kernel modules and configuring them,
  beginning with the boot critical hardware and software drivers.

• In the second case we are configuring the preliminary bootloaders chain responsible for the final start of the OS bootloader
  accordind to the same 'boot critical path'.

With both ways of OS installation we have to satisfy the single strong environmentally imposed requirement.
Any other selections and optimizations made with the first way of OS installation are optional, belonging to further configurations and not to the OS installation itself.

This landmark boot critical path itself may vary
1) from the OS point of view it may include some non-traditional storage protocol drivers such as iSCSI
2) from the boot environment point of view it may consist of complicated boot chain such ad Clover-> grub2 -> bootmgr, iPXE etc.
[There are some marginal projects on the Internet tricking Windows to boot from the in-memory RW boot device created by grub4dos, by means of a custom boot critical driver. This 'tricked' OS should be considered installed properly, because there is no indiscriminating way to deny its suitability as a runtime environment for all applications]

Our next steps are to configure the OS (as a kind of [T] runtime) recursively up to next strongly defined boundaries, each one of them guarantee successful composition of more specific runtimes

[snowleopard]

@LevT2 Thanks for sharing more thoughts on this topic! I don't think I will be able to do any form of formalisation or modelling of this domain any time soon, simply because most of the notions you talk about are new or unfamiliar to me. To formalise this domain, we need a domain expert, and I'm clearly not one. Perhaps, one of my future work projects will bring me into this domain, in which case I'll be happy to revisit these ideas.

[LevT2]

Hello @snowleopard
I have just found an attempt to introduce a specially crafted language

https://github.com/hashicorp/hcl/tree/hcl2
https://www.packer.io/docs/from-1.5
https://www.hashicorp.com/resources/intermediate-hcl-configuration-languages-in-hcl2/

It is already implemented in Hashicorp Packer, but still in beta.
Looks like its author is highly motivated in this domain and has some strong practical expertise, as it is not a his first attempt.

What if he would become interested in your theoretical support?

[snowleopard]

Speaking of configuration languages, there is also Dhall: https://dhall-lang.org/. Dhall seems cool, but I never had a chance to try it.

What if he would become interested in your theoretical support?

Well, as I said in the previous comment, I am in no position to provide any theoretical support on this topic, since I am not an expert in configuration languages. I'd be very interested in reading a paper exploring this topic from a formal point of view, so if you come across any, please share them too!