Functor units and functor packs

[lpw25]

From the summary:

OCaml's module system has great facilities for structuring your code
in the large. It supports encapsulation, parameterisation and even
recursion. Unfortunately not all of these facilities are available
across multiple files. The -pack feature allows you to group
multiple files together under an interface, but there isn't really
a way to parameterise a group of files all at once, or to allow
recursion amongst a group of files.

This proposal aims to support parameterisation of groups of files
and recursion amongst groups of files by extending functors and
recursive modules to work on groups of files via the -pack
mechanism.

Unlike a "typical" RFC this work has already been largely implemented by @OCamlPro-Couderc. Much of this text has been recycled from an earlier project plan -- so its possible some parts of the design have been adjusted slightly since then.

[lpw25]

Duplicate of #3 due to that PR getting screwed up by the change from private to public.

[Drup]

I feel like we are getting closer and closer to emulate SML's compilation unit scripts, but using a multiplicity of compiler flags and some sugar coating through dune. Maybe we should finally get the compiler to accept build scripts as input. :)

However, that's independent of the current proposal, which I think goes in the right direction.

[lpw25]

I've split out the recursive packs part into its own RFC: #20. This RFC now only concerns functor units and functor packs.

[dra27]

Does the prototype implementation do anything with ocamldep?

[gasche]

(Personal summary of a dev-meeting to discuss some RFCs.) Apparently there is some practical experience with a prototype of this within Jane Street. Getting some description of the use-cases and how that fared in practice would be a nice addition to the RFC.

[dra27]

We also noted that one of the big users of this would be ocamlopt itself, which could then be refactored to have all the backends available in one single binary.

[xavierleroy]

We also noted that one of the big users of this would be ocamlopt itself, which could then be refactored to have all the backends available in one single binary.

... provided the differences in interface between the various $TARGET/arch.ml files is not a problem.

For the record: I don't think this (having all the backends in one single binary) is required or even particularly useful to support cross-compilation.

[dra27]

For the record: I don't think this (having all the backends in one single binary) is required or even particularly useful to support cross-compilation.

Required, no; useful, absolutely. You need to be compile for both the target architecture (to build the standard library, etc.) and also the host architecture (to build ocamlopt.opt, etc.). So if ocamlopt supports all its backends, once you have assembled the bytecode version of ocamlopt, you can use that single binary to produce the target standard library, and the host standard library + .opt tools. Otherwise you have to build ocamlopt twice.

[gasche]

While not directly related to cross-compilation, having several backends (and all, why not?) linked in would also be useful for people willing to read the assembly output of OCaml programs on another architecture, which I can easily see use-cases for:

• They wish to understand the low-level representation of a specific OCaml program, but they don't know the instruction set of their machine well, they know some other architecture better (our regretted mips, RISC-V would come to mind as popular choices). (I regularly wish I could do this.)
• They are adapting all emitters to a change in the backend, and want to look at the output on test programs easily.

[EduardoRFS]

@dra27 I think you will need to build two times ocamlopt anyway as a lot of ocaml users have a dependency on the compiler-libs, and from my experience cross compiling things, everything useful seems to depends on compiler-libs, but even then, to get that this PR isn't required.

But it would definitely be useful to have everything in the same binary, for cross compilation it would only simplify the build script tho

[lpw25]

As we've been working on rolling this out within Jane Street we've made a number of changes to the design. Our current iteration uses neither functors nor packs to achieve the same aims -- although the underlying implementation is mostly the same as the original patch. There is a pretty large design space here, so I think it would be worth waiting until we've had some time to experiment with using the features in practice at Jane Street. We can restart the RFC process after that. So I'm going to close this RFC for now.