Some questions...

[orbisvicis]

1. The front-page claims support for variant builds. Does this mean using python language features (ie, create & call different functions), or is there special mem-specific library support?
2. Does mem provide configuration helpers, such as: system has package A, package A supports feature C, etc. For example, pkg-config.
3. Support for macros - i.e. #define - and writing macros to configuration files - i.e. config.h.
4. Installing to outside the project directory.

If any of these are true, perhaps the example could be update to show off these more in-depth features.

[srp]

1| You can use any python language features you want with mem

2| At the moment mem isn't smart enough to use pkg-config type stuff or anything very operating system or installation specific. That could all be written, but at the moment it doesn't have any of that.

3| There isn't macro support, but writing a config.h file would be trivial. Mem works by memoizing python function calls that are annotated, so you can basically write any python method you want. In this case you would just write a python function that would write the file, and then annotate it with @mem.memoize so that it only gets ran when something changes. To see some examples looks at some of the existing tasks in mem/tasks/.

4| The core part of mem doesn't care where destination files go. That is to say that it tracks them, but it isn't picky about where they go. The existing tasks might, I don't remember.

By the way, mem is kind of bit rotting right now. I haven't done anything with it in quite a few years. It worked really well for building C and swig/python code for a (proprietary) project I worked on, but that project is no more. Right now it's using older python, it should get updated to use more recent python releases. The other big deficiency it has is that it only works single threaded for the most part. There's kind of a hack to allow it to build multiple c files from a single directory, but doesn't get anywhere near the parallelism that something like ninja or even gnu make get. I had some ideas on how to fix that using Futures, but would require a rewrite and I never found time to do that.

[orbisvicis]

Well right now parallelism is something like:

mem.tasks.gcc.obj(["a.c", "b.c", "c.c"])

obj(sources):
    # start the threads for "a.c", "b.c", "c.c"
    for source in sources:
        threads.append(start_thread(source))
    # wait on the threads
    for thread in threads:
        thread.join()
    # return the results:
    return [thread.result for thread in threads]

I don't understand how any sort of automatic parallelism is possible. The function call graph is dynamically evaluated only at runtime, so parallelism has to be implemented at the language level within fbuildroot.py (i.e., not by mem but by the fbuildroot.py author). For example, how would you implement parallelism given the corresponding code and call graph:

def build():
    f_a(...)

@mem.memoize
def f_a(...):
    dep_1 = f_a_a(...)
    dep_2 = f_a_b(...)
    dep_3 = f_a_c(...)
    return f_e(dep_1, dep_2, dep_3)

@mem.memoize
def f_a_a(...):
    dep_1 = f_a_a_a(...)
    return process(dep_1)

@mem.memoize
def f_a_a_a(...):
    # leaf node
    dep_1 = mem.add_dep(...)
    return process(dep_1)

@mem.memoize
def f_a_b(...):
    dep_1 = f_a_b_a(...)
    dep_2 = f_a_b_b(...)
    return process(dep_1, dep_2)

@mem.memoize
def f_a_b_a(...):
    # leaf node
    dep_1 = mem.add_dep(...)
    # recursion, to make things complicated
    if condition(dep_1):
        f_a(...)
    return process(dep_1)

@mem.memoize
def f_a_b_b(...):
    # leaf node
    dep_1 = mem.add_dep(...)
    return process(dep_1)

@mem.memoize
def f_a_c(...)
    dep_1 = f_a_c_a(...)
    dep_2 = f_a_c_b(...)
    dep_3 = f_a_c_c(...)
    dep_4 = f_a_c_d(...)
    return process(dep_1, dep_2, dep_3, dep_4)

@mem.memoize
def f_a_c_a(...):
    # leaf node
    dep_1 = mem.add_dep(...)
    return process(dep_1)

@mem.memoize
def f_a_c_b(...):
    # leaf node
    dep_1 = mem.add_dep(...)
    return process(dep_1)

@mem.memoize
def f_a_c_c(...):
    # leaf node
    dep_1 = mem.add_dep(...)
    return process(dep_1)

@mem.memoize
def f_a_c_d(...):
    # leaf node
    dep_1 = mem.add_dep(...)
    return process(dep_1)

                          build
                            |
                            |
.------------------------> f_a
|         __________________|________________________________
|        /                  |                                \
|     f_a_a               f_a_b                              f_a_c
|       |                 /  \                       _______/     \____________
|       |                /    \                     /    /               \     \
|    f_a_a_a        f_a_b_a    f_a_b_b        f_a_c_a    f_a_c_b    f_a_c_c    f_a_c_d
|                      |
|______________________|

I'm curious how you intended to use Futures, perhaps you could jot down some quick thoughts.