It’s that time of the month. That time where some random person on the Internet announces a new, fancy build system that’s better than all the other build systems and will take over the world.
On a more serious note, I just finished version 0.3.1 of
ExMake, a build tool similar in nature to
make tool, but generally more modern. This version is more or less
usable for building actual things.
So what is this thing, why does it exist, and how is it better than all the other build systems?
As mentioned, ExMake is a
make-like tool. It’s not something meant to replace
monolithic build system suites like Autotools. It’s simply a dependency-driven
build tool that executes recipes to produce output files. The main reason I
created it was that I was sick and tired of
make’s scripting ‘language’ which
is, at best, a text processor with macros. Other things that bothered me were
make’s lack of support for libraries, its broken recursion model, its lack of
caching of any kind, its fragile and inefficient parallel job scheduler, and
its nightmare-inducing, shell-based recipes.
ExMake is written in Elixir, a general-purpose programming language, and also uses it as its scripting language. This does mean that ExMake requires Erlang to work. This is unlikely to be a problem, however, as Erlang is available for the vast majority of operating systems and architectures.
Let me introduce ExMake by example. A simple
Exmakefile might look like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
myprog, you just invoke
exmake which will invoke the
since no other target has been named. As with
make, you can pass
-j 4 to
build up to 4 things in parallel (though it obviously won’t matter in this
case, since we’re just building one rule). ExMake does one thing very
make, however: It produces no console output at all by
default. This may seem odd, but this is in line with the Unix philosophy of
shutting up unless you have something interesting to say. ExMake will of course
produce output if an error occurs. You can ask ExMake to be loud (print all
shell invocations) by passing
You’re probably wondering about the explicit
./ prefixes added to that
shell invocation. This happens because ExMake only invokes recipes relative to
the directory it was started in. This may seem weird if you’re coming from good
make, but there’s a very good reason that ExMake does this: Its recursion
model. If you’ve ever maintained a non-trivial application using
make as a
build tool, you’ve probably dealt with
make’s insane approach to recursion.
There’s even a famous paper explaining why it’s evil and broken. ExMake does
away with this traditional approach to recursion and instead treats recursion as
a first-class citizen.
Recursion in ExMake is done with a simple directive that tells ExMake where to
go and pick up another
Exmakefile. Let’s see an example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
There’s a bit to take in here. Let me explain:
recursedirective tells ExMake to go into
utilsand pick up the
Exmakefilethat’s in there. ExMake then adds all of the rules in that file to the dependency graph.
- The top-level
cleantarget now depends on
utils/clean. This notation may seem odd, but it’s just that ExMake uses path separators for phony targets too. This dependency ensures that a top-level
exmake cleanwill also clean up in
myprognow depends on
utils/stuff.o, which is built in the
utils. This means that you could
cd utils && exmakeand then
cd .. && exmaketo build
utils/stuff.oseparately if you wanted.
- The argument list (or more accurately, argument pattern list) for
myprognow contains two identifiers in the sources list. This just means that we’re expecting the two dependencies (
utils/stuff.o) that we declared, and therefore pattern match on that list to conveniently get them.
We can now build:
1 2 3
Notice how ExMake didn’t
utils. But because we wrote our recipes to
use the paths passed to them instead of writing the paths literally, everything
worked out as ExMake simply passed the full paths to the recipes.
Now, why is this way of dealing with recursion a good idea? The reason is quite
simple: Since ExMake does not invoke itself recursively – for performance
reasons – changing the current directory would be dangerous. Sure, it would
work fine when executing a build script serially, but if you add
-j 2 to the
mix, suddenly some rules will start executing in the wrong directories! One way
to work around that would be to add locks around directory changes, but then
the performance gained by parallelizing the build is undermined.
So, it’s a tradeoff. You have to write rules more carefully, but on the other
hand, ExMake doesn’t have to jump through ridiculous hoops such as invoking
itself recursively and using IPC to communicate between the processes. This
means that ExMake’s parallel job scheduler is much more robust than that of,
One performance-related thing that ExMake puts a lot of effort into is caching.
To see just how much this matters, let’s try timing non-cached and cached
builds of the recursion example above. We do this by passing the
This will print a bunch of stuff after the build is done, but we’re only
interested in the total time.
1 2 3 4 5 6 7 8 9 10 11 12
After this build, everything has been cached. Let’s try a cached build:
1 2 3 4 5 6 7 8 9 10 11 12 13
So we went from 578 milliseconds to 211 milliseconds. This doesn’t seem like a whole lot given the small test case, but on large projects with huge dependency graphs, just loading the graph from disk is significantly faster than computing all of it on every ExMake invocation. Similarly, loading the compiled build scripts is way faster than lexing, parsing, analyzing, and emitting them over and over.
I hate how, when I use
make, I have to reinvent so much stuff to invoke the
tools I need. There’s no standard way to have libraries that can be installed
and used in makefiles. In ExMake, there are standard mechanisms to construct
and use libraries.
Let’s suppose I want to build a simple C# console application. If I use the C#
library that ships with ExMake, my
Exmakefile will look like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Let’s run it:
1 2 3 4
And that’s it. We can add source files to the sources list as we go along. The
C# library picked up a C# compiler automatically, set up all the arguments as
needed, and created the
myprog.exe rule and its recipe (with all its somewhat
complicated internal logic to handle a lot of different use cases).
Note that information such as the C# compiler to use is cached.
Of course, it doesn’t end with just the standard libraries. You can write your own libraries that can be installed globally (or locally) and included in build scripts.
For example, a super simple C library (
c.ex) might look like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38
You can use ExMake to build the library with an
Exmakefile like so:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Once compiled to
Elixir.ExMake.Lib.C.beam, it can be copied to the system so
load_lib directive will be able to load it. For example, you could
drop it into
ExMake does away with some features of GNU
make that I personally consider
either broken, insane, or actively harmful. Or any combination of those.
Last-resort rules and the
.DEFAULT target are unsupported. I have never come
across a sensible use case for these, and it seems to me that if you find
yourself needing these, you’ve done something horrifically wrong.
ExMake can’t create and update archive files. This is an odd feature that is
better left to explicit invocations of
.LOW_RESOLUTION_TIME target does not exist in any form. I don’t know of
any remotely relevant operating system or file system that needs this. I
suspect it made sense back when it was introduced, but highly doubt its value
.EXPORT_ALL_VARIABLES feature is unsupported as I strongly view this as
a glaring recipe for disaster - it could affect all sorts of things in the
programs being executed. It’s much better to explicitly export variables with
There is no support for
.NOTPARALLEL (or anything like it). This is another
case of hiding build system bugs intentionally. If two rules must not execute
in parallel, make one depend on the other.
.NOTPARALLEL is a shotgun solution
that works, but harms parallelism of the entire build script.
.ONESHELL makes little sense in ExMake since recipes are regular Elixir
expressions, not shell invocations.
In short, ExMake is a
make-like tool using a modern, general-purpose
programming language as its scripting language. It features extensive caching,
reusable libraries, sane recursion, and better parallel job scheduling.
If you’re interested, you can find more info on
the ExMake wiki, though I have yet to
write the manual. Still, this post and the API documentation for
ExMake.Lib should get you going easily.