This document is an introduction to the build-tool waf. Waf is a
tool written in Python for setting up automated build systems for a
programming projects.
It is assumed that the reader is interested in using waf for setting
up the build for a project. The instructions given were recorded on a
Linux system and might not be suitable for Windows users.
The build and distribution problem is one of the hardest problems in
applied computer science. You have to make sure that your software
works in a totally unknown environment radically different from the
one in which it was developed.
Additionally if the build fails, the user of the software must be
informed in a graceful way with a description of why.
As previously described, building software is hard. There are many
build systems out there, but only waf and autotools provides a
holistic system that takes care of every step from configure to
distribution. waf has several advantages over autotools that makes it
preferable:
- waf uses Python while autotools mixes shell scripting, M4 and
Perl.
- waf does not generate intermediate files that confuses users
like autotools does.
- waf is many times faster than autotools.
The first step in using Waf is to install it. I recommend checking it
out from Subversion instead of downloading a tarball as the former
usually is much more up to date and contains the latest bugfixes. As
of this writing, the repository is at revision 4813. Create a checkout
using:
$ svn checkout http://waf.googlecode.com/svn/trunk/ waf-read-only
To build waf, enter the waf-read-only directory generate the waf
script:
$ cd waf-read-only
$ ./waf-light --make-waf
This creates an exectuable Python script called waf. This script
should be copied to the root directory of any project you want to use
waf for. It is also possible to install waf globally, although that is
not recommended. The README file has more details.
Each build tool has its own kind of configuration files. make has
Makefiles, autoconf has configure.in-files, scons has SConstruct
files and so on. waf has wscripts. So the first step is to create an
initial bare-bone wscript file. Copy this to a file named wscript in
your project root directory:
srcdir = '.'
blddir = 'build'
def set_options(opt):
pass
def configure(conf):
pass
def build(bld):
pass
Note that this a regular Python program and that you can enter
whatever Python code you want in it.
srcdir = '.'
The srcdir variable informs waf of where the source files are in
relation to the wscript. Its value will almost always be '.' which is
the current directory.
blddir = 'build'
This variable tells waf where to put the built files. By default, waf
uses an out-of-tree build process, in contrast to most autotooled
projects which are built in-tree, which means that waf will not litter
your source directories.
def set_options(opt):
pass
The set_options function is used for modifying the global options
object passed in as the opt parameter. Here you can add extra
options both for configuring and building.
def configure(conf):
pass
This function performs the exact same task as the configure script in
autotooled projects -- it checks that all the requirements for the
software is fullfilled.
def build(bld):
pass
build defines how the software is to be built. Target rules are
specified here.
Provided that the waf program is in the same directory, it is now
possible to run this empty build process:
$ ./waf configure
Configuration finished successfully (00:00:00); project is now ready to build.
$ ./waf build
Compilation finished successfully (00:00:00)
We haven't told waf what to do yet, so it outputs nothing
interesting. waf creatures a directory called build/ which contains
the result of the build process:
$ find build/
build/
build/config.log
build/c4che
build/c4che/build.config.py
build/c4che/default.cache.py
build/default
build/.wafpickle-6
Each built file is placed in build/default, the other files are just
house-keeping for waf. Truth be told, this isn't all that interesting,
but it gets better in the next section where we make waf actually do
something useful.
Lets talk about the imagined project we are using waf for for a
while. We are creating a very simple shared library written in C
complete with documentation, tests and example programs.
Our project contains the following files and directories:
$ find
.
./wscript
./waf
./src
./src/hello.c
./src/hello.h
./src/wscript_build
./tests/test.c
./tests/demo.c
./tests/wscript_build
- build/
- As mentioned before, this is where waf puts the built files.
- docs/
- Project documentation.
- src/
- C sources and header files for the shared library.
- tests/
- Tests and example programs.
The hello.c and hello.h files are the C sources. First hello.c:
#include <stdio.h>
#include "hello.h"
void
say_hi ()
{
printf ("Ho\n");
}
And hello.h:
#ifndef HELLO_H
#define HELLO_H
void say_hi();
#endif
The other files will be introduced as needed in later sections.
Since this a C project, we must add configure checks to waf to ensure
that the user has a C compiler installed before the project can be
built. Lets begin with that:
def set_options(opt):
opt.tool_options('compiler_cc')
def configure(conf):
conf.check_tool('compiler_cc')
These lines adds C compiler options and configure checks. Rerunning
./waf configure outputs the following:
$ ./waf configure
Checking for program gcc : ok /usr/bin/gcc
Checking for compiler version : ok 4.2.4
Checking for program cpp : ok /usr/bin/cpp
Checking for program ar : ok /usr/bin/ar
Checking for program ranlib : ok /usr/bin/ranlib
Checking for gcc : ok
Configuration finished successfully (00:00:00); project is now ready to build.
This proves that waf is now verifying that the user has a C compiler
installed. Next we add target definitions for the source files in the
src directory. Create a file called wscript_build to the
src directory.
obj = bld.new_task_gen('cc', 'shlib')
obj.source = ['hello.c']
obj.target = 'hello'
These three lines work similar to a build rule in a makefile. The file
hello.c is used as input to build the shared library libhello.so. Note
that waf automatically adds the prefix "lib" and the suffix ".so"
because we are using Linux. To add this target to the build process
change the build function in the wscript file:
def build(bld):
bld.add_subdirs('src')
This instructs waf to add the rules from any wscript or wscript_build
file located in src. The code serves the same purpose as the SUBDIRS
variable in Makefile.am variables. Naturally, it is also possible to
put all build information in the main wscript file but that might get
messy because all paths gets longer. On the other hand, fewer files
might feel less cluttered.
Let's test the build:
$ ./waf build
[1/2] cc: src/hello.c -> build/default/src/hello_1.o
[2/2] cc_link: build/default/src/hello_1.o -> build/default/src/libhello.so
Compilation finished successfully (00:00:00)
Excellent! waf first executed the compilation creating hello_1.o
and then linked it to produce libhello.so. Both stored in
build/default/src/:
$ ls build/default/src/
hello_1.o libhello.so
A preprocessor define instructs the compiler to replace all subsequent
occurences of a macro with specified replacement tokens. The canonical
way to pass compile time options is to let the build system create a
"config.h" header file that all C source files includes.
Let's say we wan't to add a function to our hello library that returns
its version. First we add the necessary code to the wscript file to
create the config.h file with the necessary define:
VERSION = '1.0.0'
srcdir = '.'
blddir = 'build'
def set_options(opt):
opt.tool_options('compiler_cc')
def configure(conf):
conf.check_tool('compiler_cc')
conf.define('PACKAGE_VERSION', VERSION)
conf.write_config_header('config.h')
def build(bld):
bld.add_subdirs('src')
The two new lines in the configure function creates the macro
PACKAGE_VERSION and writes it to the config.h file. Configuring
the project creates the config.h file in the build/default/
directory:
/* Configuration header created by Waf - do not edit */
#ifndef _CONFIG_H_WAF
#define _CONFIG_H_WAF
#define PACKAGE_VERSION "1.0.0"
#endif /* _CONFIG_H_WAF */
By including config.h in hello.c, the macro can be used in the new
function:
#include <stdio.h>
#include "hello.h"
#include "config.h"
void
say_hi ()
{
printf ("Ho\n");
}
const char *
get_version ()
{
return PACKAGE_VERSION;
}
The function is also added to the header:
#ifndef HELLO_H
#define HELLO_H
void say_hi();
const char *get_version ()
#endif
It is also possible to pass the defines as compiler arguments using
gcc's -D option (or the equivalent for other compilers). Each target
object in waf has a "defines" attribute where the defines are listed.
So to accomplish what we did in the previous chapter, without using a
config.h file, change src/wscript_build to the following:
obj = bld.new_task_gen('cc', 'shlib')
obj.source = ['hello.c']
obj.target = 'hello'
obj.defines = 'PACKAGE_VERSION=\\"%s\\"' % VERSION
Note that an extra level of string escaping is needed to get the
string literal correctly passed to the compiler from the shell.
Which ever way you prefer to use is a matter of taste. But when the
list of defines grow large, the config.h method certainly becomes more
pleasant. Even if it sometimes can increase recompile times.
Integrating the documentation building step into the build tool is
usually tricky. End users usually does not need to rebuild the
documentation and most documentation tools such as Javadoc and
Doxygen does not integrate very easily into a target-based system.
Never the less, the popular documentation extraction tool GTK-Doc
for GTK+ libraries contains a bunch of autotools macros so that it
can be built using make. waf can do that too, ofcourse.
First, we have to write some documentation for our two functions in
hello.c. GTK-Doc uses a pretty self-explanatory doc comment syntax
reminiscent of Javadoc. We also add a third function, just for
fun. The resulting hello.c and hello.c files are:
#include <stdio.h>
#include "hello.h"
/**
* say_hi:
*
* Prints "Ho\n" on standard out.
**/
void
say_hi ()
{
printf ("Ho\n");
}
/**
* get_version ()
* @returns: a string containing the version of this library.
*
* Returns a string on the format "major.minor.micro" describing which
* is this librarys version.
**/
const char *
get_version ()
{
return PACKAGE_VERSION;
}
/**
* get_day_name:
* @day_index: index of a week-day, in the range 0-6.
* @sunday_first: one if a calendar with the week starting on Sunday
* is desired, zero otherwise.
* @returns: the name of the day, or %NULL if @day_index is out of
* range.
*
* Returns the name of the day corresponding to the specified day
* index.
**/
const char *
get_day_name (int day_index,
int sunday_first)
{
const char *days[] = {"Monday",
"Tuesday",
"Wednesday",
"Thursday",
"Friday",
"Saturday",
"Sunday"};
if (day_index < 0 || day_index > 6)
return NULL;
if (sunday_first)
day_index = (day_index + 6) % 7;
return days[day_index];
}
and hello.h:
void say_hi ();
const char *get_version ();
const char *get_day_name (int day_index,
int sunday_first);
#endif
Autotooled projects employing GTK-Doc all have a --enable-gtk-doc
configure option that determines whether the build process will build
the documentation or not. It is easily added to waf by modifying the
set_options function in the wscript file:
def set_options(opt):
opt.tool_options('compiler_cc')
opt.add_option('--enable-gtk-doc',
action = 'store_true',
default = False,
help = 'use gtk-doc to build documentation ' +
'[default: %default]')
Waf uses the standard library optparse module for parsing the
command line, so any option format accepted by optparse will work for
waf. See the documentation for that module for more information.
We can check that the extra option has become available:
$ ./waf configure --help
usage: waf [options] [commands ...]
* Main commands: configure build install clean dist distclean uninstall distcheck
* Example: ./waf build -j4
options:
--version show program's version number and exit
-h, --help show this help message and exit
-j JOBS, --jobs=JOBS amount of parallel jobs [Default: 2]
-f, --force force file installation
-k, --keep keep running happily on independent task groups
-p, --progress -p: progress bar; -pp: ide output
-v, --verbose verbosity level -v -vv or -vvv [Default: 0]
--destdir=DESTDIR installation root [Default: '']
--nocache compile everything, even if WAFCACHE is set
-b BLDDIR, --blddir=BLDDIR
build dir for the project (configuration)
-s SRCDIR, --srcdir=SRCDIR
src dir for the project (configuration)
--prefix=PREFIX installation prefix (configuration only) [Default: '/usr/local/']
--zones=ZONES debugging zones (task_gen, deps, tasks, etc)
--targets=COMPILE_TARGETS
compile the targets given only [targets in CSV format, e.g. "target1,target2"]
-d DEBUG_LEVEL, --debug-level=DEBUG_LEVEL
Specify the debug level, does nothing if CFLAGS is set in the environment. [Allowed
Values: 'ultradebug', 'debug', 'release', 'optimized', 'custom']
--enable-gtk-doc use gtk-doc to build documentation [default: False]
^<- here it is!
C Compiler Options:
--check-c-compiler=CHECK_C_COMPILER
On this platform (linux) the following C-Compiler will be checked by default: "gcc
suncc"
