Introduction

This document is an introduction to the build-tool waf. Waf is a tool written in Python for setting up automated build systems for a programming projects.

Intended Audience and Prerequisite Knowledge

It is assumed that the reader is interested in using waf for setting up the build for a project. The instructions given were recorded on a Linux system and might not be suitable for Windows users.

Why is a Build System Needed?

The build and distribution problem is one of the hardest problems in applied computer science. You have to make sure that your software works in a totally unknown environment radically different from the one in which it was developed.

Additionally if the build fails, the user of the software must be informed in a graceful way with a description of why.

Why Waf?

As previously described, building software is hard. There are many build systems out there, but only waf and autotools provides a holistic system that takes care of every step from configure to distribution. waf has several advantages over autotools that makes it preferable:

waf uses Python while autotools mixes shell scripting, M4 and Perl.

waf does not generate intermediate files that confuses users like autotools does.

waf is many times faster than autotools.

Installing Waf

The first step in using Waf is to install it. I recommend checking it out from Subversion instead of downloading a tarball as the former usually is much more up to date and contains the latest bugfixes. As of this writing, the repository is at revision 4813. Create a checkout using:

$ svn checkout http://waf.googlecode.com/svn/trunk/ waf-read-only

To build waf, enter the waf-read-only directory generate the waf script:

$ cd waf-read-only
$ ./waf-light --make-waf

This creates an exectuable Python script called waf. This script should be copied to the root directory of any project you want to use waf for. It is also possible to install waf globally, although that is not recommended. The README file has more details.

The First wscript

Each build tool has its own kind of configuration files. make has Makefiles, autoconf has configure.in-files, scons has SConstruct files and so on. waf has wscripts. So the first step is to create an initial bare-bone wscript file. Copy this to a file named wscript in your project root directory:

srcdir = '.' blddir = 'build' def set_options(opt): pass def configure(conf): pass def build(bld): pass

Note that this a regular Python program and that you can enter whatever Python code you want in it.

Explanation

srcdir = '.'

The srcdir variable informs waf of where the source files are in relation to the wscript. Its value will almost always be '.' which is the current directory.

blddir = 'build'

This variable tells waf where to put the built files. By default, waf uses an out-of-tree build process, in contrast to most autotooled projects which are built in-tree, which means that waf will not litter your source directories.

def set_options(opt): pass

The set_options function is used for modifying the global options object passed in as the opt parameter. Here you can add extra options both for configuring and building.

def configure(conf): pass

This function performs the exact same task as the configure script in autotooled projects -- it checks that all the requirements for the software is fullfilled.

def build(bld): pass

build defines how the software is to be built. Target rules are specified here.

Running the Build

Provided that the waf program is in the same directory, it is now possible to run this empty build process:

$ ./waf configure
Configuration finished successfully (00:00:00); project is now ready to build.
$ ./waf build
Compilation finished successfully (00:00:00)

We haven't told waf what to do yet, so it outputs nothing interesting. waf creatures a directory called build/ which contains the result of the build process:

$ find build/
build/
build/config.log
build/c4che
build/c4che/build.config.py
build/c4che/default.cache.py
build/default
build/.wafpickle-6

Each built file is placed in build/default, the other files are just house-keeping for waf. Truth be told, this isn't all that interesting, but it gets better in the next section where we make waf actually do something useful.

wafing a Project

Lets talk about the imagined project we are using waf for for a while. We are creating a very simple shared library written in C complete with documentation, tests and example programs.

File Structure

Our project contains the following files and directories:

$ find
.
./wscript
./waf
./src
./src/hello.c
./src/hello.h
./src/wscript_build
./tests/test.c
./tests/demo.c
./tests/wscript_build

build/: As mentioned before, this is where waf puts the built files.
docs/: Project documentation.
src/: C sources and header files for the shared library.
tests/: Tests and example programs.

The hello.c and hello.h files are the C sources. First hello.c:

#include <stdio.h> #include "hello.h" void say_hi () { printf ("Ho\n"); }

And hello.h:

#ifndef HELLO_H #define HELLO_H void say_hi(); #endif

The other files will be introduced as needed in later sections.

Source Targets

Since this a C project, we must add configure checks to waf to ensure that the user has a C compiler installed before the project can be built. Lets begin with that:

def set_options(opt): opt.tool_options('compiler_cc') def configure(conf): conf.check_tool('compiler_cc')

These lines adds C compiler options and configure checks. Rerunning ./waf configure outputs the following:

$ ./waf configure
Checking for program gcc                 : ok /usr/bin/gcc
Checking for compiler version            : ok 4.2.4
Checking for program cpp                 : ok /usr/bin/cpp
Checking for program ar                  : ok /usr/bin/ar
Checking for program ranlib              : ok /usr/bin/ranlib
Checking for gcc                         : ok
Configuration finished successfully (00:00:00); project is now ready to build.

This proves that waf is now verifying that the user has a C compiler installed. Next we add target definitions for the source files in the src directory. Create a file called wscript_build to the src directory.

obj = bld.new_task_gen('cc', 'shlib') obj.source = ['hello.c'] obj.target = 'hello'

These three lines work similar to a build rule in a makefile. The file hello.c is used as input to build the shared library libhello.so. Note that waf automatically adds the prefix "lib" and the suffix ".so" because we are using Linux. To add this target to the build process change the build function in the wscript file:

def build(bld): bld.add_subdirs('src')

This instructs waf to add the rules from any wscript or wscript_build file located in src. The code serves the same purpose as the SUBDIRS variable in Makefile.am variables. Naturally, it is also possible to put all build information in the main wscript file but that might get messy because all paths gets longer. On the other hand, fewer files might feel less cluttered.

Let's test the build:

$ ./waf build
[1/2] cc: src/hello.c -> build/default/src/hello_1.o
[2/2] cc_link: build/default/src/hello_1.o -> build/default/src/libhello.so
Compilation finished successfully (00:00:00)

Excellent! waf first executed the compilation creating hello_1.o and then linked it to produce libhello.so. Both stored in build/default/src/:

$ ls build/default/src/
hello_1.o  libhello.so

Preprocessor Defines

A preprocessor define instructs the compiler to replace all subsequent occurences of a macro with specified replacement tokens. The canonical way to pass compile time options is to let the build system create a "config.h" header file that all C source files includes.

Using config.h

Let's say we wan't to add a function to our hello library that returns its version. First we add the necessary code to the wscript file to create the config.h file with the necessary define:

VERSION = '1.0.0' srcdir = '.' blddir = 'build' def set_options(opt): opt.tool_options('compiler_cc') def configure(conf): conf.check_tool('compiler_cc') conf.define('PACKAGE_VERSION', VERSION) conf.write_config_header('config.h') def build(bld): bld.add_subdirs('src')

The two new lines in the configure function creates the macro PACKAGE_VERSION and writes it to the config.h file. Configuring the project creates the config.h file in the build/default/ directory:

/* Configuration header created by Waf - do not edit */ #ifndef _CONFIG_H_WAF #define _CONFIG_H_WAF #define PACKAGE_VERSION "1.0.0" #endif /* _CONFIG_H_WAF */

By including config.h in hello.c, the macro can be used in the new function:

#include <stdio.h> #include "hello.h" #include "config.h" void say_hi () { printf ("Ho\n"); } const char * get_version () { return PACKAGE_VERSION; }

The function is also added to the header:

#ifndef HELLO_H #define HELLO_H void say_hi(); const char *get_version () #endif

The defines attribute

It is also possible to pass the defines as compiler arguments using gcc's -D option (or the equivalent for other compilers). Each target object in waf has a "defines" attribute where the defines are listed.

So to accomplish what we did in the previous chapter, without using a config.h file, change src/wscript_build to the following:

obj = bld.new_task_gen('cc', 'shlib') obj.source = ['hello.c'] obj.target = 'hello' obj.defines = 'PACKAGE_VERSION=\\"%s\\"' % VERSION

Note that an extra level of string escaping is needed to get the string literal correctly passed to the compiler from the shell.

Which ever way you prefer to use is a matter of taste. But when the list of defines grow large, the config.h method certainly becomes more pleasant. Even if it sometimes can increase recompile times.

Building GTK-Doc Documentation

Integrating the documentation building step into the build tool is usually tricky. End users usually does not need to rebuild the documentation and most documentation tools such as Javadoc and Doxygen does not integrate very easily into a target-based system.

Never the less, the popular documentation extraction tool GTK-Doc for GTK+ libraries contains a bunch of autotools macros so that it can be built using make. waf can do that too, ofcourse.

First, we have to write some documentation for our two functions in hello.c. GTK-Doc uses a pretty self-explanatory doc comment syntax reminiscent of Javadoc. We also add a third function, just for fun. The resulting hello.c and hello.c files are:

#include <stdio.h> #include "hello.h" /** * say_hi: * * Prints "Ho\n" on standard out. **/ void say_hi () { printf ("Ho\n"); } /** * get_version () * @returns: a string containing the version of this library. * * Returns a string on the format "major.minor.micro" describing which * is this librarys version. **/ const char * get_version () { return PACKAGE_VERSION; } /** * get_day_name: * @day_index: index of a week-day, in the range 0-6. * @sunday_first: one if a calendar with the week starting on Sunday * is desired, zero otherwise. * @returns: the name of the day, or %NULL if @day_index is out of * range. * * Returns the name of the day corresponding to the specified day * index. **/ const char * get_day_name (int day_index, int sunday_first) { const char *days[] = {"Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"}; if (day_index < 0 || day_index > 6) return NULL; if (sunday_first) day_index = (day_index + 6) % 7; return days[day_index]; }

and hello.h:

void say_hi (); const char *get_version (); const char *get_day_name (int day_index, int sunday_first); #endif

Adding Documentation Option

Autotooled projects employing GTK-Doc all have a --enable-gtk-doc configure option that determines whether the build process will build the documentation or not. It is easily added to waf by modifying the set_options function in the wscript file:

def set_options(opt): opt.tool_options('compiler_cc') opt.add_option('--enable-gtk-doc', action = 'store_true', default = False, help = 'use gtk-doc to build documentation ' + '[default: %default]')

Waf uses the standard library optparse module for parsing the command line, so any option format accepted by optparse will work for waf. See the documentation for that module for more information.

We can check that the extra option has become available:

$ ./waf configure --help
usage: waf [options] [commands ...]

* Main commands: configure build install clean dist distclean uninstall distcheck
* Example: ./waf build -j4

options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -j JOBS, --jobs=JOBS  amount of parallel jobs [Default: 2]
  -f, --force           force file installation
  -k, --keep            keep running happily on independent task groups
  -p, --progress        -p: progress bar; -pp: ide output
  -v, --verbose         verbosity level -v -vv or -vvv [Default: 0]
  --destdir=DESTDIR     installation root [Default: '']
  --nocache             compile everything, even if WAFCACHE is set
  -b BLDDIR, --blddir=BLDDIR
                        build dir for the project (configuration)
  -s SRCDIR, --srcdir=SRCDIR
                        src dir for the project (configuration)
  --prefix=PREFIX       installation prefix (configuration only) [Default: '/usr/local/']
  --zones=ZONES         debugging zones (task_gen, deps, tasks, etc)
  --targets=COMPILE_TARGETS
                        compile the targets given only [targets in CSV format, e.g. "target1,target2"]
  -d DEBUG_LEVEL, --debug-level=DEBUG_LEVEL
                        Specify the debug level, does nothing if CFLAGS is set in the environment. [Allowed
                        Values: 'ultradebug', 'debug', 'release', 'optimized', 'custom']
  --enable-gtk-doc      use gtk-doc to build documentation [default: False]
  ^<- here it is!

  C Compiler Options:
    --check-c-compiler=CHECK_C_COMPILER
                        On this platform (linux) the following C-Compiler will be checked by default: "gcc
                        suncc"

Bjourne's blog

Jumpstarting Waf