KJam Documentation

Contents

Introduction

KJam is a program construction tool, like make. KJam recursively builds target files from source files, using dependency information and updating actions expressed in a user-provided jamfile, which is written in KJam's own interpreted language.

KJam is modelled after Jam/MR, a build tool developed by Chris Seiwald of Perforce Software. It operates in a way very similar to Jam and its interpreted language is also very similar. KJam was developed in an effort to make significant improvements to Jam while still maintaining its powerful minimalist design. KJam eliminates a number of Jam's legacy features, and concentrates on features designed to make it more useful and more manageable for larger and more complex projects.

KJam is designed to be:
Fast KJam is the fastest build system available today. It is the fastest for both full and incremental builds. It has the fastest dependency scanner of any current build system. Even for projects with many thousands of files, KJam will finish scanning and start building almost instantly. Unlike most current build systems, KJam is designed from the beginning to be multi-threaded, resulting in maximum utilization of multi-processor workstations. Through the use of batched building, a built-in shell, and parallel command execution KJam reduces build times radically. Jam was well known as one of the fastest build systems available. Our performance tests show that KJam is faster.
Scalable KJam is designed to support even the largest projects with hundreds of thousands of files. It may even be run as a distributed peer network of build servers allowing projects to share the build load over a large heterogeneous network of build machines. KJam's dependency scanner uses advanced caching techniques which make it especially efficient when dealing with very large numbers of targets. It has per target viewpathing.
Unintrusive and Clean KJam is small. It has negligible CPU overhead, uses very little memory, and it doesn't create or leave behind temporary files.
Automatic Dependencies KJam computes dependencies for its targets during building. There is no need for a separate off-line make depend step.
A built-in sh-like Shell By default, on any platform, KJam runs its command scripts using its own built-in sh-like shell. This way developers can expect a common set of minimal shell features on every platform. Using a built-in shell allows KJam to report errors in command scripts which reference the errant command directly in the jamfile, something an external shell cannot do. It eliminates another point of difference between multiple platforms making it easier to write actions which run on any platform. The internal shell is also much faster. If necessary users may still specify an external shell for command scripts which require more features.
Batched Building Some compilers, such as MSVC, support building multiple object files from multiple source files with a single invocation of the compiler. This can result in dramatically reduced build times. Unlike most build software, KJam has complete support for building targets in batches.
Customizable Developers can enhance and extend KJam by creating user defined rules to utilize other built-in directives.
Language KJam includes flow-control statements, variables, and a few other features of general purpose languages.
Multi-Platform Platform independent rules and platform specific actions can be defined separately from dependency rules.
Highly portable KJam currently runs on Windows, Linux and Apple Os X. It is designed to be easily portable.
Improved Variable Expansion KJam's variable expansion is much more powerful than Jam's. In addition to having many more modifiers, KJam's variable expansion can be controlled with full support for regular expressions. In general KJam uses regular expression syntax wherever a matching pattern is required. In Jam there are a number of different restricted expression syntaxes used in various places.
Simplified Syntax With the addition of more powerful variable expansion it has been possible to remove a number of language features and maintain equivalent expressive power. The KJam language is simpler. KJam allows named arguments in both rules and actions making the code more readable. In addition the KJam parser is more flexible and more forgiving. Braces are not required around blocks with just one statement.
Better Reporting For large and complex build systems it is important to be able to diagnose problems easily. KJam has extensive and very detailed ways of reporting its internal execution, allowing developers to zero in on problems with their build environment quickly. In multi-threaded mode KJam keeps the output from different processes together instead of interspersing it. KJam can reformat all of its output, and the output of any tools it runs to the width of your shell for improved readability.

Using this Manual

This manual is organized into five major sections:

Each section is intended for a different purpose and for a different audience. Much information is repeated.

If you are already familiar with Jam and are investigating KJam, first skip ahead to the comparison with Jam section to see the differences.

If you are new to KJam and want to get started quickly building c and c++ projects using the default rules built into KJam, then start by reading the Quick Start Guide, and then read the Jambase Reference. It should be possible to get KJam to build a fairly straightforward c or c++ project in just a few minutes.

If you intend to replace or extend the built-in Jambase files, and write your own build rules, then start by reading the Tutorial. It explains the basics of how the KJam language works.

The reference sections are designed for those who are already familiar with KJam and are writing jamfiles to get specific information about certain features. The main KJam Reference section covers just the KJam program itself and the KJam language. The Jambase Reference section documents the jambase files that come built-in to KJam. This is useful only if you are using those files. If you intend to replace the built-in Jambase files, then this section can be ignored. In general the reference sections are not a good place to start learning how to use KJam.

Licensing

KJam is currently available in two forms, KJam Preview and KJam Beta.

KJam Preview may be used freely for the development of non-commercial software. It may only be used by commercial software projects for the purpose of evaluation for a period of 60 days. KJam Preview may be freely redistributed so long as it is not modified and the package comes complete with the original documentation, and none of the files have been modified.

KJam Beta is available only as part of our Beta program. It can be used only for evaluation of KJam for suitability in your software project. KJam Beta is licensed only to specific Beta Program members and may not be redistributed to anyone.

Since KJam is still under active development, a determination of how KJam will eventually be distributed when it is finished has not been made, and for this reason all rights not explicitly granted above are reserved.

The original Jam/MR is free, open source software and can be incorporated into commercial products without licensing restrictions. It is copyright Chris Seiwald.

KJam is a complete from the ground up re-implementation of a new build system which operates in a manner similar to the original Jam/MR. It does not share any code with Jam/MR. KJam is copyright Rafael Baptista.

KJam is currently pre-release software and may still have many bugs. The author specifically disclaims all responsibility for any losses due to the use or misuse of this software. This software is provided without any warranty whatsoever.

Contact

To report bugs, request features or otherwise talk to the authors about KJam, send e-mail to kjam@oroboro.com

Quick Start Guide

The Basics

KJam comes configured with a standard set of build rules for c and c++ projects. For many simple projects it is possible to get started using KJam in just a few minutes by writing some very simple, yet surprisingly powerful Jamfiles.

Suppose you have a directory called src full of .cpp source files that you want to build into an application called myprog. You would create a file called jamfile with the following contents:

StaticExecutable myprog : src ;

To build myprog, run KJam in the same directory as the jamfile ( by typing kjam ). KJam will create an output directory, bin for all the generated files. It will search the directory src for source files. It will process any .l and .y files that it finds though flex and bison. It will scan all source files for #include statements and generate dependencies. It will compile all the .c and .cpp files it found or created. And finally it will link all the resulting object files into a binary.

Even though KJam does a full dependency analysis of your sources before every run, it will start building even very large projects almost immediately. KJam is extremely efficient.

This same jamfile should work without modification under Windows, Linux or OsX. The generated files will be stored in a directory under bin which specifies the platform and the debugging level.

If your machine has multiple processors, KJam will figure out which build steps can be safely done simultaneously and take advantage of them. If your compiler supports batched building (like MSVC does ) KJam will build sources in batches.

KJam will detect the width of your shell, and format all the output neatly for easy readbility. And even though many build steps will happen simultaneously, KJam will a not allow output from different steps to be interspersed.

Invoking Rules

Suppose you don't want to build every source file in a directory, but would rather name your source files specifically. You can write your jamfile like this:

StaticExecutable myprog : src1.cpp src2.cpp src3.cpp ;

To build a library you use the StaticLibrary rule:

StaticLibrary mylib : src1.cpp src2.cpp src3.cpp ;

To build a .dll or a shared library use the SharedLibrary rule:

SharedLibrary mylib : src1.cpp src2.cpp src3.cpp ;

Under Windows this will create a .dll and its associated export library. Under Linux or OsX this will create a .so or a .dylib file respectively. All the details of how to do that on each platform are dealt with for you - portably. The target names do not have to deal with non-portable file extensions like .dll, .so or .dylib

To build a program that links with a library, just list the libraries at the end of the rule:

StaticLibrary lib1 : lib1/src ;
StaticLibrary lib2 : lib2/src ;
StaticExecutable myprog : myprog/src : lib1.lib lib2.lib ;

The .lib extension helps KJam to know that you want to link with a static library instead of a dynamic library. But the extensions are portable between Windows, Linux and OsX. Under Linux, KJam would link with lib1.a and lib2.a. And if you had passed in lib1.a then under Windows KJam would have automatically converted it to lib1.lib.

You can also easily link with a shared library:

SharedLibrary lib1 : lib1/src ;
StaticLibrary lib2 : lib2/src ;
StaticExecutable myprog : myprog/src : lib1.dll lib2.lib ;

Under Windows, KJam will figure out that it needs to look for an export library. Under Linux, it will look for a .so shared library. The user does not have to worry about these non-portable details. KJam will even figure out all the dependencies between these different targets automatically.

By default when you run KJam without any arguments it will try to build the target all, which will build all the targets that have been declared using the above rules. So in the case of the previous jamfile, it would build lib1, lib2 and myprog.

To build only a subset of the declared targets you can declare a Group target:

SharedLibrary lib1 : lib1/src ;
StaticLibrary lib2 : lib2/src ;
StaticExecutable myprog : myprog/src : lib1.dll lib2.lib ;
Group libs : lib1.lib lib2.lib ;

Now when you ask it to build libs, it will only build lib1 and lib2. To exclude a target from the defauilt all target, declare your own all group listing the targets you want. This will override the automatically created all target:

SharedLibrary lib1 : lib1/src ;
StaticLibrary lib2 : lib2/src ;
StaticLibrary special : lib2/src ;
StaticExecutable myprog : myprog/src : lib1.dll lib2.lib ;
Group all : lib1.lib lib2.lib myprog.exe ;

Now when you run KJam with no arguments, it will not build the library special. To get that you would have to ask for it specifically by running kjam special.

These simple jamfiles automatically create clean targets. So if you want to clear out all the generated files just run kjam clean.

Setting Variables

The operation of the built-in rules can be modified by setting variables.

By default KJam will search for header files wherever it finds source files. If you would like it to use headers found in other directories, say the headers for an external library, set the INCLUDE_DIRS variable:

INCLUDE_DIRS = ../otherlib/src ;
StaticExecutable myprog : src ;

By default KJam will look for libraries to link with in the default output directory, and in the default system library directories. To add more places to look for libraries to link with, use the LIB_DIRS variable:

INCLUDE_DIRS = ../otherlib/src ;
LIB_DIRS = ../otherlib ;
StaticExecutable myprog : src : otherlib.lib ;

KJam will automatically chose a reasonable set of build flags when running your c compiler. To add additional build flags use the CCOPTS variable. To add extra link options use the LINKOPTS variable. The new options will be added to every compile or link:

CCOPTS = /DSPECIAL ;
LINKOPTS = -lspecial ;
StaticExecutable myprog : src ;

To change the output directory set BIN_DIR. By default BIN_DIR is set to bin/$(PLATFORM)/$(CURFLAV). You can set this to anything you want, though it is highly recommended that BIN_DIR always include $(CURFLAV) or $(BUILD_CODE), to avoid problems with mixing generated files built with different debugging levels. So for example if you want to have all your generated files go to a special directory shared by all projects, instead of a local one, and you don't care about multiple platforms, you might set BIN_DIR this way:

BIN_DIR = ../shared_bin/$(CURFLAV) ;

These variables can also be set on a per target basis using the on keyword. For example to add special link options to just lib1:

LINKOPTS on lib1.a = -lspecial ;
StaticLibrary lib1 : lib1/src ;
StaticLibrary lib2 : lib2/src ;
StaticLibrary lib3 : lib3/src ;

To build targets with special compilation options, you can use the StaticObject rule. For example in the following case only special.cpp is built with the extra flags:

StaticObject special : special.cpp : /DSPECIAL ;
StaticExecutable myprog : special.obj src1.cpp src2.cpp src3.cpp ;

Notice that when you need to, you can pass compiled object files as sources.

The built-in Jambase supports building targets at different optimization levels. You can do this by setting the CURFLAV variable. Set this variable in the environment. On Linux and OsX remember to export it. There are 4 levels, release, optimized, debug and extra_debug. At each setting the output files will go to their own directory, to avoid mixing output files built with different compiler flags.

KJam can also build projects with platform specific source files. Suppose you have a project with one or more source files which are different on Windows and Linux. Simply create posix and win32 subdirectories in the directory where source files would normally be found, and put the platform specific source files in each directory. KJam will build the appropriate sources for the platform.

Managing Sub-Projects

Jamfiles can call other Jamfiles that manage individual subprojects. Suppose you have 5 directories, where each directory has its own jamfile and can build a separate library or executable. There may be dependencies between some of these sub-projects. This is easy to do:

SubProject lib1 ;
SubProject lib2 ;
SubProject lib3 ;
SubProject app1 : lib1 lib2 ;
SubProject app2 : lib1 lib3 ;

With the jamfile above you can now build all the libraries and applications by running KJam in the parent directory. You can build any individual sub-project and the projects it depends on by running KJam with the name of the sub-project as the target. KJam will build the sub-projects in the right order. You can also switch to any individual sub-project directory to build and test just that sub-project.

A clean target is also automatically created. Building clean will clean all the sub-projects.

When building a sub-project KJam will invoke a copy of itself in the subproject's directory and will build the default all target. To have KJam build other targets as well, list those as the third argument to the SubProject rule:

SubProject app1 : lib1 lib2 : all verify install ;

It is also possible to make multiple SubProject targets each of which builds a particular sub-project target:

SubProject app1 : lib1 lib2 ;                    // build the 'all' target
SubProject <verify>app1 : lib1 lib2 : verify ;   // build the 'verify' target
SubProject <install>app1 : lib1 lib2 : install ; // build the 'install' target

Now you can invoke each of those targets in the given subproject from the main build file. Here is a jamfile which builds all your sub-projects, and has an install target:

SubProject lib1 ;
SubProject lib2 ;
SubProject lib3 ;
SubProject app1 : lib1 lib2 ;
SubProject app2 : lib1 lib3 ;

SubProject <install>app1 : app1 : install ;
SubProject <install>app2 : app1 : install ;

Group all : lib1 lib2 lib3 app1 app2 ;
Group install : <install>app1 <install>app2 ;

Note that you override the automatically created all target to exclude the install sub-project targets. Also notice that the install targets depend on their respective applications being up-to-date.

Given the jamfile above, here are some commands you can run and what they would do:

kjam                      # re-build all the libraries and applications
kjam lib2                 # re-build just lib2
kjam app2                 # re-build app2 and its dependents lib1 and lib3
kjam clean                # clean all the sub-projects
kjam install              # re-build all the applications, and install them all
kjam "<install>app1"      # re-build app1, lib1 and lib2, and then install just app1

Tutorial

Introduction

KJam is a software build tool like make. KJam is designed to read in a user created jamfile, similar to the way make reads a makefile. It uses this to compute dependencies between targets. Most targets are either source files, such as c source files, or the generated files made by running system commands with the source files as input. KJam determines which targets exist in the file system, and what date they were last updated, and computes which targets need to be updated. These targets are then updated using defined update actions, consisting of shell scripts with operating system commands to update the targets.

The KJam Language

The KJam language consists of a series of statements each terminated with a semicolon. For example:

SOURCES = main.c file.c ;

There are four kinds of statements: (1) statements which set variables, (2) define a rule or action, (3) invoke a rule or action, and (4) flow of control statements. A rule is basically a KJam function. An action is an operating system command script which is called to update one or more targets. We will get to those later. First lets examine variables.

Setting Variables

Variables are set or modified using an assignment operator like this:

SOURCES = main.c file.c ;

The type of all variables in KJam is a list of strings. Variables which hold a single string are considered to be a list of strings with only one element. In the above statement, a variable called SOURCES has been created which holds a list with two elements, the string main.c and the string file.c.

In KJam, variable names and their values may include almost any character, including many characters which are also used as operators. For this reason it is necessary to put whitespace around most names. For example, in the above statement spaces are required around the operator = and before the final ;.

Had the statement above been written like this:

SOURCES=main.c file.c;

KJam would have interpreted this line as a the invocation of a rule called SOURCES=main.c being invoked with an argument file.c; and would have included the contents of the following line as further arguments to this rule invocation. The KJam parser works this way because very often it is necessary to create variables whose values include many special characters which are often found in file paths ( :;/\.* ) and in arguments to tools ( -=+ ). Not allowing these characters in values and variable names would require that many of these strings be expressed with quote marks. Maintainers of build systems are usually very engaged in managing the names of file targets, paths and command line arguments. The language is designed to make these things easy to manipulate at a minor cost to the flexibility of the language for writing statements.

KJam supports three variable assignment operators. This will overwrite the existing value of a variable:

SOURCE = main.c file.c ;         

This will append the given value at the end of the existing value of the variable:

SOURCE += main.c file.c ;               

This will create a variable with the given value if the variable does not already exist:

VALUE ?= default ;

You can use more than one variable name on the left hand side to set multiple variables at the same time. In the following case three variables are created, all of which are set to a hold a list of two strings.

VAR1 VAR2 VAR3 = value1 value2 ;

Literals

The value of a variable can be set to a literal, to the value of another variable or to a combination.

In KJam, for convenience literals don't have to be quoted, though they can be. If you have organized your jamfiles well, normally there will be one or more jamfiles full of rules and actions which you don't have to deal with very often, and the local jamfile which is full of literals containing just the names of your targets, important paths in your projects and sometimes additional command line options. Its nice when you have a file full of these literals not to have to put quotes around everything.

Every literal is interpreted as a string, even numbers. Unquoted literals may be composed of almost any character except whitespace or the " character. To create literals with these characters you must put quotes around it. For example here are several literals:

LIST = file.c 5 x=y "a b" "\"" "\n" ;

This list holds five elements. All of them are strings. The third value is the string x=y. The fourth value is the string a b. The fifth element is the value ". And the last is a string holding the newline character. Notice that the value of a quoted literal does not include the quote marks. Notice also that for literals expressed in quote marks, certain special characters can be expressed using c-like string syntax with escape codes for special characters.

KJam variable names are also strings. So it is possible to name a variable whatever you want by using quote marks. For example, this is legal:

" " = value ;

Variable Expansion

You can refer to the value of a variable by surrounding its name with $( and ). For reasons which will become clear, in KJam getting the value of a variable is called variable expansion. For example:

MSG = "This is a message" ; 
print $(MSG) ;

The first line creates a variable holding a single string value. The second line invokes a built-in rule which prints that value. If a variable holds a list with more than one element, its expansion will be a list containing all of its elements. To refer to just one element in a variable with multiple elements use the subscript operator []:

SOURCES = file1.c file2.c file3.c ;
print $(SOURCES[1]) ;

This code would print file2.c. You can also use the subscript operator to get a range of elements from a list:

SOURCES = file1.c file2.c file3.c file4.c file5.c ;
print $(SOURCES[1-3]) ;
print $(SOURCES[2-]) ;

The second line would print file2.c file3.c file4.c. The third would print file3.c file4.c file5.c.

It is possible to use variable expansion on both the left hand or right hand side of a variable definition:

$(SOURCES) = file1.c file2.c file3.c ;
SRCS = $(SOURCES) ;

In this case the value of SRCS is set to hold a list of three strings: file1.c file2.c file3.c. Using variable expansion on the left hand side works also:

VARS = var1 var2 var3 ;
$(VARS) = value1 value2 ;

This sets three variables, var1 var2 and var3 to each hold a two string list, value1 value2.

The name of the variable given in an expansion may itself be an expansion. For example:

FILENAME = file ;
TYPE = FILE ;
SRCS = $($(TYPE)NAME).c ;

The value of SRCS is now set to file.c.

Variable Expansion Products

When variable expansions are combined with literals, with other expansions, the resulting expansion is the product of the elements. For example:

DIRS = dir1/ dir2/ ;
FILES = file1 file2 file3 ;
SRCS = $(DIRS)$(FILES) ;

The value of SRCS is a list with 6 elements: dir1/file1 dir1/file2 dir1/file3 dir2/file1 dir2/file2 dir2/file3. The product may have literals in it:

DIRS = dir1 dir2 ;
FILES = file1 file2 file3 ;
SRCS = ./$(DIRS)/$(FILES).c ;

Now the value of SRCS is a list with 6 elements: ./dir1/file1.c ./dir1/file2.c ./dir1/file3.c ./dir2/file1.c ./dir2/file2.c ./dir2/file3.c. If one of the expansions is an empty list then the resulting product is also an empty list:

// assume that DIRS is not defined
FILES = file1 file2 file3 ;
SRCS = ./$(DIRS)/$(FILES).c ;

Now the value of SRCS is a list with zero elements. It is also possible to explicitly set a variable to an empty list:

SRCS = ;

This is the same thing as undefining a variable. Variables may have some elements in the variable list set to a null string:

SRCS = file1.c "" file2.c ;

In this case the value of this variable is set to three strings. The second string is an empty string, but still holds its position. When expanding the product of two variables both of which contain empty strings, some of the combinations will be empty. New empty strings are automatically removed when generating products from lists with empty strings:

X = A "" B ;
Y = a "" b ;
PROD = $(X)$(Y) ;

The value of PROD is a list with 8 elements: Aa A Ab a b Ba B Bb.

Variable expansion also happens inside quoted strings the same way as when the variables are not quoted:

FILES = file1 file2 file3 ;
SRCS = "./dir/$(FILES).c";

The value of SRCS is now a list with the following three elements: ./dir1/file1.c ./dir1/file2.c ./dir1/file3.c.

Variable Expansion Modifiers

A variable expansion can also be written in the form $(NAME:MODIFIER). When applying a modifier to an expansion, the value returned may be changed in a number of ways. Each modifier consists of at least a single letter which determines its function. Here are some examples:

SRCS = Dir1/File1.c ;
TEMP1 = $(SRCS:U) ;  // return SRC in all upper case: e.g. DIR1/FILE1.C
TEMP2 = $(SRCS:L) ;  // return SRC in all lower case: e.g. dir1/file1.c
TEMP3 = $(SRCS:D) ;  // return just the directory part of the file name e.g. Dir1
TEMP4 = $(SRCS:B) ;  // return just the base part of the file name e.g. File1
TEMP5 = $(SRCS:S) ;  // return just the suffix part of the file name e.g. .c
TEMP6 = $(SRCS:H) ;  // return the whole file name without the directory 
                     // e.g. File1.c

SRCS = /project/* ;
TEMP7 = $(SRCS:A) ;  // returns all the directories in the file system which 
                     // match this path expression.
                     // e.g. /project/src /project/include /project/bin etc.

SRCS = Dir1/*.c ;
TEMP8 = $(SRCS:M) ;  // returns all the files in the file system which match this 
                     // path expression.
                     // e.g. Dir1/File1.c Dir1/File2.c Dir1/File3.c etc.

Some modifiers can take an argument, which is the letter followed by = and a string. For example:

TEMP6 = $(SRCS:D=newdir) ;  // replace the directory name with the given string. 
                            // e.g. newdir/File1.c
TEMP7 = $(SRCS:B=newfile) ; // replace the base part of the file name with the given 
                            // string. e.g. Dir1/newfile.c
TEMP8 = $(SRCS:S=.o) ;      // replace the suffix part of the file name with the 
                            // given string. e.g. Dir1/File1.o

Multiple modifiers may be applied to a single expansion. In this case it may be required to quote any arguments so that the arguments don't run together with the following modifier:

TEMP9 = $(SRCS:D="newdir"S=".o"U);   // return NEWDIR/FILE1.O

The modifiers are applied in the order that they are written in the modifier list. Putting the above modifiers in a different order produces different results:

TEMP10 = $(SRCS:UD="newdir"S=".o");   // return newdir/FILE1.o

It is possible to select some elements from a list using modifiers with regular expressions. For example:

FILE = file1.c file2.c file3.c file1.o file2.o file3.o file1.s file2.s file3.s ;
C_FILES = $(FILES:I=".c$");                   // select just the c files.
SO_FILES = $(FILES:X=".c$");                  // exclude only the c files.
CO_FILES = $(FILES:I=".c$"I=".o");            // include both c and o files.
CO_FILES = $(FILES:I=".c$"I=".o"X="^file1");  // include both c and o files, but 
                                              // not files that start with file1.

It is also possible with regular expression modifiers to replace parts of a matched expression. In regular expression syntax, parts of an expression delimited by () are called numbered subexpressions. Subexpression 0 is the entire expression. Expressions 1-9 are the first nine encountered () subexpressions. The modifier N supplies a match subexpression, and subsequent numbered modifiers either return or replace the part of the expression matched by that subexpression. For example:

TEXT = retired fragrance mortar ;
RTEXT1 = $(TEXT:N="r([^r]*)r"1);        // return all the text between r's. 
                                        // e.g eti ag ta
RTEXT2 = $(TEXT:N="r([^r]*)r"0);        // return all the text between r's, 
                                        // including the 'rs themselves. 
                                        // e.g retir ragr rtar
RTEXT1 = $(TEXT:N="r([^r]*)r"1="__");   // replace the text between r's with __. 
                                        // e.g r__red fr__rance mor__r

All variable expansion happens during the parsing phase so it is not possible to run system commands and assign their output to a KJam variable. In KJam external programs are only ever executed by command scripts during the updating phase.

Literal Expansion

Modifiers may also be applied to literals using the literal expansion syntax. As we saw above literals may be expressed directly as unadorned strings and as quoted strings. They can also be expressed by surrounding them with @( and ). For example the following three statements are equivalent:

TARGET = target.obj ;
TARGET = "target.obj" ;
TARGET = @(target.obj) ;  // literal expansion syntax

When expressed using the third method, you can also apply expansion modifiers just like you can do with variable expansion. This is useful in cases where you find yourself assigning a literal string to a variable just so you can apply expansion modifiers to it. Here are several simple examples:

TARGET_DIR = @(target.obj:FD) ;   // sets TARGET_DIR to the directory where 
                                  // target.obj is bound to
UPPERCASE = @(mIxEdCaSe:U) ;      // sets UPPERCASE to MIXEDCASE
LIBS = @(libs/*:M) ;              // sets LIBS to all the files found in the 
                                  // libs directory

Literal expansions and variable expansions may be nested in each other. So for example to find all the files with a given root file name but different three different file extensions:

EXTS = .txt .log .out ;
LOGFILES = @(mylogfile$(EXTS):F) ;

Suppose you have a list of source file name bases with the extensions already removed, and you want to get a list of any corresponding object files if they exist. You could do something like this:

SOURCES = source1 source2 source3 ;
EXISTING_OBJECTS = @($(SOURCES).obj:M) ;

As you can see combining literal and variable expansions can be very powerful.

Rules

KJam rules are functions which are interpreted and invoked during the parsing phase. They are used to set variables, to create dependencies between targets and to associate build actions with targets. Rules are defined like this:

rule RuleName ARG1 : ARG2 : ARG3
{
    // rule statements go here, such as:
    VAR1 = $(ARG1) ;

    // flow of control statements
    if ( $(VAR1) )
    {
        // and rule invocations
        OtherRule $(ARG1) : $(ARG2) : $(ARG3) ;
    }
}

A rule definition always starts with the keyword rule, and is followed by the name of the rule. This name is used later to invoke the rule. Rules definitions are then followed by a list of argument names. Rules may be defined with zero or more arguments. The arguments become variables defined during the invocation of that rule. This is then followed by the body of the rule which can include any KJam statement except the definition of another rule or action.

The first argument to a rule is interpreted by KJam as the target of that rule. This is normally the name of the files which this rule will build. The second argument is normally the name of the files which are used as the sources for the targets. Invoking a rule will cause KJam to create a dependency between the sources and the targets. This will be used later to compute which files need updating. Arguments after the second do not have a special meaning in KJam.

To invoke a rule, a statement begins with the rule name, and is followed by the arguments. Argument lists in KJam are always separated by :, and regular lists are simply series of tokens separated by whitespace. So a rule invocation with lists would look like this:

RuleName a b : c d e : f g h ;

Here the arguments to RuleName are three lists, "a b", "c d e", and "f g h".

Flow of Control Statements

Rule bodies may contain flow of control statements. Without flow of control statements, all statements in a rule would be executed in the order they appear. With flow of control statements, you have greater control over what gets executed and in what order. The most basic flow of control statement is if:

if ( $(ARG))
{
   echo $(ARG);
}

This statement will print out the value of $(ARG) only if it is defined.

ARG = a b c ;
while( $(ARG) )
{
   echo $(ARG);
   ARG = $(ARG[2-]);
}

This statement will print out: "a b c", "b c", and "c".

Defining Actions

KJam actions are operating system command scripts. They are run during the updating phase to build targets. Actions are defined like this:

action ActionName ARG1 : ARG2
{%
    cc $(ARG2) -o$(ARG1)
%}

An action definition always starts with the keyword action, and is followed by the name of the action. This name is used inside rule bodies to invoke the action.

Following this comes an argument list. In actions the first argument is the a list of targets which will be build built by this action. The second argument is a list of precursor files required to build it. More arguments are allowed, but have no special meaning to the KJam language.

Last comes the body of the action inside {% %} braces. The body of the action is command script which is passed to an operating system shell which then will run external applications to create the target files. Normally this shell is an sh-like shell which is built-in to KJam. By setting JAMSHELL however this shell may be a native operating system shell like bash, or Windows cmd.exe. Any expansions in the body of the script will be expanded before the script is passed to the shell.

Built-in Shell

The syntax of the built-in shell should be familiar to anyone who has written shell scripts. It supports piping output between processes with | and redirecting it to files using >, >> and <. For example:

ls | sort > sorted.txt

This would get a directory listing, send it to the sort command, and save the sorted output into a file called sorted.txt. The redirection also allows the error output to be redirected:

cc $(SRC) 1> output.txt 2> error.log

In this case the program 'cc' is run, and the regular output is redirected to output.txt, and the errors are redirected to error.log. The error stream can be redirected to the output stream and visa versa:

cc $(SRC)  2>&1 > output.txt

Here both stream are piped to output.txt. The > operator will create the output file, replacing any existing file by that name. The >> operator will concatenate the output to any existing file:

echo foo > tmp.txt 
echo bar >> tmp.txt

The contents of tmp.txt would be foobar.

The input to a command chain can also be redirects by use of the < operator. In this case the input to the command grep is taken from the given file main.c:

echo foo > tmp.txt 
echo bar >> tmp.txt

The contents of tmp.txt would be foobar.

The KJam built-in shell does not support any flow control statements found in shells, such as if, case, or while. It also does not support setting environment variables, job control or any common interactive features. The supported features should be enough to write most build actions.

If a certain build action requires more sophisticated shell features then an external shell must be used. To call out to an external shell, the JAMSHELL variable should be defined to point to the external shell to use. This JAMSHELL variable could be set in global scope to replace the built-in shell everywhere, or it can be set right before the action invocation to use a different shell just for the given action.

Variable Scope

By default all variables are defined in global scope. That is once a variable is defined, its value is available everywhere, in any rule or action until the value is overwritten by another variable definition. Variables may also be declared in local scope. To declare a variable in local scope you precede the definition with the keyword local:

local SOURCES = src1.c src2.c ;

This will define the variable for all statements inside the same block of statements. That is the variable remains defined until the next closing brace } is found. Variables in local scope will mask variables in global scope. So while a local variable is in scope, a global variable with the same name is not accessible. When the local variables goes out of scope, the global variable comes back into effect.

Variables may also be defined to be matched to a particular target such that they are in effect only while that target is part of the first argument to a rule or action. This is done with the on keyword:

HDRSCAN on src.c = $(HDRPATTERN) ;

Here, the variable HDRSCAN is set to $(HDRPATTERN) only while the file src.c is a target. While building a certain target, variables defined on that target will mask local or global variables with the same name.

Explicit Dependencies

In KJam most dependencies are expressed implicity by how rules are invoked. KJam defines two built-in rules which may be used to create explicit dependencies between targets. The rule depends creates a normal dependency between sources and targets:

depends target.o : source.c ;

This creates a dependency between target.o and source.c. When source.c is updated, KJam will determine that target.o needs to be updated as well.

include source.c : header.h ;

This rule creates an include dependency between source.c and header.h. This will cause KJam to determine that if header.h was updated that is should behave as if source.c was updated as well. Usually you will want to set up a build system such that the include dependencies are created automatically. This can be done with the built-in variable HDRSCAN and HDRRULE.

When $(HDRSCAN) is set on a target, if that target is a file, KJam will scan that file for inclusion statements. The value of $(HDRSCAN) should be set to a regular expression which matches inclusion lines in the given source file. The pattern should match the entire inclusion statement. The pattern should include a single () grouping which indicates the filename being included.

Whenever a match is found by KJam using $(HDRSCAN), KJam will invoke a the rule named by the $(HDRRULE) variable. $(HDRSCAN) and $(HDRRULE) must have the same number of elements. If these variables hold more than one element, then each file will be scanned by multiple patterns, and the approriate rule will be called.

Comments

KJam interprets every line as a KJam statement, unless the line is a comment. KJam support c-style /**/ multi-line comments, c++ like // comments, and makefile like # comments:

/* this is a comment and is not interpreted by KJam */
// so is this
# and so is this.

Making a basic Jamfile

Putting together the things we have learned we can make a basic jamfile:

CC ?= cl.exe /nologo ;
LIBEXE ?= lib.exe /nologo ;

HDRPATTERN = "^[\t ]*#[\t ]*include[\t ]*[\"]([^\"]*)[\"]" ;
HDRRULE = HeaderRule ;

rule Library TARGET : SOURCES : LIBS
{
   HDRSCAN on $(SOURCES) = $(HDRPATTERN) ;
   LOCATE  on $(TARGET)  = $(TARGET_DIR) ;

   local C_SOURCES = $(SOURCES:I="\.(c|cpp)$") ;
   local OBJECTS = $(C_SOURCES:S=.obj) ;

   depends $(TARGET) : $(OBJECTS) $(LIBS) ;

   for SRC_FILE in $(C_SOURCES)
      Object $(SRC_FILE:S=.obj) : $(SRC_FILE) ;

   LinkLibrary $(TARGET) : $(OBJECTS) $(LIBS) ;
}

rule Object TARGET : SOURCE
{
   CompileObject $(TARGET) : $(SOURCE) ;
}

rule HeaderRule INCLUDER : INCLUDEDS
{
   includes $(INCLUDER) : $(INCLUDEDS) ;
   HDRSCAN on $(INCLUDEDS) = $(HDRPATTERN) ;
}

action CompileObject TARGET : SOURCE
{%
   $(CC) $(CCOPTS) /c $(SOURCE) /o $(TARGET)
%}

action LinkLibrary TARGET : SOURCES
{%
   $(LIBEXE) $(SOURCES) /out:$(TARGET:S=.lib)
%}

LIB1_SOURCES = src1.c src2.c src3.c ;
LIB2_SOURCES = src4.c src5.c src6.c ;

Library lib1.lib : $(LIB1_SOURCES) : ;
Library lib2.lib : $(LIB2_SOURCES) : lib1.lib ;

In the above example all of the code is in a single jamfile. Normally you want to separate the rules from the main rule invocation files where the top users source files and desired target files are named. To do this you use the include directive.

The main jamfile would look like this:

include "jambase" ;

LIB1_SOURCES = src1.c src2.c src3.c ;
LIB2_SOURCES = src4.c src5.c src6.c ;

Library lib1.lib : $(LIB1_SOURCES) : ;
Library lib2.lib : $(LIB2_SOURCES) : lib1.lib ;

The jambase file would include all the rules and actions. For most users they would never have to modify the jambase file. They would just add targets and sources to the jamfile. If you have written your rules well the user jamfiles will behave as if there are typed build targets. For example:

LIB1_SOURCES = src1.c src2.c src3.c ;
LIB2_SOURCES = src4.c src5.c src6.c ;
PROG_SOURCES = prog1.c ;
DATA_SOURCES = data.dat data2.dat ;
 
Library    lib1.lib    : $(LIB1_SOURCES) : ;
Library    lib2.lib    : $(LIB2_SOURCES) : lib1.lib ;
Dll        lib1.dll    : $(LIB1_SOURCES) : ;
Dll        lib2.dll    : $(LIB2_SOURCES) : lib1.dll ;
Executable program.exe : $(PROG_SOURCES) : lib1.lib lib2.lib ;
DataFile   data.zip    : $(DATA_SOURCES) ;

Running KJam in Network Mode

Normally KJam is simply invoked by running the KJam binary. In this mode KJam runs on the local machine and spawns build commands on the local machine. KJam can be run in network mode. In this mode a KJam server is started on build capable machines all over a local network. The servers listen for build instructions on a common port. Then a KJam is run in network build client mode. Just as in normal non network mode KJam will parse a jamfile, bind targets to the local file system, build a dependency graph and determine which targets need updating. This is all done locally just as in non-network mode. It will then issue the build commands to the KJam servers on the network.

To do this, first share the build directory on the network. Then on each machine on the network run KJam in server mode. The simplest way to do this is to just run it on the command line with the -z option. KJam can also be set up to run as a linux daemon or as a window service. All of the machines should be capable of running whatever programs are in the build actions. So for example, if a c-compiler is required for some of the actions, it must be installed on each server machine. All the server machines should also be able to see the shared drive.

So for example assume a network with three machines, M1, M2 and M3. Suppose on M1 the project directory is on c:/project. On M1 we share c:/project ss //M1/project. Then from M2 and M3 asking for a directory listing of //M1/project should work. Now on M1 M2 and M3 launch KJam -z. This will create a peer network of KJam servers. That is all the servers will automatically find each other on the network and organize themselves into a load balanced peer network.

On the client machine make //M1/project the working directory. This will not work if the current directory is set as c:/project, because this path is not visible on the network, and so the filename bindings will not work. From this directory start KJam with the -y option. This should build your project as normal, except that the build commands will be distributed onto all three machines.

Be aware that some programs when run on a shared directory my launch a little more slowly than they would if launched on a local directory. This means that for some programs which have very short execution times there may not be any time savings from running the build distributed on the network. For build steps which take a significant amount of time, network building can be sped up by a tremendous amount.

It is possible to control on which machines certain commands run. So for example suppose on a network you have 5 machines. Four of them, M1 M2 M3 and M4 have a compiler installed can run compiler build steps. And two of them, M4 and M5 have are connected to a common database and so can build data. In this case you would launch the servers on M1 M2 M3 with the flags "kjam -z -x compiler_system", M5 would have a server launched as "kjam -z -x data_system" and M4 would be started with "kjam -z -x compiler_system -x data_system". In the jamfile, the build actions for compiling would have the modifier "on compiler_system", and build actions for data would have the modifier "on data_system":

action on compiler_system CompileObject TARGET : SOURCE
{%
    $(CC) $(SOURCE) -o $(TARGET)
%}

action on data_system MakeData TARGET : SOURCE
{%
    makeData $(SOURCE) -o $(TARGET)
%}

Once this is set up, when build jobs are sent to the KJam build network, build commands are only sent to the appropriate systems.

KJam Reference

Usage

KJam [ -a ] [ -b ] [ -C ] [ -e ] [ -g ] [ -k ] [ -n ] [ -q ] [ -v ] [ -y ] [ -X ] [ -S ]
     [ -c path ]
     [ -d debug ] 
     [ -f inputFile ] 
     [ -h indent ] 
     [ -j jobs ] 
     [ -o outputFile ] 
     [ -s var=value ] 
     [ -t target ]
     [ -z [ port ]]
     [ -l power ]
     [ -x type ]
     [ -p subnet ]
     [ -i [ port ]]
     [ -P [ file ]]
     [ -L [ file ]]
     [ target ... ]

Description

KJam is a program construction tool, like make. KJam's design is based on Jam.

KJam recursively builds target files from source files, using dependency information and updating actions expressed in a user-provided jamfile, which is written in KJam's own interpreted language.

Command Line Options

If target is provided on the command line, KJam builds target; otherwise KJam builds the target 'all'.

Build Options: These options are available when KJam is running as a build client.

-a Build all targets, even if they are up-to-date.
-b Build targets separately, even when they can be built together.
-f file Read file instead of jamfile.
-g Build from newest sources first.
-j n Run up to n shell commands concurrently. This option is ignored in network client mode.
-k Skip file system prescan.
-n Don't actually execute the updating actions.
-o file Write the updating actions to file file.
-q Stop as soon as a target fails to build.
-t target Rebuild target, even if it is up-to-date.
-T target Print information about any target matching this regular expression.
-s x=y Set variable x=y, overriding environment.
-S Build command line targets in the order given on the command line.

Network Options: These options control distributed nework building for servers and clients

-z port Run as network server. Listen for job requests on the network and run them. If optional port is given, the server will listen on the given port instead of on the default KJam server port (2297).
-y Run as network client. Spawn jobs on known KJam servers.
-l power In server mode, specify power of the server system. By default all servers are given a power rating of 1. Systems with greater (or smaller) computational resources can be given a larger ( or smaller ) rating to help the load balancer assign build jobs appropriately
-x type In server mode, specify a server type, so that KJam build clients can assign jobs to a ppropriate servers.
-p subnet Ip address and port number of subnet to scan for build servers. e.g. 127.0.0.* would scan the local subnet for servers.
-i subnet Get information about the peer network from the server at this address.

Common Options: These options apply when running as both client and server.

-c path Run KJam in the given directory.
-C Do file system scanning and target binding in a case insensitive manner.
-d n Display debugging information. If a number is given, then all flags with that number or lower are turned on. If a token is given, then only the given flag is turned on.
show no diagnostics
make show actions when executed (default)
phases show start of phases
fate show progress of bindAndScan
build show progress of build
compile show rule invocations
header show result of header scan
search show attempts at binding
varSet show variable settings
varGet show variable fetches
varExp show variable expansions
if show 'if' calculations
scan show scanner tokens
parse show parser rule matching
makeQ show even quiet actions
exec show text of actions
thread show operation of threads
targets show targets in detail
actions show actions in detail
depends show dependency graph
network show network processing
path show file path components
regexp show regular expression parsing
pack show file packing and unpacking
file show file system operations
verify verify that built files exist
NETA print out diagnostics for NetAgent library
NETAERROR print out error diagnostics for NetAgent library
NETACONN print out detail about connection and disconnection
PING print out ping information
DL_LEX print out the operation of the dl lexer
DL_YACC print out the operation of the dl parser
-e Display elapsed time and the average number of concurrent threads.
-h num Indents the output by the given number of 3 character tabs. Useful for when KJam calls itself or is called by another process.
-v Print the version of KJam and exit.
-P filename Replace any currently built-in jamfiles with the given file. Multiple -P options may be given to support embedding multi-file build systems.
-X Extract all the built-in jamfiles.
-D Do not automatically include jambase in every jamfile
-I filename Search for jambase in the given directory. Multiple -I options may be given to specify multiple possible locations. If no -I is given KJam will search in .;..;scripts;../scripts
-L fileName If given an argument, reads that license file. With no argument displays license information

Operation

KJam has four phases of operation: start-up, parsing, binding, and updating.

Start-up

Upon start-up, KJam imports environment variable settings into KJam variables. Environment variables are split at blanks with each word becoming an element in the variable's list of values. Environment variables whose names end in PATH are split at $(SPLITPATH) characters (e.g., ":" for Unix).

To set a variable's value on the command line, overriding the variable's environment value, use the -s option. To see variable assignments made during KJam's execution, use the -d varSet option.

Parsing

In the parsing phase, KJam reads and executes the user provided jamfile. It is written in the KJam language. The purpose of the jamfile is to name build targets and source files, construct the dependency graph among them, and associate build actions with targets.

Binding

After parsing, KJam recursively descends the dependency graph and binds every file target with a location in the file system.

Any string value in KJam can represent a target, and it does so if the depends includes or codepends rules make it part of the dependency graph. Build targets are files to be updated. Source targets are the files used in updating build targets. Build targets and source targets are collectively referred to as file targets, and frequently build targets are source targets for other build targets. Pseudo-targets are symbols which represent dependencies on other targets, but which are not themselves associated with any real file.

A file target's identifier is generally the file's name, which can have a full path, a path relative to the directory of KJam's invocation, or simply local (no directory). Most often it is the last case, and the actual file path is bound using the $(SEARCH) and $(LOCATE) variables.

The use of $(SEARCH) and $(LOCATE) allows KJam to separate the location of files from their names, so that jamfiles can refer to files locally (i.e. relative to the jamfile's directory), yet still be usable when KJam is invoked from a distant directory.

After binding each target, KJam determines whether the target needs updating, and if so marks the target for the updating phase. A target is normally so marked if it is missing, it is older than any of its sources. This behavior can be modified by the application of special built-in rules.

During the binding phase, KJam also performs header file scanning, where it looks inside source files for the implicit dependencies on other files caused by C's #include syntax. This is controlled by the special variables $(HDRSCAN) and $(HDRRULE). The result of the scan is formed into a rule invocation, with the scanned file as the target and the found included file names as the sources. Note that this is the only case where rules are invoked outside the parsing phase.

Updating

After binding, KJam again recursively descends the dependency graph, this time executing the update actions for each target marked for update during the binding phase. If a target's updating actions fail, then all other targets which depend on that target are skipped.

The -j flag instructs KJam to build more than one target at a time. On multiprocessor systems, or in when running in network mode this can result in a significant performance improvement. If there are multiple actions on a single target, they are run sequentially. The -g flag reorders builds so that targets with newest sources are built first. Normally, they are built in the order of appearance in the jamfile.

Language

Overview

KJam has an interpreted, procedural language with a few select features to effect program construction. Statements in KJam are rule (procedure) definitions, rule invocations, updating action definitions, flow-of-control structures, or variable assignments.

Variables

All KJam variables are one-dimensional lists of arbitrary strings. They arise as literal (whitespace-separated) tokens in the jamfiles, as the result of variable expansion, or as the return value from a rule invocation.

Rules

The basic KJam language entity is called a rule. A rule is simply a procedure definition, with a body of KJam statements to be run when the rule is invoked.

Rules take up to any number of named arguments and can have a return value (a single list). A rule's return value can be expanded in a list by enclosing the rule invocation with [ ].

The syntax of rule invocation makes it possible to write jamfiles that look a bit like makefiles.

Actions

A rule may have updating actions associated with it. Actions also take any number of named arguments. For purposes of determining dependencies, the first argument is interpreted by KJam as a list of targets, and the second argument as a list of sources. Actions also contain a script consisting of operating system shell commands to execute when updating the built targets of the rule.

When a rule with updating actions is invoked, those actions are added to those associated with its built targets before the rule's procedure is run. Later, to build the targets in the updating phase, the actions are passed to the OS command shell, with the input arguments expanded into the text of the script.

Statements

KJam's language has the following statements:

rulename arg1 : arg2 : ... : argN ;

Invoke a rule or action. The named rule or action is invoked with values in arg1 through argN. These arguments will get expanded into the body of the rule when it is evaluated, or into the script body of the action when it is executed.

rulename undergoes variable expansion. If the resulting list is more than one value, each rule or action is invoked with the same arguments, and the result of the invocation is the concatenation of all the results.

action [ modifiers ] rulename [ : args ] [ on typelist ] {% commands %}

Define a rule's updating actions, replacing any previous definition. args defines the names of the actions arguments. When an action is executed the arguments passed to the action in the action invocation are substituted into the command script. commands specifies the command script. This script is passed into the operating system shell and executed.

The following action modifiers are understood:

existing Includes only sources whose bound files are currently existing. Useful for the implementation of clean rules.
ignore The return status of the commands is ignored. Normally if any of the commands return an error, the target will be assumed to have failed to build, even if a file is generated. Actions with the ignore modifier will behave as if the commands succeeded even if an error code is returned by the command script.
fail_expected The return status of the commands is inverted. Actions with the this modifier will behave as if an action succeeded if it failed, and visa versa.
piecemeal [num] Commands are repeatedly invoked with a subset of the sources small enough to fit in the command buffer on the current OS. If the optional number argument is given, the subset will never be larger than this number. This keyword is not compatible with the 'immediate' action modifier.
quietly The action is not echoed to the standard output.
together The sources from multiple invocations of the same action on the same built target are concatenated and if possible run as a single action. This keyword is not compatible with the 'immediate' action modifier.
updated Only sources whose associated targets are marked for updating are included. Actions with this keyword must have the same number of sources and targets, and the sources and targets must match.
retry [num] Normally kjam only tries to run each action once. If the action fails, kjam reports the error and terminates. The retry modifier tells kjam to try the action more than once, until the action succeeds. retry by itself will make kjam retry an action an unlimited number of times. retry plus a number will tell kjam to retry the action that number of times. retry 1 is equivalent to not setting retry at all.
timeout [num] [num] Normally all actions are given an unlimited amount of time to execute. This can be a problem when an action hangs. The timeout action modifier tells KJam to terminate an action if it takes longer than a certain amount of time. timeout allows the action to run for only a given number of seconds. If the action takes longer it is terminated. KJam will optionally try to run the action a certain number of times. The first numerical argument is the maximum number of seconds to allow the action to run. The second argument is the total number of times to try the action. timeout with no arguments is equivalent to timeout 60 1. That is allow the action to run for up to 60 seconds and run it only once. Timeouts shorter than about 5 seconds are rarely useful because KJam runs in parallel. Even a process which is expected to return instantly my take a few seconds to run if several other processes have started at the same time.
thread [num] Limits the number of simultaneous threads which will be spawned to process actions of this type. For example thread 1 will only will process this action with a single thread, such that multiple calls to it will never occur in parallel. When this keyword is not used, KJam will spawn as many threads as neccesary until it reaches KJam's overall thread limit ( set with the -j command line option. This option is useful in cases where a particular action cannot be run in parallel ( because it modifies a global resource, or uses a tool which is not thread safe ), or where there are limits to the performance benefits for running it in more than a certain number of threads ( such as actions which interact with limited hardware resources ).
immediate Causes the action to be run immediately upon parsing. Without this keyword actions run during the updating phase and only if one of their target needs updating. With the immediate keyword the action is run unconditionally, and because it is run during the parsing phase, changes it makes to the file system come in time to still affect dependency calculations. This keyword is useful for building subprojects, or updating sources from source code control. This keyword is not compatible with the 'piecemeal' and 'together' action modifiers.
unique Removes duplicate elements from this action's targets and sources.

In addition actions may have the on modifier which is followed by a list of server types. The on modifier is used when KJam is running in client-server network mode. In network mode, KJam servers may be given one or more types corresponding to the available system resources. For example if a server is running on a machine configured as a build server that server may advertize itself as type "build". A server configured as a rendering server or a database server may advertize as "render" or "database". Actions with on modifiers will only be spawned by the load balancer on on KJam servers of the given type. This allows distributed build systems which will only run certain commands on the appropriate system types.

break

Breaks out of the closest enclosing for or while loop.

continue

Jumps to the end of the closest enclosing for or while loop.

export varnames

Causes the named KJam variables to be exported to any spawned shells. When a new command shell is spawned all exported KJam variables are copied to the shell's environment variables.

for ( var in list ) { statements }

Executes statements for each element in list, setting the variable var to the current element in the list.

if ( cond ) { statements } [ else { statements } ]

Executes the given statements if the conditional expression cond is true. If the else clause is present then those statements will be run if the conditional expression is false.

The expression can be one of:

a true if any a element is a non-zero-length string
a == b list a matches list b string-for-string
a != b list a does not match list b
a < b a[i] string is less than b[i] string, where i is first mismatched element in lists a and b
a <= b every a string is less than or equal to its b counterpart
a > b a[i] string is greater than b[i] string, where i is first mismatched element
a >= b every a string is greater than or equal to its b counterpart
a in b true if a is a subset of b. That is if all elements of a can be found in b, or if a has no elements
a intersects b true if a intersects b. That is if a and b have at least one element in common.
! cond condition not true
cond && cond true if both conditions are true, false otherwise.
cond || cond true if either condition, or both are true.
( cond ) precedence grouping

include file ;

Causes KJam to read the named file. The file is bound like a regular target (see Binding above) but unlike a regular target the include file cannot be built. If the include file cannot be found an error is generated. Marking an include file target with the nocare rule makes it optional.

The include file is inserted into the input stream during the parsing phase. The primary input file and all the included file(s) are treated as a single file. That is, KJam infers no scope boundaries from included files.

[local] varnames [ on targets ][ ( = | += | -= | ?= ) values ] ;

Defines a variable. Variables associate a name with a list of zero or more string elements. An undefined variable is indistinguishable from a variable with an empty list. A defined variable may have one more elements which are null strings. The value of a variable may be referenced in KJam statements and in command scripts as $(variable).

varname is the name of the new variable. The value of the new variable is the list of strings specified by values. Variables may be global, local or target specific. A global variable it will expand to the given value everywhere until it is redefined with another global variable definition, or unless it is masked by a local or target specific variable. If a variable is defined locally, it will have that value until execution exits the nearest containing { }, or unless it is redefined locally within that block, or masked by a target specific variables. For target specific variables, the variable takes on the given value only during the target's binding, header file scanning, updating and during the evaluation of "on target" statement blocks.

The simplest variable definition, without additional keywords is global:

variable = values ;

Local variable definitions are specified by using the keyword local:

local variable = values ;

Target specific variables are defined by using the keyword on:

variable on target = values ;

The keyword on and local may not be combined in a single variable definition.

The variable definition uses one of three assignment operators:

= Creates a new variable, or overwrites and existing variable.
+= Creates a new variable, or adds new elements onto the end of the list of an existing variable.
-= Removes any elements of the variable on the right hand side from the variable on the left, if it exists.
?= Creates a new variable if one does not already exist, but will not overwrite an existing variable.

Multiple variable names may be set simultaneously by putting multiple variable names on the left hand side of the assignment operator. In the following case three variables are set to the same two string value:

var1 var2 var3 = value1 value2 ;

Variable names on the left hand side are subject to variable expansion before assignment. So it is possible to use the value of a variable as the name of a variable for assignment. The following case does the same thing as the one above:

VARNAME = var1 var2 var3 ;
$(VARNAME) = value1 value2 ;

on target { statements } ;

Run statement under the influence of target's target-specific variables. These variables become local copies during statement's run, but they may be updated as target-specific variables using the usual "variable on targets =" syntax.

return values ;

Within a rule body, the return statement sets the return value for an invocation of the rule and terminates the rule's execution.

rule rulename [ : args ] { statements }

Define a rule's procedure, replacing any previous definition. args is a series of argument names. When the rule is invoked, the arguments passed during invocation in are matched with the argument names as defined in the rule definition, and are set as local variables. These local variables may be referenced in variable expansion during rule invocation.

switch ( value ) { case pattern1 : statements ; case pattern2 : statements ; ... default : statements ; }

The switch statement executes zero or one of the enclosed statements, depending on which, if any, is the first case whose pattern matches value. The patterns can be any standard regular expression. Some regular expressions use characters with special meaning to the kjam language ( such as [] ) and must be expressed in quotes. The special default label will match any input value.

while ( cond ) { statements }

Repeatedly execute statements while cond remains true upon entry. (See the description of cond expression syntax under if ).

Variable Expansion

During parsing, KJam performs variable expansion on each token that is not a keyword or rule name. Such tokens with embedded variable references are replaced with zero or more tokens. Variable references are of the form $(v) or $(v:m), where v is the variable name, and m are optional modifiers.

Variable expansion in an action's command script is similar to variable expansion in statements, except that the action string is tokenized at whitespace regardless of quoting.

The result of a token after variable expansion is the product of the components of the token, where each component is a literal substring or a list substituting a variable reference. For example:

$(X) -> a b c
$(X) -> ta tb tc
$(X)z -> az bz cz
$(X)-$(X) -> a-a a-b a-c b-a b-b b-c c-a c-b c-c

The variable name and modifiers can themselves contain a variable reference, and this will be part of the product as well:

$(X) -> a b c
$(Y) -> 1 2
$(Z) -> X Y
$($(Z)) -> a b c 1 2

Because of this product expansion, if any variable reference in a token is undefined, the result of the expansion is an empty list. If any variable element is a null string, the result propagates the non-null elements:

$(X) -> a ""
$(Y) -> "" 1
$(Z) ->
_$(X)$(Y)_ -> _a_ _a1_ __ _1_
_$(X)$(Z)_ ->

Individual elements of a variable, or a subsection of the elements of a variable can be referred to using []:

[n] Select element number n (starting at 0). If the variable contains fewer than n elements, the result is a zero-element list.
[n-m] Select elements number n through m.
[n-] Select elements number n through the last.
[x,x] Multiple elements and ranges can be selected by separating selection elements with ,

These can be used like so:

$(X) -> a b c d e f g
$(X[0]) -> a
$(X[2]) -> c
$(X[2-4]) -> c d e
$(X[3-]) -> d e f g
$(X[1,3,5-]) -> b d f g

When getting the value of a variable, the returned value can be modified by referring to the variable through the use of modifiers. Many modifiers consist of a single letter which normally selects a subset of each value. Some modifiers are followed by =, and an argument. Many modifiers only make sense if the variable string can be interpreted as a file name. A variable reference may include any number of modifiers.

The modifiers are:

A[=dir] If the current variable is a path with wildcard characters, expands the variable to the full set of directories in the file system which match the expression. If no argument is given the directory search is conducted from the current directory. If the optional modifier is given the directories are searched from the given location.
B[=base] Return the filename base. If an argument is given, replace the base with the given string.
C Replace non-printable characters with their backslashed equivalents. e.g. newline will be replaced with \n etc.
D[=dir] Select directory path. If an argument is given, replace the directory with the given string.
E=value If the returned list would be empty, return the given value.
e[=subkey] Windows only: Interprets the variable as a registry key, and returns its value. If the optional subkey is given the subkey value is appended to the variable value before looking it up in the registry.
F[=file] If the string is a target, return the bound name of the element, otherwise return the element unmodified. If an argument is given then the target will be bound explicitly to the given file name. If the file does not exist, then the binding is set to 'missing'. This will override any other binding that would normally be done automatically using the $(SEARCH) or $(LOCATE) variables. WARNING: the :F modifier causes a target to get bound to a file or directory as soon as it is evaluated. This may cause the SEARCH and LOCATE variables which may be evaluated later to be ignored.
f Return value if it exists as a file in the file system.
g Returns only those elements which can be interpreted as global paths.
G[=grist] Return the target grist. If an argument is given, replace the grist with the given string.
H[=file] Return the filename without the directory. If an argument is given, filename with the given string.
I=regexp Include only elements which match the given regular expression.
J Join all the elements into a single string.
K Remove duplicate elements.
L Replace uppercase characters with lowercase.
M[=dir] If the current variable is a path with wildcard characters, expands the variable to the full set of files in the file system which match the expression. If no argument is given the file search is conducted from the current directory. If the optional modifier is given the files are searched from the given location.
n Interprets the variable as a path and normalizes it, removing unnecessary elements.
O[=split] Split a string along boundaries defined by split. If none is given space is assumed. If split is a string with more than one character, multiple characters will be used as the split point.
P Return parent directory.
Q Given a relative path, returns the reverse path, such that if a script was to cd to the given path, a cd to the reverse path would return the script back to the original directory. For example the path one/two/../three/./four would have the reverse ../../... Some paths, such as global rooted paths, or relative paths which start with .. have no reverse. Paths with no reverse return an empty variable. All reverse paths consist of just .. elements.
R Normally zero length elements are removed. If R is given, these elements are preserved.
r Returns only those elements which can be interpreted as relative paths.
S[=ext] Return the filename extension. If an argument is given, replace the file extension with the given string.
s Return all the subdirectories of the given directory path.
T Return the given list in alphabetical order.
U Replace lowercase characters with uppercase.
u Return all the files in subdirectories of the given directory path.
V Replace any backslashed character code with their proper replacement. e.g. \n will be replaced with the newline character etc.
W=num Add whitespace to the end of the variable until it is at least the given length.
W Remove leading and trailing whitespace.
X=regexp Exclude elements which match the given regular expression.
x Execute the value of this symbol as a system command, and return the output as the value of the variable.

The following modifiers make it possible to select or replace parts of any element which match any part of a given regular expression, by using a combination of the N modifier a numbered modifier:

N=regexp Specifies the regular expression to use as the match to select elements from.
0[=value] With no argument returns the part or the element which matched the expression. With an argument this returns the element with the part matched by the expression replaced by value.
1[=value] 3[=value] 4[=value] 5[=value] 6[=value] 7[=value] 8[=value] 9[=value] If the regular expression has () groups, these modifiers refer to these groups. With no argument this will select only the part which matched the given group. With an argument this modifier will replace the given group.

Some modifiers accept or require arguments. To give a modifier an argument follow it with = and the text of the argument like so:

SOURCES = foo.c bar.c ;
OBJECTS = $(SOURCES:S=.o) ;
# $(OBJECTS expands to foo.o bar.o

All text following the = is considered to be part of the argument until the expansions closing ). To express arguments with special characters, or to have multiple modifiers with arguments they must be quoted with " or ' like so:

TEXT = foo bar goo ;
TEXT2 = $(TEXT:N="oo"0="xx") ;
# $(TEXT2) expands to fxx bar gxx

TEXT = "foo=" "bar=" "goo=" l
TEXT2 = $(TEXT:N='='0='+') ;
# $(TEXT2) expands to foo+ bar+ goo+

Modifier arguments may themselves include variable expansions with their own modifiers. When a modifier argument expands to more than one element, only the first element will be used as the modifier argument. For example:

EXT = bmp png jpg ;
SOURCE_IMAGES = pony.mdl horse.mdl monkey.mdl ;
IMAGE_FILES = $(SOURCE_IMAGES:BS="$(EXT)") ;
# $(IMAGE_FILES) expands to pony.bmp horse.bmp monkey.bmp 

Normal variable expansion happens before the command shell is spawned. In network mode this happens on the client. It is possible to cause variable expansion to happen after the shell has spawned and during shell script execution, using server variable expansion syntax: $!(VAR)

In network mode, when variables are expanded this way, the variable settings used will be those on the server. This is also the way to expand using the environment variables as they are set in the spawned command shell. For example, to set an environment variable and then echo out its value:

{%
   shellvar = foo
   echo $!(shellvar) 
%}

Literal Expansion

In the same way that modifiers can be applied to variables they can also be applied to literals. You can specify a literal either by using its name directly, in quotation marks, or surrounded by @( and ). For example the following three lines are equivalent:

LITERAL = string ;
LITERAL = "string" ;
LITERAL = @(string) ;

Using the third form allows for the use of modifiers. For example to uppercase a string literal:

LITERAL = @(string:U) ;

To get the directory where a certain literal file can be found:

MYFILE_DIR = @(myfile.txt:FD) ;

Variable expansions and literal expansions can be nested in each other. By combining literal and variable expansions with modifiers extremely powerful expressions can be specified. For example to find all the directories where log files with a certain name and different file extensions can be found:

LOGFILE_EXT = .log .txt .out ;
MYFILE_DIRS = @(myfile$(LOGFILE_EXT):FD) ;

Like variable expansion, literal expansion may be done on the server using @!(. In most cases this functions exactly like normal literal expansion and is only useful in combination with variable server expansion, that is with $!(.

Grist

Grist is used to make targets which would otherwise have the same file name unique. It is sometimes the case that there will be two different files which have the same name but exist in different directories. Sometimes this can be solved by using part of the directory name as part of the target name. Sometimes this is not practical. KJam allows target names to be prefixed with a "grist" string which is not used by KJam for binding, but is used in order to distinguish targets for the purposes of updating or computing dependencies.

The grist string is a prefix on a target surrounded by < > :

TESTS = <project1>test.data <project2>test.data ;

Built-in Rules

KJam has 18 built-in rules. They are in three groups. The first modifies the dependency graph. The second modifies how targets are bound and built, and the third are utility rules.

Dependency Building: The following rules are used to explicitly add links to the dependency graph.

depends targets1 : targets2 ;

Builds a direct dependency. Makes each of targets1 depend on each of targets2. Generally, targets1 will be rebuilt if targets2 are themselves rebuilt are or are newer than targets1.

includes targets1 : targets2 ;

Builds a sibling dependency. Makes any target that depends on any of targets1 also depend on each of targets2. This reflects the dependencies that arise when one source file includes another. The object built from the source file depends both on the original and included source file, but the two sources files don't depend on each other. For example:

depends  foo.o : foo.c ;
includes foo.c : foo.h ;

foo.o depends on foo.c and foo.h in this example.

codepends targets ;

Specifies that all the targets on the list co-depend on each other. That is if another target depends on one of these it will also depend on all the others in the list. For example:

codepends foo bar ;
depends baz : foo ;

Then baz will also depend on bar.

cleardep targets ;

Removes all dependencies of the given targets accumulated up to that point.

clearinc targets ;

Removes all sibling or 'include' dependencies of the given targets accumulated up to that point.

registry keys : value ;

On Windows only, sets the named keys in the windows registry to the given value. The last element is interpreted as the name of the data element of key to set.

registryd keys : value ;

On Windows only, sets the named keys in the windows registry to the given value as the default unnamed key data element.

Modifying Binding:

The seven rules always, leaves, nocare, notfile, noupdate, and temporary modify the dependency graph so that KJam treats the targets differently during its target binding phase. Normally, KJam updates a target if it is missing, if its file system modification time is older than any of its dependencies (recursively), or if any of its dependencies are being updated. This basic behavior can be changed by invoking the following rules:

always targets ;

Causes targets to be rebuilt regardless of whether they are up-to-date (they must still be in the dependency graph). This is used for the clean and uninstall targets, as they have no dependencies and would otherwise appear never to need building. It is best applied to targets that are also notfile targets, but it can also be used to force a real file to be updated as well.

leaves targets ;

Makes each of targets depend only on its leaf sources, and not on any intermediate targets. This makes it immune to its dependencies being updated, as the "leaf" dependencies are those without their own dependencies and without updating actions. This allows a target to be updated only if original source files change.

nocare targets ;

Causes KJam to ignore targets that neither can be found nor have updating actions to build them. Normally for such targets KJam issues a warning and then skips other targets that depend on these missing targets. nocare can be used when making header scanning rules to let KJam know that some header files may not exist. This gets around problems where files may conditionally include headers found on only some systems.

notfile targets ;

Marks targets as pseudo-targets and not real files. No time-stamp is checked, and so the actions on such a target are only executed if the target's dependencies are updated, or if the target is also marked with always. The default KJam target all is a pseudo-target.

noupdate targets ;

Causes the time-stamps on targets to be ignored. This has two effects. First, once the target has been created it will never be updated. Second, manually updating target will not cause other targets to be updated. This is useful for rules which create directories. Such rules only care that the target directory exists, not when it has last been updated.

precious targets ;

Normally if an action which produces a target fails the target is assumed to be malformed and deleted if it exists. If a target is labelled as precious it will not be deleted even if its action fails.

temporary targets ;

Marks targets as temporary, allowing them to be removed after other targets that depend upon them have been updated. If a temporary target is missing, KJam uses the time-stamp of the target's sources. temporary can be used to make intermediate files like .obj files, so that they can be deleted after they are archived. Targets marked as temporary are not automatically deleted by KJam.

Utility Rules: The remaining rules are utility rules.

echo args ;

Prints out the contents of each argument as a list of items, separating the lists and arguments for clarity.

print args ;

Prints out the contents of each argument with all the arguments concatenated together, in order to make nicely formatted error messages.

exit args ;

Blurts out the message args to stdout and then exits with a failure status.

show_target args ;

Shows the current state of the given targets, including its bound name, binding flags, dependencies, and local "on" variables.

show_rule args ;

Shows the current state of the given rules, its flags and argument list.

show_variables ;

Shows all variables that apply in the current rule.

show_exports ;

Shows all variables that will be exported to the shell.

ordered_actions args ;

The named targets will execute their action one at a time in the order they are specified. Normally targets with multiple action will try to execute all their actions simultaneously.

Built-in Variables

KJam has several special variables which change the behavior of the program.

These two variables control the binding of file target names to locations in the file system. $(SEARCH) is used to find existing sources while $(LOCATE) is used to fix the location for built targets.

Rooted (absolute path) file targets are bound as is. Unrooted file target names are also normally bound as is, and thus relative to the current directory, but the settings of $(LOCATE) and $(SEARCH) alter this.

  • If $(LOCATE) is set then the target is bound relative to the first directory in $(LOCATE). Only the first element is used for binding.
  • If $(SEARCH) is set then the target is bound to the first directory in $(SEARCH) where the target file already exists.
  • If the $(SEARCH) search fails, the target is bound relative to the current directory, as if $(LOCATE) and $(SEARCH) had not been set.

The $(SEARCH) path can be defined using wildcard characters to match multiple directories:

      $(SEARCH) = obj/*/debug ;

HDRSCAN and HDRRULE Variables

These two variables control header file scanning. By using $(HDRSCAN) and $(HDRRULE) it is possible to eliminate the need for an explicit dependency making step, or the use of external dependency building tools such as makedepend.

When $(HDRSCAN) is set "on" a target, if that target is a file, KJam will scan that file for inclusion statements. The value of $(HDRSCAN) should be set to a regular expression which matches inclusion lines in the given source file. The pattern should match the entire inclusion statement. The pattern should include a single () grouping which indicates the filename being included.

Whenever a match is found by KJam using $(HDRSCAN), KJam will invoke a the rule named by the $(HDRRULE) variable. Typically this allows the user to create a rule to recursively scan files for inclusion and to build the dependency graph.

Both $(HDRSCAN) and $(HDRRULE) must be set for header file scanning to take place, and they should be set target-specific and not globally. If they were set globally, all files, including executables and libraries, would be scanned for header file include statements. Careful use of these variables allows for different $(HDRSCAN) patterns to be used for different source file types.

The $(HDRSCAN) and $(HDRRULE) variables must have the same number of elements. Normally, if there is only one pattern to scan for a given file type, then both variables will be set to lists with a single element. But this it not requires. You can set these variables to lists with multiple elements. This will cause the header scanning to use multiple regular expressions, and to call a different rule for each pattern. This can be useful for languages with more than one type of inclusion directive, and where the different types of inclusion directives must be treated differently.

The scanning for header file inclusions is not exact. The scanning mechanism errs on the side of inclusion. That is, it is more likely to return filenames that are not actually used by the compiler than to miss include files because it can't tell if #include lines are inside #ifdefs or other conditional logic. It is normally a good idea to apply nocare to each header file found during scanning so that if the file isn't found it doesn't cause the compilation to fail.

Scanning for regular expressions only works where the included file name is literally in the source file. It can't handle languages that allow including files using variable names (as the KJam language itself does).

JAM_SHELL Variable

By default KJam uses its own internal sh-like shell. This allows KJam command scripts to run in a common execution environment no matter what shells are installed on the target platform. The built-in shell may not be appropriate for all applications. To call out to a specific operating system shell, a jamfile may set the $(JAM_SHELL) variable:

JAM_SHELL = /bin/sh -c % ;

The % is replaced with the text of the command script. If % is not provided, the text of the command script is added at the end.

Jam Built-in Variables

Time and Date Variables: All time and date variables are evaluated at program start-up time.

JAM_DATE Time and date in asctime format, which is local time.
JAM_DATE_RFC1123 Time and date in RFC1123 format, which is GMT time.
JAM_DATE_ANSI Time and date in ANSI/ISO format, which is in local time.
JAM_MONTH_NAME The current month name (e.g. January)
JAM_WEEKDAY_NAME The current weekday name (e.g. Monday)
JAM_YEAR The current year (e.g. 2012)
JAM_MONTH The current month ( from 1-12 )
JAM_YEAR_DAY The number of days since Jan 1 ( from 0-365 )
JAM_MONTH_DAY The number of days since the start of the month ( from 1-32 )
JAM_WEEK_DAY The number of days since Sunday ( from 0-6 )
JAM_HOUR The number of hours since midnight ( from 1-24 )
JAM_MINUTE The number of minutes after the hour ( from 0-59 )
JAM_SECOND The number of seconds after the minute ( from 0-61 )
JAM_IS_DST Defined to "true" if daylight savings time is in effect, false otherwise.

Other Variables:

JAM_VERSION KJam version, as reported by kjam -v.
JAM_CWD The directory KJam was invoked in.
JAM_USER The name of the current user, as reported by the operating system.
JAM_PLATFORM Identifies the major operating system family that this version of KJam is built to run on. For Windows this value is "Win32", on Linux the value is "Linux"
JAM_UNAME For unix versions of KJam this variable indicates the specific operating system version as reported by a call to the uname() system call. The variable is a vector with five elements, machine, os version, os release number, node name, and system name.
JAM_NAME The name of the KJam executable. Usually set to kjam, but may be different if you rename KJam. Useful for actions which need to recursively call KJam.
JAM_CMD_OPTIONS The command line options passed to KJam.
JAM_CMD_TARGETS The top level build targets passed to KJam on the command line.
JAM_CMD_FILES The names of the jamfiles to read. JAM_CMD_FILES is set when input files are passed on the command line with -f, or it can be set from inside the jambase file. If this is not set in either place kjam will set it to the defailt jamfile. JAM_CMD_FILES cannot be set from inside jamfiles. By setting JAM_CMD_FILES in the jambase users can set the name the default jamfile to something other than jamfile
JAM_WRAP This controls KJam's output autowrapping feature. Set this variable to a set of three numbers, start, end and margin. start sets the column output gets wrapped to when a line is too long. end sets the maximum length of a line. Lines longer than this will be cut short and the remainer of the output will be printed indented to start. margin will cause the wrap feature to cut lines short on the first whitespace character on the column indicated by end - margin. For example JAM_WRAP = 10 $(COLUMNS:E=80) 20 ; will set reasonable wrap settings for most shells. Leaving JAM_WRAP unset, or setting it to 0 0 0 will turn off the autowrapping feature.
JAM_TARGETS All the targets defined so far.
JAM_BUILDABLES All the targets buildable defined so far, that is targets with at least one action.

Built-in Shell

KJam uses its own built-in sh-like shell to execute command scripts during the updating phase. The KJam shell separates the script into command chains on newline boundaries. Each command chain consists of a series of operating system commands separated by | pipe operators. Each command is passed to the operating system and run. Its output stream is redirected to the input stream of the next command in the chain. The error stream of each command is redirected to a common error stream for the command chain. If any command returns a non zero return code, all of the commands in the chain are terminated and the shell will return this error code.

The output and error streams of the command chain may be redirected to one another or to a file by appending one of the following to the end of the command chain:

> file
1> file
Redirects the output stream to the given file. If the file exists, its previous contents are replaced.
>> file
1>> file
Redirects the output stream to the given file. If the file exists, the output is appended to the file's existing contents.
2> file Redirects the error stream to the given file. If the file exists, its previous contents are replaced.
2>> file Redirects the error stream to the given file. If the file exists, the output is appended to the file's existing contents.
$> var
$1> var
Redirects the output stream to the given KJam variable. If the variable exists, its previous contents are replaced.
$>> var
$1>> var
Redirects the output stream to the given KJam variable. If the variable exists, the output is appended to the variable's existing contents.
$2> var Redirects the error stream to the given KJam variable. If the variable exists, its previous contents are replaced.
$2>> var Redirects the error stream to the given KJam variable. If the variable exists, the output is appended to the variable's existing contents.
1>&2 Redirects the output stream to the error stream.
2>&1 Redirects the error stream to the output stream.

Any output redirected to a file named nul will cause that output to be deleted.

A command chain may take its input from a file by use of input redirection:

< fileRedirects the input stream to this command chain to come from the given file.
$< fileRedirects the input stream to this command chain to come from the given kjam variable.

The built-in shell includes several built-in commands which the shell interprets itself, instead of calling out to the operating system.

cat files Contatenates all the given files and copies their contents to the output stream.
cd dir Change the current working directory for the shell.
chmod files Update the time permissions for the given files.
cp [-h]src dst Copies the file or directory given by src to the directory or filename given by dst. If the source is a directory the contents of the directory and its subdirectories will be copied. In a recursive copy, when the -h option is given hidden files and directories in subdirectories will not be copied. On Linux the non-copied files are files starting with ".". On Windows hiddden files either have the hidden file attribute set, or start with "." for compatibility with Linux
echo arg Prints the given arguments to the output stream.
mkdir dirs Creates all the given directories. The directories may be more than one layer deep.
mv src dst Moves the file given by src to the directory or filename given by dst.
rgxrpl [-x expr] [-[0-9] expr] [-g expr] [-d #] Manipulates the input stream using regular expressions, and copies the modified data to the output stream.
rm [-r] files Removes the given files. If -r is given, it will remove the given files and recursively delete the contents of any given directories.
set Prints out the current shell environment variables.
touch files Update the time for the given files. If any file does not exist, it creates a new empty file by that name.
warn arg Prints the given arguments to the error stream.
kjsh arg Runs a given shell script in a new instance of the KJam built-in shell. The script can be passed in as a file name argument, or streamed into the shell's standard input stream.
ccvt arg Convert a text or binary file into a c source file that will compile into a buffer that holds that file.
show_variables Show all the kjam variables available generate this shell action script.
show_shellvars The shell variables defined in this script.
show_variables Show all the variables that will be exported to the environment of any processes spawned by this shell script.

These built-in commands may mask existing commands on your system. To use the operating system versions, use a complete path to that command, an alias or simply surround the command name in quotes, or run your script using the operating system's own shell.

Comments begin with # and continue to the next newline character.

The KJam shell is very limited, and supports only the shell features normally used in short command scripts found in make files. For example it does not have support for flow control, like sh's if, do, while and case statements.

To set shell varibles use =, like so:

foo = bar

To get the value of a shell environment variable use server variable expansion syntax:

echo $!(foo)

Using normal variable expansion syntax will not work, because those variables will be expanded before the command shell is spawned, when the action script is still being assembled.

kjsh

This launches a new instance of the KJam built-in shell, and runs the given shell command script. The script can be passed in either by passing in the name of a file containing the script, or by passing the contents of the script through the shell's standard input stream. If multiple shell scripts are passed in as arguments to kjsh, each one will run in turn in different instances of the shell.

The subshell can still modify KJam's global environment variables, and will process variable expansions just like shell scripts called as part of KJam actions.

The script will be run immediately when called, and will not be scheduled as a parallel job the way most actions are run. The script will not continue until the subshell returns.

chmod

This tool changes the file permissions. Command takes arguments of the form:

chmod [-Rf] mode files

The -R option changes the permissions of all files and subdirectories under a given directory. The -f option will cause chmod to not issue errors concerning file permissions if any are encountered.

The mode can either be an octal number representing the actual bit pattern of the file persmissions, or a symbolic mode of the form:

[who] op permission [op permission]

Where who is one of: u, g, o, a, which represents setting permissions for user, group, other or all respectively.

Where op is on of: +, -, which represents adding that permission or removings it respectively.

and where the permission itself is one of the following:

r read permission
w write permission ( ignored on Windows )
x execute permission ( ignored on Windows )
X execute permission ( ignored on Windows )
t sticky bit ( ignored on Windows )
I setuid ( ignored on Windows )
G setgid ( ignored on Windows )
s setuid and setgid ( ignored on Windows )
h make hidden ( windows only )
a set archive flag ( windows only )
T set temporary flag ( windows only )

rgxrpl

This is a tool to manipulate text using regular expressions. The program reads data from standard input, changes it using the regular expressions given on the command line, and writes the modified data to standard output. Each command line option will either match some part or the input, or will modify matched data in some way. The command line operations are applied to the data from left to right as they appear on the command line. The operations are:

-x expr Scan the data for matches with the following regular expression
-[0-9] expr Replace data matched with a previous regular expression with the given string. -0 will replace the entire matched expression. -1 will replace just the first subexpresion.
-d number Delete the matched expression from the output. If the number is 0 the entire expression is deleted. Numbers 1-9 will delete the numbered sub-expression.
-g expr Only copy lines which contain a match to the output stream. Like grep.

Regular Expressions

The KJam language uses regular expressions anywhere where flexible matching of text is desired. As noted above this includes using regular expressions as arguments to the X, I and N variable expansion modifiers, as the labels in a switch statement, and as the value of for the HDRSCAN built-in variable. KJam's implementation of regular expressions matches closely with the syntax of standard Posix regular expressions, and should be familiar to many users.

A regular expression consists of one or more non-empty branches separated by |. A regular expression matches any text which matches at least one of those branches. For example, the expression one|two will match the string one and the string two.

A branch consists of one or more concatenated atoms. There are several types of atoms, and each one can be used to match a given text. The types of atoms are:

  • a regular expression enclosed in (), matches the given regular expression,
  • . matches any single character,
  • ^ matches the beginning of a line,
  • $ matches the end of a line,
  • \A matches the beginning of input,
  • \Z matches the end of input,
  • < matches the start of a word,
  • > matches the end of a word,
  • \w matches any letter or punctuation character,
  • \s matches any whitespace character,
  • \d matches any digit,
  • \ followed by a digit, ( see back references )
  • a character other than ^.[$()|*+?{\, matches that character,
  • a \ followed by any character except wsd or a digit, matches the character following the \,
  • a bracket expression.

By itself each atom matches exactly one copy of the matching input text. Each atom may be followed by one of the following, which changes how many times the atom matches:

  • * which causes the previous atom to match zero or more times
  • + which causes the previous atom to match one or more times
  • ? which causes the previous atom to match zero or one times
  • a bound

A bound is { followed by a number, possibly followed by , possibly followed by another number, followed by }. If there are two numbers the first must be equal to or less than the second one. An atom followed by a bound containing one integer i matches exactly i times. An atom followed by a bound containing two integers i and j of matches at least i times and not more than j times.

A bracket expression is a series of characters enclosed in []. It normally matches any single character from the list. If the list begins with ^, it matches any single character not in the list. If two characters in the list are separated by -, this is shorthand for the full range of characters between those two in the collating sequence.

For example the atom [abc]* would match the first 5 letters of bcbacXacd. The atom [^X]* would also match the first 5 letters of bcbacXacd. The atom [A-Z0-9] would match the first 5 letters of DS14Tabc.

A bracket expression may also contain inside its [] one of the following special character classes:

  • [:alpha:] matches any letter, and is equivalent to [A-Za-z]
  • [:alnum:] matches any letter or digit and is equivalent to [A-Za-z0-9]
  • [:upper:] matches any upper case letter and is equivalent to [A-Z]
  • [:lower:] matches any lower case letter and is equivalent to [a-z]
  • [:digit:] matches any digit and is equivalent to [0-9]
  • [:xdigit:] matches any hexadecimal digit and is equivalent to [0-9xX]
  • [:space:] matches any whitespace character including space, tab, newline, and carriage return
  • [:punct:] matches any punctuation character
  • [:cntrl:] matches any non-printable non whitespace character
  • [:graph:] matches any printable or whitespace character
  • [:print:] mathces any printable non-whitespace character

For example the atom [[:alpha:]_]* would match the string this_is_a_test.

A regular expression may also contain back references. All regular expression atoms, that is any atom surrounded by () are numbered. They are numbered in the order that their ( appears in the expression. So the leftmost regular expression atom would be numbered 1, the next one 2 etc. The entire expression is numbered as zero. A back reference atom matches the exact same string as the numbered atom it refers to.

For example a(b*)c\1d matches any string with the same number of b's on either side of the c such as the following strings: acd, abcbd, abbbcbbbd.

Network Execution

KJam may be run in a distributed client-server mode over a network. In this mode, KJam is started on each machine as a server process, with the -z option. If all the servers are on the same subnet and using the default KJam network port (2297), then all the servers will find each other and organize themselves into a peer network. It is possible to create peer networks of servers on alternative ports, and across subnets by using additional command line options.

KJam is run as a network client with the -y option. In this mode by default KJam will attempt to connect to a KJam server on the local machine, on the KJam network port (2297). It will run through its normal operation just like in non network mode, except when running command scripts, instead of running them locally, the scripts will be sent to be run on one of the KJam servers on the peer network. The peer network's own load balancing algorithms will distribute command scripts across the peer network. Multiple KJam clients may be interacting with the peer network at any time. By using additional command line options is it possible for the KJam client to connect to any server or port to get its initial pointer into the KJam peer network.

If KJam is started as a network client and it cannot find a peer network it will scan for one on the local subnet. While this works, it is considerably slower than either making sure there is a KJam server running on the local machine, or providing appropriate information about where one can be found.

In most cases it is important that the file system used by KJam to bind files on the local system be visible on all of the remote KJam servers in the exact same way. The commands passed to the remote servers will be run on those servers with the bindings determined on the KJam client. The files themselves are not transferred, but rather the remote server will attempt to read them from its view of the file system. Similarly, the output files and the programs being run are also not transferred. All of these things must be available on the remote server system. There are several easy ways to accomplish this. The simplest is to make the part of the file system that the KJam client is running in be visible on the network from all the machines. ( For example on windows by 'sharing' ). Then run the client in the subdirectory using the shared version of the directory. ( e.g. //SYSTEMNAME/PROJECTNAME ). In unix file systems this problem should be easier to address if all of the machines have had their local file systems mounted into one large virtual file system.

In network mode, special attention should be paid to the configuration of compilers and the versions of libraries and headers on all the KJam servers. It is possible with subtle changes in these tools for the build to appear to complete correctly but for the binary to have problems. It should also be noted that when running in network mode most programs will be running on one system but interacting with a file system on another. This often runs more slowly than if the entire process was local. For some programs which execute in very little time, or have heavy file access patterns it may be faster to run those command scripts entirely locally, and only distribute onto the network command scripts with more computationally intensive scripts.

Server command line options:

-z [port]Invokes KJam as a network server. The -z option can take an additional argument giving an alternate port number for the server to listen for peers and client. Only one -z and port number may be given.
-p scan By default the server will search for peers on the local subnet ( 127.0.0.* ). Use -p to make the server search in other places. The argument to -p can be a complete ip address ( e.g. 192.168.100.100 ), or certain subnet to scan ( e.g. 192.168.100.* ). The address may be expressed by itself, in which case KJam will scan the default KJam port number, or with an explicit port number to scan ( e.g. 192.168.100.100:5555 ). Multiple -p options may be given.
-l power This option sets the computational power this server will advertize itself as. Power may be any floating point number. By default all servers will advertize themselves as power 1. More powerful servers may be given a higher number, and less powerful ones a lower number. This power number affects how the load balancing algorithms distributes command scripts.
-j jobs This option sets the maximum number of simultaneous commands scripts the given server can run. A good rule of thumb for most servers is to set this to two times the number of physical processors.
-x type Sets this server to advertize itself as a server of a given type. Some build actions may be specified in the jamfile as only allowed to run on systems of a certain type. Multiple -x options may be given for systems which are configured for multiple job types.

Client command line options:

-y Runs KJam in network client mode.
-p scanBy default the client will look for a server on the local machine, and on the standard KJam network port ( 127.0.0.1:2297). Use -p to make the client search in other places for its initial KJam server. The argument to -p can be a complete ip address ( e.g. 192.168.100.100 ), or certain subnet to scan ( e.g. 192.168.100.* ). The address may be expressed by itself, in which case KJam will scan the default KJam port number, or with an explicit port number to scan ( e.g. 192.168.100.100:5555 ). Multiple -p options may be given. It is not necessary for the given -p directives to cover the entire set of possible KJam server addresses. It is only necessary that the client be able to locate one of the peers quickly. Once that happens, the peer network will send the KJam client a complete list of available KJam servers on the peer network.

It is possible to make KJam spawn jobs on only certain servers. To do this the servers can advertize one or more server 'types' using the -x option. Each action can be optionally labeled with a list of server types which the action can run on. When an action script has a server type, the peer network will only schedule that job to run on a server which advertizes that type. In addition to the server types specified by the -x option, KJam servers advertize several default types. All servers will advertize the local machine name as one of their types. This is so that scripts can be easily written where certain actions run only on a specifically named system. Linux machines will advertize as "Linux". All Microsoft Windows machines will advertize "Win32". Depending on the version of Windows, those machines will also advertize one of the following types: "WinXP" "Win2000" "WinNT" "Vista" "Longhorn" "Win2003ServerRc2" "Win2003Server" "Win95" "Win98" "WinME" "Win32s". Servers will also advertize the processor architecture, "i386", "i686" or "AMD64".

Case Sensitivity

The KJam language itself is always case sensitive. Case is important when referring to variables, targets, rules, and actions.

By default KJam also interacts with the file system in a case sensitive manner, even on operating systems with case insensitive file systems like Windows. In most cases, even on Windows this should produce correct behavior if the jamfiles are written to carefully match the case of the actual file names.

In some cases this can be a problem when the same file is referred to in multiple places in the sources ( e.g. include directives ) with different cases. In this case KJam may fail to properly track some dependencies, or fail to find some targets. The preferred, cleaner solution is to fix the sources such that files are referred to with the correct case. If this is not possible KJam can be run in case insensitive mode using the -C option.

The -C option is almost never useful when KJam is running on an operating system with a case sensitive file system ( like Linux or Os X ). It can sometimes be useful on these platforms when generating build scripts for Windows using the -o option.

The -C option will cause a small performance penalty to file system scanning speed.

Managing Built-in Jamfiles

KJam comes configured with a set of built-in jamfiles. This allows KJam to support many simple c and c++ projects with extremely simple user-defined jamfiles.

For more complex projects most users will want to write their own custom jamfiles. These can be written as separate jamfiles and stored in the filesystem like regular files, or they may be embedded into KJam, replacing the built-in rules.

To extract the built-in jamfiles, run KJam with the -X option. KJam will write out all its embedded rule files in the local directory with their original names. The KJam binary will still hold copies of the embedded files.

To replace the built-in jamfiles, run KJam with one or more -P filename options. KJam will remove any existing embedded jamfiles, and will embed all the files passed in through the -P option. If all the files passed in are found, and KJam is able to embed all the passed in files successfully, KJam will write out a new binary named kjam_new which has the new files embedded. This new binary will run as if all the embedded jamfiles were in the same directory as the kjam binary. Once it has been confirmed that the new binary runs correctly, it can be safely copied over the original KJam binary.

To remove all the embedded files, and replace them with nothing, use a single -P with no arguments.

Performance Considerations

KJam is designed to be able to compute dependencies between thousands of targets, and to determine which ones need to be updated in just a few seconds. Even for large projects, KJam's response time should be very fast. When writing a jamfile the following things may adversely affect performance:

File System Latency: Each time KJam runs it must find every target in the file system, and get its time stamp. To scan for headers it may also need to read the contents of many files. If it is interacting with a slow file system, such as one which is mounted over the network, this may cause KJam to run much more slowly.

Too many directories in $(SEARCH): During the binding phase, each file is searched for in each directory specified in $(SEARCH). The number of checks scales as the product of the number of targets by the number of directories to search. For large projects with many files and directories it is important to limit the number of directories that each file is searched for. Divide up the targets into groups ( for example, by library, or separate data files from source files etc. ), and search for targets in each group in the smaller subset of directories in which they occur.

$(HDRSCAN) pattern too complex: A regular expression is used to scan files for header references. If there are a large number of files, or if the files are large the time spent scanning can be quite significant. An attempt to match each pattern is made at every byte offset in the file. That is for a 5k file, 5k matches must be attempted. Patterns where the first character in the pattern always occurs will execute much more quickly than patterns where the first element is optional. For example, the first header matching expression is much more efficient than the second one:

HDRPATTERN = "#[\t ]*include[\t ]*[\"]([^\"]*)[\"]" ;

HDRPATTERN = "[\t ]*#[\t ]*include[\t ]*[\"]([^\"]*)[\"]" ;

Very long files to scan: Some projects may include automatically generated source files which can be very large ( many megabytes ). It can take a significant amount of time to read in and scan these files for headers. In addition it is usually unnecessary to scan such files any further than the first few lines. To limit the how far into large files to scan set $(HDRSIZE) to the length in bytes of the depth to scan each file. Only the first $(HDRSIZE) bytes of each file will be checked. Setting this variable to zero or a non-number will cause the entire file to be scanned.

Overuse of flow control: Many things which are possible to do with flow control constructs like for and while can be done faster by use of expansion modifiers. For example:

for ( I in $(C_FILES) ) { O_FILES += $(I:S=".o") ; }

will execute faster when expressed as:

O_FILES = $(C_FILES:S=".o") ;

Re-evaluating expansions unnecessarily: In KJam it is much more efficient to create a new temporary variable than to re-evaluate a variable expansion. For example, the following code:

LOCATE on $(TARGET) = $(BIN_DIR)$(OBJ_SUBDIR) ;
depends $(TARGET) : $(BIN_DIR)$(OBJ_SUBDIR) ;

can be expressed faster as:

OUTPUT_DIR = $(BIN_DIR)$(OBJ_SUBDIR) ;
LOCATE on $(TARGET) = $(OUTPUT_DIR) ;
depends $(TARGET) : $(OUTPUT_DIR) ;

Diagnostics

In addition to generic error messages, KJam may emit one of the following:

warning: unknown rule X

A rule was invoked that has not been defined with an "actions" or "rule" statement.

updating N target(s)

Targets are out-of-date and will be updated.

can't find N target(s)

Source files can't be found and there are no actions to create them.

can't make N target(s)

Due to sources not being found, other targets cannot be made.

warning: X depends on itself

A target depends on itself either directly or through its sources.

don't know how to make X

A target is not present and no actions have been defined to create it.

X skipped for lack of Y

A source failed to build, and thus a target cannot be built.

Warning: Target X will be built as a side effect of building another target.

A target has been requested which causes at least one other target to build which was not requested. This may happen for build actions which have multiple targets when only one target is requested but the action is forced to build all the targets.

Bugs and Limitations

A poorly set $(JAM_SHELL) is likely to result in silent failure.

Actions set with the 'together' modifier may not have more than three arguments.

When building in parallel sometimes the tools which KJam calls may try to access a file while it is being accessed by another tool in another thread or process. This may cause some of these programs to fail due to a file being locked or busy. KJam itself always checks for these conditions and should never have problems with files which are temporarily locked or busy. Some tools however do not check for this condition and may in rare instances fail unpredictably. The only way to fix this problem would be to rewrite those tools to interact with the file system more carefully.

KJam's dependency scanner is regular expression based. It can be used to create dependency scanners which trigger off simple include expressions, but cannot be used to evaluate inclusion statements which are affected by macros. Therefore it is possible for KJam to generate extra dependencies for included files which in reality are not actually included because the include statement is controlled by a conditional statement. It is often the case that conditionally included files may not actually exist on some platforms resulting in missing file errors. If this is the case, these errors can be silenced by labelling those files as nocare. In a correctly written jamfile this can only result in extra dependencies, and never in missing dependencies. It is possible if a jamfile has incorrectly set search paths for an include file which is necessary to not be found and yet for an error not to be generated because the header file was labeled nocare. This results in missing dependencies which can affect the correctness of builds. When jamfiles are being developed which include the keyword nocare, this condition should be checked for by running the KJam with the -d search command line option, and the list of files not found should be carefully checked for correctness.

Built-in Jambase Reference

The KJam program implements a programming language designed to implement build systems. Users implement their build systems by writing jamfiles which specify to KJam how to build the user's project. Normally a user will want to divide up their Jamfiles into two, one which specifies rules and actions and which is shared across the user's projects, and another which calls rules to build specific target files from specific source files. By convention the former is usually called the jambase and the latter the jamfile. The KJam language is designed to make it possible to move all the complexity into the jambase file, and to keep the jamfile very easy to read and edit.

KJam ships with a default jambase, integrated into the KJam binary. This built-in jambase file makes it very easy for users to manage most standard c and c++ projects using MSVC on Windows and GCC on Linux. The built-in Jambase is simple to use, and extremely flexible.

Here is an example of what a jamfile using the built-in jambase would look like:

StaticLibrary     mylib  : libsrc1.cpp libsrc2.cpp libsrc3.cpp ;
StaticExecutable  myprog : main.cpp : mylib.lib ;

One of the advantages of the built-in jambase file, is that the resulting jamfiles are completely portable between Windows and Linux. The jamfile above would build equally well under Linux or Windows.

To write jamfiles using the built-in jambase, users only need to call one of the build in rules. Most of the rules take the name of the target file as their first argument and the names of the source files as the second argument. There are rules for generating libraries and executables, either linked with static libraries or with shared libraries.

The operation of the rules can be modified by setting variables which have a special meaning to the built-in rules. For example there are variables to more directories to to search for include files, and more directories to search for header files. The names and locations of specific tools may be overriden as well.

For example, to create a jamfile that links libraries in additional directories:

LIB_DIRS = ../other ;
StaticExecutable  myprog : main.cpp : other.lib ;

Variables may also be used to pass special compile options to individual object files. For example:

StaticObject special.obj : special.cpp : /DSPECIAL_DEF ;
StaticExecutable  myprog : main.cpp special.obj ;

Most rules also allow the list of sources to include all the files in a particular directory, by listing the directory name in place of all the source files, like so:

StaticExecutable  myprog : srcdir ;

By default the built-in rules will automatically generate a 'clean' target and an 'all' target.

In order to support new languages and tools, users will need to extend the built-in jambase file, or to write their own. KJam will use the built-in jambase file unless the user specifically overrides it with the -D command line option. Using the -X and -P options users may extract the built-in jambase files, modify them and if desired reintegrate them with the KJam binary.

It is important to note that in addition to the built-in jambase file, which can be overridden with a user jambase file, many built-in rules such as the includes and depends rules are an integral part of KJam and cannot be overridden.

Variables

All Platforms

INCLUDE_DIRS

By default KJam will search for headers in the local directory and in the system header files directories. To search for headers in other directories set this variable to the list of directories to search. On windows KJam will also search for headers in any subdirectory of these directories called win32.

LIB_DIRS

By default KJam will search for libraries in the local directory and in the system library directories. To search for libraries in other directories set this variable to the list of directories to search.

CCOPTS

All elements of this variable are passed as additional options to the c compiler. The built-in rules already define most standard compilation options to control the type of build, and where to search for headers etc. Sometimes it is necessary to specify special additional compilation options. It is possible to pass additional CC options to all objects by defining this variable globally. To pass additional options to a single object file, set this variable on that object like this:

    CCOPTS on special.o = /DSPECIAL ;

On Windows KJam's built-in rules will compile c and cpp sources in batches. This may give you unexpected results when defining CCOPTS on a single target. It is safer to use the StaticObject or SharedObject rules.

CCDEFINES

All elements of this variable are passed as additional options to the c compiler as command line definitions. For MSVC this will mean that each element will be passed as a /D option, for GCC each element will be passed in as a -D option. This differs from defining CCOPTS directly. CCOPTS will set compiler exactly as they are specified, so the settings will be specific to a given compiler. CCDEFINES will set use the appropriate syntax for all compiler which support setting command line definitions, and is therefore more portable.

LINKOPTS

All elements of this variable are passed as additional options to the linker. The built-in rules already define most standard link options. Sometimes it is necessary to specify special additional link options. It is possible to pass these additional link options to all libraries and executables by defining this variable globally. To pass additional options to a single target, set this variable on that target like this:

    LINKOPTS on special.exe = -lspecial ;

These options are passed to the linker exactly as they are specified, and may not be portable from one platform to another.

BIN_DIR

By default the built-in rules will generate all target files in the defaule target directory. By default the target directory is defined as bin/$(PLATFORM)/$(CURFLAV). To build in a special non-default directory, set this variable to the desired directory. If this is set globally then all targets will be built in that given directory. To build only a certain target in a special directory, set this variables on that target like this:

   BIN_DIR on special.lib = special_dir ;

CURFLAV

KJam's built-in rules are designed to build targets in one of 4 configurations, with different amounts of debugging support. The level of debugging support is controlled by the CURFLAV variable. Normally this variable can be set as an environment variable in the user shell. On linux you must remember to 'export' the variable after setting it. Or this variable may be passed on KJam's command line as a -s option. You can also set CURFLAV in the jamfile, but this would be of limited usefulness.

The four settings are:

release All targets are compiled with maximum compiler optimizations turned on, including options which makes using a debugger very difficult, including building without a frame pointer, debugger stack frame support and without debugging symbols. When using GCC all libraries and executables are stripped. All code is compiled with NDEBUG and RELEASE defined. This is useful for compiling release builds which will be distributed to end users.
optimized All targets are compiled with most compiler optimizations, and should run at nearly full speed. Optimizations which make attaching a debugger difficult are not used. All binaries are build with complete symbol tables. Binaries produced with this level of optimization can be debugged, but due to some optimization object code will no longer match with source line for line. All code is compiled with NDEBUG defined. This is useful for internal testing of builds at nearly release speeds, but with some debugging support.
debug All targets are compiled with no compiler optimizations, and with all debugging features turned on. The resulting binaries may be significantly slower than optimized and release builds. All code is compiled with DEBUG defined. This is useful for debugging development code.
extra_debugAll targets are compiled with the same options as DEBUG, but code is compiled with DEBUG and EXTRA_DEBUG defined. This allows programmers to write certain time consuming error detection code which may cause very significant performance penalties. Such code may be necessary to address certain difficult bugs.

All targets are generated into a directory which includes the current CURFLAV, such that targets build with different levels of debugging support should never intermix. It is recommened when manually defining BIN_DIR that the new definition include $(CURFLAV) somewhere to avoid problems with mixing targets at different debugging levels.

Defining CURFLAV also causes the build rules to defined a single letter BUILD_CODE, ( r o d or x ). This BUILD_CODE may also be used to differentiate targets in places where a very short string is desired.

LEX

The name of the lexical analyzer generator to use. By default on Windows this is set to flex.exe. On Linux this is set to flex.

YACC

The name of the parser generator to use. By default on Windows this is set to bison.exe. On Linux this is set to bison.

CC_VER

This variable defines compiler version to use. On Windows this can be get to MSVC6, MSVC7, MSVC8, gcc and djgpp. On linux it is always set to gcc. KJam will attempt to discover the which compilers are installed, and will set CC_VER to an appropriate value. Setting CC_VER manually change which compiler toolset KJam will use. For versions of MSVC, this also sets the default MSVCDIR that the build rules use, and some of the compiler options.

CC_IS_MSVC

This is set to a non-null value if a version of MSVC is being used.

CC_IS_GCC

This is set to a non-null value if a version of GCC is being used.

Microsoft Visual C/C++ Specific Variables

MSVCDIR

This variable defines where the currently installed version of MSVC may be found. The build rules come predefined to look for the compiler in the default installation directories. If MSVC is installed in a different directory on your system, it may be necessary to define it here.

DLL_DEFINES

All elements of this variable are passed to MSVC as /D definitions when compiling a c or cpp file to be linked into a .dll. This variable is useful for code which is written to compile as both a .dll and as a static .lib. The definitions can be used to define special dllimport and dllexport declarations for dll interfaces.

LINKER

The name if the linker to use. By default this is set to link.exe.

Rules

StaticExecutable TARGET : SOURCES : LIBS : DLLS : TARGET_DIR

This rule will build an executable binary. It takes the following arguments:

TARGET This is the name of the executable to generate. On windows this will have .exe appeneded to it, if it doesn't already end in .exe. On Linux, the .exe will be stripped off if it exists.
SOURCES This is a list of the sources. The sources can be source files, such as .c .cpp .l .y. They can be finished object files ending in .o or .obj.

The source list may also include directories. Directories listed will cause the entire contents of the directory to be treated as source files for this rule.

All source files will be passed to the appropriate compiler. All generated files will be written to the currently defined $(BIN_DIR), and will be linked into the final target binary.

LIBS This is a list of libraries to link with. These can be .lib, .dll, .a or .so files. On Linux .a and .so files will be converted to .lib and .dll files respectively. On Windows the opposite conversion will happen.
DLLS This is a list of dynamically linked or shared libraries to link with and to copy to the target directory $(BIN_DIR). Many programs which use .dlls or shared libraries will only run if the libraries can be found in the LD_PATH. By copying these libraries locally it makes the job of testing binaries which link to non-standard libraries ( which may not be found in system library directories ) easier. The only difference between the DLLS and LIBS arguments is the copying step. As such it never makes sense to pass statically linked libraries through the DLLS argument.
TARGET_DIR All generated build files are output to $(BIN_DIR). If TARGET_DIR is set, then the final target file will be written to $(TARGET_DIR) instead of $(BIN_DIR). The same effect can be achieved by setting BIN_DIR on the target, but using this argument is often briefer and more readable. It is adivsable when setting TARGET_DIR that it include $(CURFLAV) or $(BIN_DIR) to avoid problems with different debugging levels.

SharedExecutable TARGET : SOURCES : LIBS : DLLS : TARGET_DIR

This rule will build an executable binary. On Windows it will compile all object files with $(DLL_DEFINES) defined. This is useful when building an executable that can either link statically with a static version of a library or dynamically with a .dll version. On Linux this rule is identical to StateExecutable.

StaticLibrary TARGET : SOURCES : LIBS : DLLS : TARGET_DIR

This rule will build a static library. It takes the following arguments:

TARGET This is the name of the library to generate. On windows this will have .lib appeneded to it, if it doesn't already end in .lib. On Linux, the it will have .a appeneded to it.
SOURCES This is a list of the sources. The sources can be source files, such as .c .cpp .l .y. They can be finished object files ending in .o or .obj.

The source list may also include directories. Directories listed will cause the entire contents of the directory to be treated as source files for this rule.

All source files will be passed to the appropriate compiler. All generated files will be written to the currently defined $(BIN_DIR), and will be linked into the final target library.

LIBS This is a list of libraries to link with. These can be .lib, .dll, .a or .so files. On Linux .a and .so files will be converted to .lib and .dll files respectively. On Windows the opposite conversion will happen.
DLLS This is a list of dynamically linked or shared libraries to link with and to copy to the target directory $(BIN_DIR). Many programs which use .dlls or shared libraries will only run if the libraries can be found in the LD_PATH. By copying these libraries locally it makes the job of testing binaries which link to non-standard libraries ( which may not be found in system library directories ) easier. The only difference between the DLLS and LIBS arguments is the copying step. As such it never makes sense to pass statically linked libraries through the DLLS argument.
TARGET_DIR All generated build files are output to $(BIN_DIR). If TARGET_DIR is set, then the final target file will be written to $(TARGET_DIR) instead of $(BIN_DIR). The same effect can be achieved by setting BIN_DIR on the target, but using this argument is often briefer and more readable. It is adivsable when setting TARGET_DIR that it include $(CURFLAV) or $(BIN_DIR) to avoid problems with different debugging levels.

SharedLibrary TARGET : SOURCES : LIBS : DLLS : TARGET_DIR

This rule will build a shared library, either a pair of .dll and .lib on Windows, or a .so on Linux. On Windows it will compile all object files with $(DLL_DEFINES) defined. It takes the following arguments:

TARGET This is the name of the library to generate. On windows this will generate a .dll file and a .lib export library file. On Linux, the it will generate a .so file.
SOURCES This is a list of the sources. The sources can be source files, such as .c .cpp .l .y. They can be finished object files ending in .o or .obj.

The source list may also include directories. Directories listed will cause the entire contents of the directory to be treated as source files for this rule.

All source files will be passed to the appropriate compiler. All generated files will be written to the currently defined $(BIN_DIR), and will be linked into the final target library.

LIBS This is a list of libraries to link with. These can be .lib, .dll, .a or .so files. On Linux .a and .so files will be converted to .lib and .dll files respectively. On Windows the opposite conversion will happen.
DLLS This is a list of dynamically linked or shared libraries to link with and to copy to the target directory $(BIN_DIR). Many programs which use .dlls or shared libraries will only run if the libraries can be found in the LD_PATH. By copying these libraries locally it makes the job of testing binaries which link to non-standard libraries ( which may not be found in system library directories ) easier. The only difference between the DLLS and LIBS arguments is the copying step. As such it never makes sense to pass statically linked libraries through the DLLS argument.
TARGET_DIR All generated build files are output to $(BIN_DIR). If TARGET_DIR is set, then the final target file will be written to $(TARGET_DIR) instead of $(BIN_DIR). The same effect can be achieved by setting BIN_DIR on the target, but using this argument is often briefer and more readable. It is adivsable when setting TARGET_DIR that it include $(CURFLAV) or $(BIN_DIR) to avoid problems with different debugging levels.

SharedLibraryNL TARGET : SOURCES : LIBS : DLLS : TARGET_DIR

This rule is identical to the SharedLibrary rule, except on Windows this rule does not expect that a .lib export library will be generated. This is useful for .dll files which define no exported symbols. This is sometimes the case for plug-in style shared libraries which are loaded manually by an application.

CreateDirectory TARGET

Creates all directories specified by $(TARGET). The directories are created relative to where KJam is run. $(BIN_DIR) is not appended to the directories. If a deep path is given requiring that a series of intermediate directories be created, those directories will be automatically created also.

StaticObject TARGET : SOURCE : LOC_CCOPTS

Compile a single object file from a single source file. This rule is not normally needed. Usually it is enough to list he desired source files in the source argument to one of the library or executable rules. However it is sometimes necessary to pass special options when generating a single object file. To do this, process the object file using StaticObject, and then pass the resulting object file as a source to the appropriate library or executable rule. For example:

   StaticObject special.obj : special.cpp : /DSPECIAL_DEF ;
   StaticExecutable  myprog : main.cpp special.obj ;

The object file may be specified with the extension .o or .obj, and the extension will be autmatically changed to the appropriate extension depending on the platform.

SharedObject TARGET : SOURCE : LOC_CCOPTS

This rule operates exactly like StaticObject, except that on Windows $(DLL_DEFINES) will be defined.

Group TARGET : SOURCES

Define a non-file pseudo target which will build a set of targets.

TARGET This is the name of the pseudo target to generate
SOURCES This is the list of targets which will be generated.

By default the built-in rules will always generate a group called all which will include all targets declared by all invocations of StaticExecutable, SharedExecutable, StaticLibrary and SharedLibrary. To create an all target which does not automatically include all of these targets, then manually declare the all group, like this:

Group            all : lib1 lib2 prog ;
StaticLibrary    lib1 : src/lib1 ;
StaticLibrary    lib2 : src/lib2 ;
StaticExecutable prog : src/prog : lib1.lib lib2.lib ;
StaticLibrary    special : src/special ;   // not built unless you specifically ask for it.

SubProject TARGET : DEPENDENTS : PROJTARGET

This rule will execute KJam in the directory given by TARGET. By default it will build 'all', unless it is called with PROJTARGET set to one or more targets.

TARGET The name of the directory of the subproject to build.
DEPENDENTS A list of the targets this subproject depends on.
PROJTARGET This is argument is set, then KJam will build the named target instead of all.

This rule is useful for making jamfiles which act as drivers for building the contents of other jamfiles. For example, suppose you have several libraries ( lib1, lib2, lib3 ) each in their own project directory, with their own jamfiles to build those libraries. To build those libraries and an application which depends on them:

SubProject lib1 ;
SubProject lib2 ;
SubProject lib3 ;
SubProject app1 : lib1 lib2 lib3 ;    // the application depends on 
                                      // successful building of the libs

IncludeSubProject PROJECT : JAMFILE

This rule will include the named jamfile in the names project directory to be included in the current jamfile and have its targets added to the current jamfile. All the targets of the included jamfile will be added to the dependency graph making several projects act like a single larger project.

PROJECT The name of the directory of the subproject to include.
JAMFILE The name of the jamfile in that directory to include. If this is not specified then jamfile will be used by default.

This rule is useful for making jamfiles which depend on other jamfiles being updated when they run. For example if app1 requires lib1 and lib2, then in app1's jamfile you could add the following lines:

IncludeSubProject lib1 ;
IncludeSubProject lib2 ;
StaticExecutable app1 : lib1 lib2 ;

Because the named jamfiles are included all the variables currently defined will affect the included jamfiles. So variables that should not affect the included jamfiles must either be set after the IncludeSubProject rule is invoked or unset before the rule is invoked.

Unlike the BuildSubProject rule, the targets in the included projects are updated in the update phase when all the other targets are updated. The included projects are not necessarily built completely. Rather only targets in those jamfiles which are needed by the calling jamfile are updated. The 'all' target in the included jamfiles is not updated or added to the all target of the calling jamfile. But dependencies between those jamfiles are respected.

BuildSubProject TARGET : PROJTARGET

This rule will execute KJam in the directory given by TARGET, but will execute the build command immediately. It does not check dependencies. The build command will always run. By default it will build 'all', unless it is called with PROJTARGET set to one or more targets.

TARGET The name of the directory of the subproject to build.
PROJTARGET This is argument is set, then KJam will build the named target instead of all.

This rule is useful for making jamfiles which depend on other jamfiles being updated when they run. For example if app1 requires lib1 and lib2, and you want to make sure that both those libs are fully up to date when you build app1, then in app1's jamfile you could add the following lines:

BuildSubProject lib1 ;
BuildSubProject lib2 ;
StaticExecutable app1 : lib1 lib2 ;

Clean TARGET : SOURCES

This rule will delete all the files given by $(SOURCES) when the user asks to build the pseudo target given by $(TARGET). By default the built-in rules will configure the clean rule automatically such that it is not usually necessary for users to call this rule directly. If there is a file generated that is not automitically cleaned up, then it may be necessary for the clean rule to be called manually.

This rule is also sometimes useful when multiple clean targets are desired. Suppose for example that in addition to the normal clean target, a clean_binaries target was desired which only removed the final binaries and not the intermediate object files. This could be done like so:

StaticExecutable prog1 : prog1/src ;
StaticExecutable prog2 : prog2/src ;
StaticExecutable prog3 : prog3/src ;
Clean clean_binaries : prog1 prog2 prog3 ;

Now when the user builds clean the binaries and objects are removed, and when they build clean_binaries only the final binaries are removed.

Help TARGETS : MESSAGE

This rule will schedule an action that prints an informational message about each target. If the user choses to build the help target then all the help messages are printed. This way jamfiles can print out a list of their valid targets. Targets that are defined using the built-in rules will call the Help rule themselves, and so will already print a message when the help target is built. For user defined rules, and pseudo targets, the Help rule should be called manually.

TARGET The name of the targets this help message references
MESSAGE An informational message about these targets.

For example given the following jamfile:

StaticExecutable prog1 : prog1/src ;
StaticExecutable prog2 : prog2/src ;
notfile progs ;
depends progs : prog1 prog2 ;
Help progs : "A pseudo target that builds prog1 and prog2";

Now when the user builds help KJam would print the following:

prog1 prog1.exe    StaticExecutable
prog2 prog2.exe    StaticExecutable
progs              A pseudo target that builds prog1 and prog2

Comparison with Jam/MR

KJam is modelled closely on Jam. It shares most of its language primitives, and functions in a similar manner. However, KJam is different than Jam in many important respects. Jamfiles written for Jam/MR will not run under KJam and vice-versa, though it is straight forward to adapt most files written for Jam to run under KJam.

Differences:

Added features:

Missing features:

F.A.Q.

Build Questions

Q. I'm getting the following error:

Warning: need to build target mytarget.exe but it has no commands.

What could be wrong?

A. Make sure that you have an action which takes mytarget.exe as one of the elements in the first argument passed to it. Confirm this by printing out the arguments to the action right before you call it:

rule MakeProgram TARGET
{
   echo $(TARGET) ;
   MakeProgramAction $(TARGET) ;
}

action MakeProgramAction TARGET
{%
   # ... action script goes here ...
%}

Another common error is to forget that you must pass in the argument to the action by evaluating the variable, instead of passing in the variable name directly. For example this would be wrong:

   MakeProgramAction TARGET ;     # wrong: argument is not expanded....
   MakeProgramAction $(TARGET) ;  # probably what you want. 

Q. KJam will sometimes build a target whose dependents have not changed. What could be wrong?

A. This can happen several different ways.

  1. You may have a timestamp race condition. That is a file x is depending on a target y whose timestamp changes during the build and where the first time you build y is older than x, such that a rebuild of x is not necessary. The next time, since y has been updated, x is older than y and so x gets rebuilt, even though no files have changed.
  2. You might have a rule which has more than one target. For example a certain rule has targets A and B. And you have another target C which depends on B. If you ask KJam to build A and C, B will get built as a side effect. The next time you issue the same build command, A may need to get rebuilt because during the last build the timestamp of B which it depends on was updated. To help diagnose this situation KJam will always issue a warning when a target is built as a side-effect of building another target.
  3. It can happen when files get updated as a side effect of a build. For example if you have a log file which gets updated by some targets, and is depended on by others. One run of the build updates the log file, the next time you run the build the targets which depend on the log file get rebuilt.
  4. It can happen with directory targets. The directory's timestamp will be modified every time a file is added or removed from the directory. This may cause targets which depend on this directory to need to update during a subsequent build. To solve this problem make sure to flag directory targets with the noupdate keyword. Noupdate causes KJam to ignore a target's real timestamp, and to consider the target to be old. With directories you normally only care to make sure they exist.
  5. On machines with case insensitive file systems like Windows, it can happen when you have two files that differ only in case. KJam is testing the timestamp on one file, but the file being generated by the build command is saved as a different file with different capitalization.

Q. KJam is building certain targets multiple times unncessarily. What could be wrong?

A. This can happen when in your jamfiles you invoke the same build action more than once, with the same argument list, but the arguments in a different order. For many build actions argument order doesn't matter, and so it may seem that KJam's insistence on building multiple times is unnecessary. But for some build actions order of arguments does matter. For example if the action was to concatenate files. KJam doesn't know what the build action intentions are so it assumes that it must call the action multiple times with the arguments in each given order.

To diagnose this problem, add a print statement to the build rule which is building more times than you wish and confirm that it is being called with the same arguments multiple times from different places. If this is the case, you can solve this problem one of two ways. Either sort the arguments to the build rule so that the rule is invoked in the same order each time using the T variable expansion modifier, or label the action as piecemeal, together or updated. When an action is labeled this way KJam can safely conclude that argument order is not important.

Q. Sometimes KJam seems to skip necessary targets, causing the build to fail. What could be wrong?

A. You may have a condition where a build action unexpectedly removes a target which exists at the start of the build. For example some tools, when they encounter an error will delete the output file that they are processing. Sometimes this happens when a build action purposely deletes temporary targets.

This can cause build system errors where KJam appears to skip building necessary targets, and the build fails later when that target is needed. KJam determines the total list of files which need updating in an early scanning phase. If during the later updating phase files are deleted, KJam's list of files to build is not updated. KJam will unexpectedly ( but correctly ) skip building the deleted files.

To diagnose this problem, run KJam with the -d search option and see if KJam is finding the file that it is refusing to build during its first pass. If the file is found during the scanning phase, but is missing by the time it is needed, then check your actions for commands which may be deleting this file. If the file is indeed missing at the start, but KJam is refusing to build it, then check that the dependencies between these files are set correctly using the -d fate option.

Q. Sometimes KJam seems to hang. What could be wrong?

A. The most common cause is that one of the build steps has hung for some reason.

First look at the load on the machine. If the processor load is high then probably your build steps are just taking a long time. Since KJam builds multiple steps at the same time it is possible that individual steps may seem to take longer than normal. Overall parallel builds are faster, even on a single processor machine because when one process is blocked waiting on resources like the disk, other build steps can be running.

If the processor load is low, then one of the following things may have happened:

  1. Some tools in your build process may not be designed to handle building in parallel. If they try to access a file which is locked by another process they may hang waiting for the lock to be released. Try running the build with -j 1, such that KJam will not launch any processes in parallel, or define the action with the keyword thread 1, which limit the number of threads used by that action. Consider updating the tool to make it check that all file accesses take into account the posibility of a locked file. The KJam tool itself always checks if a file handle is locked before using it and should not hang for this reason.
  2. A file that one of the build tools depends on is locked by a process which is hung. This especially common after terminating a build prematurely ( will ctrl-c or a signal ). Check the process list for zombie process which don't use cpu and also never seem to terminate. Look for instances of the compiler or linker. Some compilers spawn many processes when building and may not terminate all of them properly when terminated. If your clean step is unable to remove certain files, it is likely that there is a hung process locking them. Manually terminate such processes.
  3. If your build environment calls KJam recursively, look for instances of KJam which are hung. This can happen when the build jobs that they spawn hang. It is sometimes easier to just terminate those instances of KJam than to look for the job which they are waiting on.
  4. If the hang only happens in when using KJam in network mode, then consider that one of your build steps may be waiting for user input. Most build tools will never request interactive user input. However, there are cases where an unexpected error may cause a tool to pop up an error dialog window on a remote machine. This can happen on Windows for example if a program fails to load a required .dll. In this case check to make sure that all build tools are properly configured on all the build servers.

Q. In network mode, when I call another version of KJam to build a subdirectory it hangs. Why?

A.Check to make sure that there are enough copies of KJam server running on the network and that they are configured to run multiple jobs each. When a new copy of KJam in network mode is launched as a build action it will keep a build job locked as busy while it waits for any build jobs from its spawned version of KJam to terminate. The spawned version of KJam will wait until one of the build servers is not busy. But since the parent version is waiting for the child to finish, the server will never become available. A deadlock results.

To cure this condition make sure that there are enough build jobs available by making sure that every KJam server is configured to host more than one job ( by default KJam is configured to host up to 4 simultaneous jobs ) or that there are multiple build servers available on the network.

Q. In network mode, why is KJam unable to connect to any build servers?

A. There are many things which could cause KJam to have trouble connecting to build servers. First make sure networking between the two machines is working properly testing other network services such as pinging the remote machine or attempting a file transfer.

KJam pings machines on the local network before checking to see if there is a KJam build server running on that machine. Most systems will respond to ping packets by default. It is possible that the remote machines have been specially configured to ignore ping packets. In this case KJam would fail to connect to them.

By default KJam build servers listen for connections on port 2297. Make sure that this port is not being blocked by a firewall. It is also possible that the KJam build servers on your network have been configured to listen on a different port.

Q. Why is debugging output is badly formatted and hard to read?

A. The debugging output is formatted for shells with at least 160 columns. This can be achieved with almost all modern shells.

Windows uses cmd.exe as its default shell. This shell is extremely primitive and limited to only 80 columns. It is highly recommended that Windows users upgrade their shell to something more powerful. There are many options. Consider for example installing cygwin and running rxvt and bash.

Another option would be to pipe the debugging output to a file and review it later using a text editor.

Q. My jamfile has a build action which creates a directory, and another build action which generates its output into that directory. When I run KJam, the build action fails saying the directory is missing. Yet when I look in the file system, the directory has indeed been created. Whats wrong?

A. By default KJam is multi-threaded. This means that KJam probably ran the directory creation action and the build action at the same time. When the build action started up the directory had not yet been created. Even though the build action failed, KJam will continue to process all build actions which have already started. This is why the directory was created correctly - just too late.

To solve this problem you need to make sure that the generated file depends on the directory in which it will get written. This is not necessary in build systems where the directory can be expected to always exist before the build is run. This is true for example in build environments where the object directories are under source code control and are populated when the project is updated. For build environments where the directories are generated by KJam, you must specify a dependency using the depends keyword.

You can diagnose prolems like this by checking the dependency graph with the option -d fate. You can see exactly what threads get spawned and when by running with -d thread.

Q. KJam dependency scanning does not evaluate macros. Some of my #include statements are are controlled by conditional macros. How do I deal with this?

A. KJam's dependency scanning uses a highly optimized regular expression scanner. This runs much faster than a general purpose c or c++ parser which evaluates macros. This can result in KJam finding dependencies on header files which are not actually included because of conditional macros. These header files may not actually exist on the current platform. For example when building on linux, you may be including windows headers which don't exist on linux machines. This will cause the build to fail because the headers are not found. To solve this problem label such header targets with the nocare keyword. This will cause the build to continue even if these headers are not found ( the desired behavior ).

The combination of extra dependencies and the nocare keyword effectively solves this problem. The extra dependencies are usually benign. The conditionally included header files are typically system headers which change rarely. At worst the extra dependencies will occasionally case KJam to perform an unneccesary build step when a header is changed which is not actually included.

If a search path is set incorrectly it is possible to have missing dependencies. Suppose you have a file which is included, and is also labeled with nocare. If the search path is set incorrectly that file might not be found. Because it is labeled nocare no error is reported and the build continues as normal. This is effectively a missing dependency. When developing jamfiles which use the keyword nocare it is recommended that the build be run at least once with the -d search command line option, and the list of files not found should be carefully checked by hand to make sure only the expected files are not found.

Q. One of my build tools is unstable and will occasionally crash, causing the build action to terminate with an error. This breaks my build process. Is there anything I can do?

A. You can instruct KJam to try to run certain unstable actions more than once until they succeed with the retry action modifier. If you set retry KJam will retry the action an unlimited number of times. If you give it a number KJam will attempt to run that action the given number of times before giving up and reporting an error. This can get your build process around some occasionally unstable tools.

Be aware that KJam cannot tell when a tool terminated abnormally due to a crash versus terminating abnormally due to a normal compiler error. This means that build actions labeled retry which hit a compiler error will be run multiple times unnecessarily. For build actions which can terminate with compiler errors you may want to write your jamfiles such that the retry option is only applied to the action during special automated builds ( like in an autobuild checkin verification system ). There is no downside to setting retry on actions which should always succeed unless there is a crash.

Q. One of my build tools is unstable and will occasionally hang, causing the build to hang. This breaks my build process. Is there anything I can do?

A. You can instruct KJam to allow certain actions to run no longer than a given amount of time with the timeout modifier. timeout is normally followed by a number which is the timeout limit in seconds. The timeout limit should be fairly generous because it is possible for an action which is not hung to sometimes take longer than normal if the build system is heavily loaded or if it is waiting on a locked file system resource.

The second argument is the number of times to retry an action which has timed out. This way if an action hangs, KJam will terminate it and then try to run it again. Judicious use of timeout can greatly increase the robustness of automated build processes, even when using tools which crash or hang unexpectedly.

Q. Is there a difference between the way KJam retries action with the retry modifier and the timeout modifier?

A. The retry modifier will rerun actions a certain number of times when they fail because a tool has crashed or returned failure. The timeout modifier will retry actions which have timedout. The number of retries for these two reasons are tracked separately.

Q. Some of my SEARCH and LOCATE variables appear to be being ignored. What could be wrong?

A. First make sure that these variables are being set correctly using the -d varSet option, and that they are in fact not being used to bind your targets using the -d search option. If you see the variable being set correctly, and yet not being used by during binding then your problem may be that you are using the :F option and it is binding your target before the SEARCH and LOCATE variables are set.

Normally targets are only bound to files and directories during the binding phase. The :F option will force binding for that target to happen in the middle of the parsing phase. This can be extremely useful, but you have to be aware of where the SEARCH and LOCATE variables are being set. If they are set after the :F is evaluated then they will not be used for binding. When a later attempt at binding these targets happens, such as during the normal binding phase, KJam will see that the target is already bound and will not rebind it.

To solve this problem make sure that any :F expansion modifiers are applied only after SEARCH or LOCATE are set. In addition if SEARCH and LOCATE are set on a target those values will only get used in the binding phase. You can achieve a similar effect as LOCATE on when using the :F modifier by creating a local LOCATE variable. The :F option is always safe inside an action because they are run after the binding phase in the update phase.

Q. I though KJam was supposed to generate dependencies automatically. Why do I have to use depends and includes?

A. depends and includes can be used to generate dependencies between specific targets manually, but this is not their main use. For most build systems you will never need to do this.

depends and includes are mostly used to set up dependency relationships between categories of objects in rules. For example for telling KJam that all .obj files depend on some matching .src file. They are also used by the $(HDR_SCAN) rule to scan source files for dependencies using regular expressions. KJam uses this information so that it can figure out the dependencies between targets automatically. Once you set this up in the rules of your jambase file, you will not have to worry about setting dependencies between individual targets in your jamfiles.

Q. What is the difference between depends and includes?

A. depends A : B C creates a dependency relationship. It says that target A needs to be updated if sources B or C change.

includes A : B C says that dependents of A are also dependents of B and C. If B or C change then all targets that depend on A need to be updated. Think of it as a way of forwarding dependency relationships up the dependency graph toward the final target.

Q. I am using the built-in jambase. What must I do to create a new top level build rule like StaticLibrary? I want the clean and all targets to be updated as well.

A. You need to create the top level rule, and one action for each supported platform. For example lets create a new type called DataFile. For simplicity its action is just to copy a file from one place to another:

    rule DataFile TARGET : SOURCE
    {
       if ( $(AUTOALL) == true ) depends all : $(TARGET) ;
      
       LOCATE on $(TARGET)  = $(BIN_DIR) ;
       CreateDirectory $(TARGET_DIR) ;
       CleanTarget $(TARGET) : $(BIN_DIR) ;
   
       depends $(TARGET) : $(SOURCE) $(BIN_DIR) ;
       
       DataFileAction $(TARGET) : $(SOURCE) ;
    }
    
    action quietly DataFileAction TARGET : SOURCE
    {%
       echo "Making Data      $(SOURCE) -> $(TARGET)"
       cp $(SOURCE) $(TARGET)
    %}

Include the rule and action at the top of your jamfile, or insert it into the built-in jambase files using the -P and -X options. If the actions have any platform specific commands you will need to build one action for each supported platform and put the action in the platform specific jambase file.

Now you can use the new rule like any of the other built-in rules:

        DataFile test : test ;
    

Q. I labeled an action as piecemeal 1. I call that action with two long lists of matched sources and targets, like this:

myAction file1.obj file2.obj file3.obj : file1.src file2.src file3.src ; 

KJam is splitting up the calls to the action into three calls, one for each target, as expected. But it isn't splitting up the sources. How do I get it to do this?

A. You need to indicate to KJam that the list of sources matches the list of targets one for one by labelling the action with the keyword updated.

This keyword tells KJam that in a single call to a particular action it only needs to update the targets whose corresponding sources are out of date. It also indicates that there is a one to one mapping between the list of targets and sources. In this case when it splits up an action labeled piecemeal it will split both the targets and sources.

Without the updated keyword KJam has no way of knowing that the list of targets and sources matches one for one, so it will call the action for any target with the list of sources exactly as you passed them in.

Q. I'm getting warnings like this one:

Warning: Target main.obj will be built as a side effect of building another target.

What does this mean? Should I try to fix it?

A. This warning means that the build rules are organized in such a way that KJam finds that it must build targets which do not seem to be necessary. For example, suppose you have a program called a.exe, that depends only on a.obj, which in turn depends on a.cpp. Now suppose that the rule to make a.obj also builds another target b.obj. Since b.obj is not a dependent of a.exe, KJam would warn you that this unecessary target is will be built. This can only happen when you create build actions which have multiple simultaneous targets, either because the first argument to the action has multiple targets listed, or because that action is labeled as together.

There are three possible actions you should consider:

  1. If the side effect target really is a dependent of some other necessary target then you need to fix the dependencies. Use -d depends and -d fate to check the dependency graph.
  2. If the side effect target is not a dependent, then consider calling the build action for this target alone without other simultaneous targets, or remove the together keyword from its build action.
  3. Sometimes multiple targets must be built at the same time. For example tools like yacc, bison and flex will generate both source and header files each time they run. Or there may be performance advantages to building targets in a batch, such as when using compilers like MSVC. In these cases you may determine that the side effect warning is ok, and take no action.

Q. My build has link and clean commands that are more than 8k characters long. Does KJam support long build commands? What is the maximum length of command lines in KJam actions?

A. The simple answer, avoiding a bunch of detail, is 32k on Windows, and unlimited on most Posix platforms.

On Posix platforms like Linux and OsX, the APIs that spawn new processes don't have specific limits on the length of the command lines. How long the command lines can actually be will depend on the implementation, but in practice on these platforms command lengths are long enough as to be practically unlimited.

Windows is more limited. The Windows APIs that spawn new programs are limited to 32767 bytes. Unlike most build tools KJam uses its own internal shell. It calls these APIs directly and supports full 32k commands. This command limit applies only to the parts of a command line which actually must spawn a process to run. For example in the case of two commands connected by a pipe ( | ), each command can be 32k long. There is no limit to the number of pipes. Many commands are built in to the shell ( such as cp, mv, rm, mkdir etc. ) and since they do not spawn new processes those commands also have no length limit.

Most Windows build systems ( such as Gnu Make, Scons, Jam etc. ) spawn commands using an external shell, usually the windows shell program cmd.exe. This program is limited to a maximum command length of 8192 bytes. If KJam is specially configured to use cmd.exe ( using the JAM_SHELL variable ), then KJam will also be limited to 8k command lines.

Windows can still run 16-bit DOS programs in an emulation mode. These old programs rely on an obsolete process spawning API which limits their command lines to 127 bytes. In the unlikely event that you are using one of these old programs, and issue commands longer than 127 bytes, KJam will warn you about this issue.

Q. Does KJam support DJGPP?

A. Yes. DJGPP binaries are 16-bit DOS binaries. All 16-bit DOS binaries are limited to short command lines under 126 bytes, which is too short for many build steps. In order to work around this limitation DJGPP binaries can use 'response files'. Pipe all the arguments to the program into a file, and then pass the filename to the DJGPP binary as a single command line argument preceded by an @. The file should have a unique name, incorporating the name of the target being built for example, so that one build thread does not overwrite the response file of another. Once the command has terminated the response file should be deleted. For example a DJGPP build action might look like this:

   echo $(CXXOPTS) $(SOURCE) -o $(TARGET) > $(TARGET).opts
   $(CXX) \@$(TARGET).opts
   rm $(TARGET).opts 

Language Syntax Questions

Q. How do I concatenate strings with spaces between them?

A. Suppose you have a variable holding a set of strings:

LIST = one two three ;

To concatenate them with spaces between the elements do:

CONCAT = "$(LIST:J=' ')" ;

This will not work:

CONCAT = $(LIST:J=" ") ;

The reason it doesn't work is because the Jam language parser interprets this as two ill-formed names: $(LIST:J=" and "). Surrounding the name with quote marks causes the parser to treat it as one name.

Q. When environment variables are converted into Jam variables they are split into pieces on whitespace boundaries. This can be a problem for windows file paths with spaces in them. How do I deal with this?

A. Variables that end in path are split on ; on Windows and on : on Linux. So you can get around most of these problems by using a variable that ends in path. If this is not possible, then you can reconstruct the original path using the J variable expansion modifier, and then resplit the path on ; or : boundaries.

MSVCDIR = "$(MSVCDIR:J=' 'O=;)" ;

Q. The use of ; is not consistent. You need it in the rules, but not in the actions. Why?

A. The action scripts are not part of the KJam language. Except for one pass of variable substitution, the body of an action is passed to a command shell, and uses command shell syntax. Depending on what shell you are using the syntax would change. The default is to use the KJam built-in shell, which follows sh syntax as closely as possible. The user can plug in their own shell if they want ( dos cmd.exe, bash ). Many shells allow ; as a command separator. In this context processing of the ; is entirely under the control of the given shell.

Q. It seems like when a variable doesn't exist and you use it in an echo statement, nothing prints out.

     echo "test $(BADVAR)" ;

Prints nothing at all (not even 'test')

A. Yet the following works as you expected:

     echo test $(BADVAR) ;

Here is what is happening. When you quote a string you are creating a single token which gets expanded. When a token value is expanded, the value becomes the product of all the variable expansions that it is made of. If one of the variables is empty, then the product is empty. This useful feature is what allows you do things like:

   /I$(INCLUDE_DIRS) /L$(LIB_DIRS)

When you don't put quotes around it, then you are echoing out two separate tokens, delimited by whitespace, which is what you expect.

Q. I have a list of include directories, and I want to transform them into a series of include flags to pass to MSVC like so:

/I "C:/Program Files/one/include" /I "C:/Program Files/two/include" ;

How do I write an expression that does this?

A. It is tricky to make variable expansions that use the quote character or whitespace, because these characters have a special meaning to the KJam parser. The trick to using whitepace in an expansion is to put quote marks around the whole expansion so that the KJam language parser treats the whole thing as a single expression. The trick to using " is to escape it by prefixing a backslash. Like so:

    DIRS = "C:/Program Files/one/include" "C:/Program Files/two/include" ;
    FLAGS = "/I \"$(DIRS)\"" ;

Q. How can I make one of the arguments to a variable expansion modifier to be a literal \" ?

A. It can be tricky to make variable modifier arguments which use \ and " characters because these have a special meaning to the KJam parser, and also to the regular expression parser. Do it like this:

    # the argument to the second modifier 0, is \"
    TEST = "foo\bar" "goo\bar" ;
    print $(TEST:N="\\"0="@(\)\"") ;
    # prints foo\"bar goo\"bar

Q. Variable modifier arguments seems to treat the \ character inconsistently. Arguments to N, X and I treat \ as an escape character, and arguments to 0-9 do not. For example:

    TEST = foo.bar goo.bar ;
    print $(TEST:N="\."0="\.") ;
    # prints foo\.bar goo\.bar 
    # shouldn't this print foo.bar goo.bar?

Shouldn't the \ in the 0 argument be removed the way it is in the argument to N?

A. Some modifiers, like N, X and I interpret their arguments as regular expressions, and others like 0-9 interpret them as literal strings, which are used exactly as they are specified. Regular expressions have many characters which have special meanings, like . * [ ] ^ & etc. To use these characters as literal characters without special meaning they must be prefixed with the \ escape character. Non regular expression arguments, do no special processing of their arguments and so will treat the \ character as just another regular character. The only exception to this is the use of \ before " or '. This is interpreted by the KJam language parser as an escaped " or ' in all cases.

Q. On Windows the output of KJam is wrapped neatly to the width of the shell, but on Linux, the output seems to wrap at 80 columns no matter how wide I make my shell. Whats wrong?

A. Most shells which support resizing, like bash, set an environment variable COLUMNS to the width of the shell. The built-in KJam jamfiles use this variable to set the text wrapping feature to the shell width. If KJam is unable to find the COLUMNS environment variable, it will set the wrapping feature to a default width of 80 columns.

On Windows environment variables set in the user shell are passed to any process spawned from that shell. So on Windows, the output text wrapping feature will work automatically when you resize your shell ( if you are using a shell which supports horizontal resizing - not cmd.exe )

On Linux, by default, processes do not get local environment variables. So on Linux you have to specifically tell KJam the width of the shell. You can do this one of two ways. You can either export the COLUMNS environment variable, by typing export COLUMNS at your shell prompt, or you can pass the value of the COLUMNS environment variable to KJam on its command line ( e.g. kjam -s COLUMNS=$COLUMNS ... ). It is convenient to create an alias that does this for you automatically whenever you run KJam.