.: Software Configuration Framework

After moving from autoconf to scons, the goal for a modular build system remained the same. There are various libraries and executables in the source tree, and each of them builds against some subset of the libraries and some subset of external packages. Within the source tree, the SConscript target for a program should only need to name its required packages without regard for the location of those packages. For example, a program needing the 'util' library specifies 'util' as a package requirement which causes the proper libraries and include paths to be added to the environment. The 'util' library itself may require other libraries, and those libraries should also be added to the environment automatically. A program which builds against the 'util' library should not need to know that the util library must also link against library 'tools'.

Here's the current approach. The idea is that every package, whether installed on the system or built within the source tree, defines a function for configuring the build environment, like an SCons tool. For example, the function PKG_LOG4CPP() from file 'pkg_log4cpp.py' (see below) adds the right libraries and include paths to the environment for building against the log4cpp library. The package file is placed in a standard location in the build tree, the toplevel 'config' directory.

The convention is that each PKG_ function is defined in a python file of a similar name, such as PKG_LOG4CPP defined in pkg_log4cpp.py. This makes it easy to map a package name to its source file, load it with SConscript, and apply the function to the environment. The SCons.Environment class has been extended with a Pkg_Environment class which adds methods like Packages() and Require(). The Pkg_Environment class is defined in config/atd_scons.py. The Packages() method implements the notion of mapping a required package to its config file, loading the file with SConscript, then mapping the name to the actual tool function defined in the package. The Require() method is used by instances of Pkg_Environment() to both load and apply the set of PKG_ functions required to build the source code in that environment.

For the moment the definitions for PKG_ functions not in files under config/ must appear inside the source tree in an SConscript file. Therefore the particular SConscript file must have already been read before the PKG_ function is used. Thus the rtfcommon/SConscript file must be read before any of its dependents use the PKG_RTFCOMMON function. In practice this works well though, since the lower-level libraries are loaded first, before any of their dependents.

Here is a simplified example of the pkg_log4cpp.py file, the way it looked before being converted converted to use the Package class (see Automatic Builds of External Packages):

import os
import string

def PKG_LOG4CPP(env):
    env.ParseConfig('log4cpp-config --cflags --libs')
    prefix=string.strip(os.popen("log4cpp-config --prefix").read())
    version=string.strip(os.popen("log4cpp-config --version").read())
    # only add this define once
    env.AppendUnique(CCFLAGS=["-DLOG4CPP_FIX_ERROR_COLLISION", ])
    env.AppendDoxref("log4cpp:%s/doc/log4cpp-%s/api" % (prefix, version))

Export('PKG_LOG4CPP')

The exported PKG_LOG4CPP function is like a typical SCons tool in that it modifies the environment passed into it. Other environments can add a dependency on the log4cpp library by including PKG_LOG4CPP in their list of required tools. For example, the SConscript file below (pkg_logx.py) for the logx library automatically sets up the log4cpp dependencies since the logx library uses log4cpp.

def PKG_LOGX(env):
        env.Append(LIBS=[env.GetGlobalTarget('liblogx'),])
        env.AppendDoxref("logx")
        env.Require (Split('PKG_LOG4CPP'))

Export('PKG_LOGX')

The PKG_LOGX function appends the liblogx target to the list of libraries and then Require()s the PKG_LOG4CPP function, which in turn appends the log4cpp dependencies to the environment. Any other module in the source tree which applies only PKG_LOGX() to its environment will in turn have PKG_LOG4CPP() applied automatically. If the logx library someday requires another library or other include paths, none of the other modules in the source tree which use logx will need to change their SConscript configuration.

Unlike the pkg_log4cpp.py file, liblogx is a library built _within_ the source tree, and so the actual liblogx target node is added to the LIBS construction variable. The liblogx target node must be retrieved from a global registry of such nodes maintained by the Pkg_Environment class. The PKG_LOGX function could just as well determine whether the logx library should be linked from within the source tree or from some external installation, and then modify the environment accordingly. The source module which depends upon the logx library need not change either way.

The addition of the liblogx target to LIBS needs some explanation. By default, SCons converts all of the nodes in the LIBS list to strings, then tries to convert them to standard library options by removing recognized prefixes and suffixes and then adding the appropriate library option. For example, on linux scons would remove 'lib' and '.a' from 'liblogx.a' and just add '-llogx' to the compiler command line. However, it is safer to link directly to the specific file rather than risk other installations of liblogx.a getting picked up from somewhere else on the library path. Therefore the atd_scons.py module modifies each Pkg_Environment to use a different function to assemble the library list and the library paths for the compiler command line. For one, these functions eliminate duplicates on the command line. More importantly, though, they insert library targets on the command line with their full path rather than first converting them to a string and then to a library link option. See _atd_concat in atd_scons.py.

Every time a PKG_ module is required, the module applies its additions to the environment, as well as applying each of the modules on which it depends. Thus the lists of libraries, library paths, and include paths usually contain duplicates. To control this bloat, the Pkg_Environment class overrides the construction variable settings for _LIBDIRFLAGS, _LIBFLAGS, and _CPPINCFLAGS. The new settings replace the call to _concat in SCons.Defaults with atd_scons._atd_concat. That function leaves the paths to internal target nodes untouched rather than trying to convert them into library options. Also, duplicate libraries are removed by leaving only the very last instance of a library. Likewise, duplicate library and header paths are removed except for the very first. Finally, relative header paths--the ones which point within the source tree--are always moved to the front of the include paths while external paths remain at the back. This reduces the possibility of unintentionally including a header from outside the source tree when it should have come from inside.

The Pkg_Environment class also provides a factory method Create() for creating new environments with a given name. Assigning a name to each environment allows the toplevel SConstruct file to customize a 'global' default environment as well as individually named modules. The global environment is itself like a package or tool in that it's a function which configures the Environment passed into it:

def RaddSetup(env):
   gcc_root= '/usr/local/gcc/3.2.2-redhat8.0/'
   env.Replace (CXX= [os.path.join(gcc_root,'bin', 'g++')],
                CC=  [os.path.join(gcc_root,'bin', 'gcc')],
                LINK=[os.path.join(gcc_root,'bin', 'g++')],
                RANLIB='echo skipping ranlib')
   env.Append (CCFLAGS=['-Wall', '-Wno-unused'],
               CPPPATH=['.','#'])

Other standard environments can then be built up from that one with some basic tools defined in atd_scons.py:

def DebugSetup(env):
   env.Append(CCFLAGS='-g') # ,LDFLAGS='-g')

def OptimizedSetup(env):
   env.Append(CCFLAGS=' -O2')

For RADD, the global 'env' is instantiated and then customized with a list of tools using GlobalSetup(). The GlobalSetup tools will be applied to every Pkg_Environment instantiated with the Create() factory method.

env = Pkg_Environment()
env.GlobalSetup ([RaddSetup])
Export('env')

All calls to the env.Create() method in SConscript files will get their own environment with the global setup already applied. This adds a hook to which customized configuration can be applied to a set of packages without changing all of those package's SConscript files.

# logx/SConscript
Import('env')
my_env = env.Create('logx')
my_tools = my_env.Require(Split('PKG_ACE PKG_LOG4CPP'))

def PKG_LOGX(env):
       env.Append(LIBPATH= ['#/logx',])
       env.Append(LIBS=['logx',])
       env.Apply (my_tools)

Export('PKG_LOGX')

lib = my_env.Library('logx', Split("""
ACE_Appender.cc LogLayout.cc LogAppender.cc
"""))
Default(lib)

The logx library gets its environment through the Create() factory method then applies the required tools to that environment through the Require() method.

Here's the ascope/SConscript file. It uses the global default environment settings as well as specifying the 'qt' tool by using Create() to create its environment:

# ascope/SConscript
Import('env')
my_env = env.Create ('ascope', tools = ['default', 'qt'])
my_env.Require(Split("""
PKG_QWT PKG_LOGX PKG_ACEX PKG_RDOW PKG_ELDORA   PKG_RTFCOMMON
"""))
my_env.Replace(CXXFILESUFFIX = ".cpp")

p = my_env.Program(target='ascope', source = Split("""
main.cpp beamdata_model.cpp handle_product.cpp ShmProductHandler.cpp
plot_model.cpp plot_view.cpp
ascope_win.cpp ascope.ui
"""))

Default(p)

This successfully generates and compiles all the moc and ui source files and builds ascope.

If ascope needs to be built with optimization while the rest of the tree is built with debugging, then that can be specified in the SConstruct file:

env.Customize('ascope', [OptimizedSetup])

The Customize call causes the OptimizedSetup() function to be applied to the environment returned by env.Create('ascope', ...). Someday Customize() might accept a list of package names, or a regular expression pattern, but not yet.

Config Directory

For SVN projects, such as the hiaper display program, the config directory can be checked out in a central location and shared by symbolically linking it into the top level of each working source tree, or else the config directory can be checked out from CVS directly into the working tree. In either case subversion ignores the config directory, so it must be updated explicitly.

Configuration

scons -u OPT_PREFIX=/opt

# toplevel config.py
OPT_PREFIX="/opt"

The OPT_PREFIX path is automatically included in the appropriate compiler options. Several of the smaller packages expect to be found there by default (eg, netcdf), and so they don't add any paths to the environment themselves.

The INSTALL_PREFIX option is the path prefix used by the installation methods of the Pkg_Environment class:

InstallLibrary(source)
InstallProgram(source)
InstallHeaders(subdir, source)

The INSTALL_PREFIX and OPT_PREFIX configuration options are available by default to any scons build framework using the atd_scons.py module.

It is also possible for any package script or SConscript to add more configuration options to the build framework. The atd_scons.py module creates a single instance of the SCons Options class, and that is the instance to which the OPT_PREFIX and INSTALL_PREFIX options are added. The atd_scons function Pkg_Options() returns a reference to the global Options instance, so further options can be added at any time by adding them to that instance.

For example, the spol package script adds an option SPOL_PREFIX as a spol-specific installation prefix.

# from config/pkg_spol.py

print "Adding SPOL_PREFIX to options."
atd_scons.Pkg_Options().AddOptions (
        PathOption('SPOL_PREFIX', 'Installation prefix for SPOL software.',
                   '/opt/spol'))

The option is only added once, the first time the pkg_spol.py script is loaded. After that, every call to the PKG_SPOL() function also calls Update() on the target environment to make sure the spol configuration options are setup in that environment.

# config/pkg_spol.py
# ...
        atd_scons.Pkg_Options().Update(env)

Finally, the end of the top-level SConstruct file should contain a call to the SCons Help() function using the help text from the Pkg_Options() instance. This ensures that any options added by any of the modules in the build tree will appear in the output of 'scons -h'.

Help(atd_scons.Pkg_Options().GenerateHelpText(env))

SCons Doxygen

The Apidocs() method of the Pkg_Environment class simplifies use of the doxygen builder. Given a list of sources, it generates both the Doxyfile and doxygen targets. The doxygen output is put in a subdirectory of the directory named by the APIDOCSDIR construction variable, "#apidocs" by default.

There is no alias for doxygen as there was for automake. Instead, name the top-level documentation directory as the target to update all of the documentation underneath it:

cd raddx
scons apidocs

The Doxyfile builder generates a Doxyfile using the sources as the INPUT, and it accepts several parameters to customize the configuration. The builder expects one target, the name of the doxygen config file to generate. The generated config file sets directory parameters relative to the target directory, so it expects Doxygen to run in the same directory as the config file. The documentation output will be written under that same directory.

    DOXYFILE_FILE

    The name of a doxygen config file that will be used as the basis for
    the generated configuration.  This file is copied into the destination
    and then appended according to the DOXYFILE_TEXT and DOXYFILE_DICT
    settings.

    DOXYFILE_TEXT

    This should hold verbatim Doxyfile configuration text which will be
    appended to the generated Doxyfile, thus overriding any of the default
    configuration settings.
                        
    DOXYFILE_DICT

    A dictionary of Doxygen configuration parameters which will be
    translated to Doxyfile form and included in the Doxyfile, after the
    DOXYFILE_TEXT settings.  Parameters which specify files or directory
    paths should be given relative to the source directory, then this
    target adjusts them according to the target location of the generated
    Doxyfile.

    The order of precedence is DOXYFILE_DICT, DOXYFILE_TEXT, and
    DOXYFILE_FILE.  In other words, parameter settings in DOXYFILE_DICT and
    then DOXYFILE_TEXT override all others.  A few parameters will always
    be enforced by the builder over the DOXYFILE_FILE by appending them
    after the file, such as OUTPUT_DIRECTORY, GENERATE_TAGFILE, and
    TAGFILES.  This way the template Doxyfile generated by doxygen can
    still be used as a basis, but the builder can still control where the
    output gets placed.  If any of the builder settings really need to be
    overridden, such as to put output in unusual places, then those
    settings can be placed in DOXYFILE_TEXT or DOXYFILE_DICT.

    Here are examples of some of the Doxyfile configuration parameters
    which typically need to be set for each documentation target.  Unless
    set explicitly, they are given defaults in the Doxyfile.
    
    PROJECT_NAME        Title of project, defaults to the source directory.
    PROJECT_VERSION     Version string for the project.  Defaults to 1.0

The Doxygen builder uses these environment construction variables with the given defaults:

    env['DOXYGEN'] = 'doxygen'
    env['DOXYGEN_FLAGS'] = ''
    env['DOXYGEN_COM'] = '$DOXYGEN $DOXYGEN_FLAGS $SOURCE'

Here are two typical examples for using the doxygen builders. The first sets the PROJECT_NAME by passing it in the DOXYFILE_DICT construction variable.

sources = Split("""
 Logging.cc LogLayout.cc LogAppender.cc system_error.cc
""")
headers = Split("""
 CaptureStream.h EventSource.h Logging.h Checks.h
 system_error.h
""")

doxconfig = { "PROJECT_NAME" : "logx library" }
    
env.Apidocs(sources + headers, DOXYFILE_DICT=doxconfig)

This example passes Doxyfile configuration text directly using the DOXYFILE_TEXT construction variable. The source files to be scanned by doxygen are passed to the builder as the 'source' parameter. Each source file is added to the INPUT parameter in the generated Doxyfile. This may seem more cumbersome than using Doxygen's recursive directory and file pattern features. However, strict control on the source files has several benefits. For one, it makes the dependency's explicit so that SCons can reliably recreate documentation when source files change. Also, new source files which might still be under development will not be accidentally included in the public API documentation. Likewise, source files for internal utilities and private interfaces will not be part of the documentation unless explicitly included. The Doxygen builders allow multiple variations for documentation, from internal details to the public API, and its likely that those variations work from different sets of source files.

doxyfiletext = """
PROJECT_NAME           = "DataSpace Library"

MACRO_EXPANSION        = YES
EXPAND_ONLY_PREDEF     = YES
EXPAND_AS_DEFINED = DATAMEMORYTYPETRAITS 
EXPAND_AS_DEFINED += ENTITY_OBJECT ENTITY_VISIT ENTITY_PART ENTITY_COLLECT
EXPAND_AS_DEFINED += ENTITY_BASIC
"""

env.Apidocs(sources+headers, DOXYFILE_TEXT=doxyfiletext)

As a final example, here is how the doxygen builders are used to generate the top-level documentation:

doxyconf = """
OUTPUT_DIRECTORY       = apidocs
HTML_OUTPUT            = .
RECURSIVE              = NO
SOURCE_BROWSER         = NO
ALPHABETICAL_INDEX     = NO
GENERATE_LATEX         = NO
GENERATE_RTF           = NO
GENERATE_MAN           = NO
GENERATE_XML           = NO
GENERATE_AUTOGEN_DEF   = NO
ENABLE_PREPROCESSING   = NO
CLASS_DIAGRAMS         = NO
HAVE_DOT               = NO
GENERATE_HTML          = YES
"""

df = env.Doxyfile (target="apidocs/Doxyfile",
                   source=["mainpage.dox","REQUIREMENTS","config/README"],
                   DOXYFILE_TEXT = doxyconf)
dx = env.Doxygen (target="apidocs/index.html", source=[df])

The targets are a little different in this case, since the html output (and thus the index.html file) is being placed directly into the apidocs directory rather than into a subdirectory. Therefore the two builders are setup explicitly rather than with Pkg_Environment.Apidocs().

The Doxyfile builder also takes care of cross-references between modules and between external packages. The PKG_ function for a module can append a doxygen reference to the DOXREF construction variable using the AppendDoxref() method of Pkg_Environment. If the module is internal, then it only needs to append its module name:

        env.AppendDoxref("logx")

If the package is external but has html documentation online, then the reference should include the root of the html documentation:

    env.AppendDoxref("log4cpp:%s/doc/log4cpp-%s/api" % (prefix, version))

When the Doxyfile builder parses this reference, it will automatically run the 'doxytag' program to generate a tag file from the external documentation.

Building Subsets of the Source Tree

The most obvious way is to eliminate subdirectories from the source tree. scons issues a warning for every SConscript file it cannot find, but it continues anyway. The raddx SConstruct file actually checks for the existence of each SConscript subdirectory and skips the ones that do not exist, just to avoid the warning message.

The raddx SConstruct file also supports a SUBDIRS option. The SUBDIRS option contains the specific list of subdirectories whose SConscript files should be loaded. When working on a particular subset of the raddx tree, say spol, it is possible to limit builds to the current subdirectory and any modules on which it depends. Unfortunately, for the moment those modules need to be known in advance and explicitly included in SUBDIRS. For example, if working in the dataspace directory, this command builds only the dataspace library and the logx and domx libraries which it requires:

scons -u SUBDIRS="logx domx dataspace"

The above command only builds the current directory, but it first loads and scans all of the SConscript files in the entire project. The SUBDIRS version only loads SConscript files from three subdirectories, so it is much faster.

Like other options, SUBDIRS can be specified on the command-line or in the configuration file, config.py.

Some packages define their PKG function within the SConscript file in their source directory, rather than in the config directory. To build any modules which depend on such packages, the package's subdirectory must be included in the SUBDIRS list. Otherwise the package's PKG function will never be loaded and the requirement of that package by an environment will fail.

SUBDIRS: The list of subdirectories from which to load SConscript files.
    default:
  rtfcommon logx acex domx inix dorade dbx rtf_disp
  eldora/eldora
  radd eldora rdow acex/RingBuf spol
  lidar

CVS

Mixing Subversion and CVS

env.SourceCode('logx', env.Command('logx/SConscript',
                                   None, 'cvs -d cvs:/code/cvs co logx'))
SConscript('logx/SConscript')
env.SourceCode('logx', None)

The first SourceCode call registers a source code builder to run to checkout any source code under the 'logx' directory. When the SConscript() call is seen, SCons notices that logx does not exist yet, then finds the source code builder registered for that path and runs it. Next the SConscript file can be read from the logx directory. The second SourceCode() call removes the source code registry for the logx directory since it is no longer needed. Without that, SCons tries to check out header files from the logx directory that it thinks could exist there.

Automatic Builds of External Packages

The Package base class holds the canonical name for the package, a list of key files contained in the package archive, the SCons actions for building the package, the targets created when the package is installed, and a default name for the package's archive file. For example, the netcdf package has the name NETCDF. Unpacking the netcdf archive creates the file "src/INSTALL". When built and installed, the netcdf package installs the header file "$OPT_PREFIX/include/netcdf.h", so that is one of the SCons targets for package's builder. All of these steps together are a cascade of dependencies in SCons. When SCons discovers source code with a dependency on the netcdf.h header file (through the source scanner), SCons will initiate the installation of the netcdf package.

In the usual case, a package like netcdf would be required by a SConscript file somewhere, using the Pkg_Environment.Require() method. Then the package's PKG_ function takes care of setting up the environment to build against the particular package. The pkg_ file in the config directory implements the PKG_ function by calling into a singleton subclassed from atd_package.Package. The subclass contains the specifics for the package, but most of the work can be passed to the Package class through the checkBuild() method. The checkBuild() method first checks whether all of the install targets exist. If so, then nothing needs to be added to the environment to build the package. However, if something is missing, then Package calls the setupBuild() method. That method calls generate(), which actually creates a builder for the package archive and adds it to the environment. The rest of setupBuild() adds the builder instances, one for unpacking the archive and another for building it. The generated builder is specific to the package subclass. It uses an emitter to emit that package's install files as the builder targets.

First, it establishes the targets that the package installs, usually header files and libraries.

# Note that netcdf.inc has been left out of this list, since this
# current setup does not install it.

netcdf_headers = Split("""
ncvalues.h netcdf.h netcdf.hh netcdfcpp.h
""")

headers = [ os.path.join("$OPT_PREFIX","include",f)
            for f in netcdf_headers ]

# We extend the standard netcdf installation slightly by also copying
# the headers into a netcdf subdirectory, so headers can be qualified
# with a netcdf/ path when included.  Aeros does that, for example.

headers.extend ([ os.path.join("$OPT_PREFIX","include","netcdf",f)
                  for f in netcdf_headers ])
libs = Split("""
$OPT_PREFIX/lib/libnetcdf.a
$OPT_PREFIX/lib/libnetcdf_c++.a
""")

Then there is the list of actions used to build and install the netcdf source tree, after it has been extracted from the compressed archive file. Note the chdir.MkdirIfMissing method comes from the config/chdir.py module, while Copy is one of the standard SCons actions. The copy step is only necessary to copy the netcdf headers into their own netcdf subdirectory, a convention the HIAPER display project has started using so that netcdf include directives can be qualified with the "netcdf/" path.

netcdf_actions = [
    "./configure --prefix=$OPT_PREFIX FC= CC=gcc CXX=g++",
    "make",
    "make install",
    chdir.MkdirIfMissing("$OPT_PREFIX/include/netcdf") ] + [
    Copy("$OPT_PREFIX/include/netcdf", h) for h in
    [ os.path.join("$OPT_PREFIX","include",h2) for h2 in netcdf_headers ]
    ]

Here is the actual definition of the NetcdfPackage subclass. The Package base class is passed enough information in the constructor to know how to find, extract, build, and install the package. Because it knows the list of installed files, the Package class also knows how to check whether netcdf appears to have installed or not yet. The setupBuild() and require() methods extend the base class by setting up the environment as needed specifically for the netcdf library. For example, there are two library dependencies, the C and C++ libraries. Also, when building the netcdf package explicitly, then the specific library targets can be added to the LIBS construction variable by referring to the 'libnetcdfpp' and 'libnetcdf' global targets. Otherwise, as is usually the case for external libraries which have already been installed, only the library base names are added to LIBS.

class NetcdfPackage(Package):

    def __init__(self):
        Package.__init__(self, "NETCDF", "src/INSTALL",
                         netcdf_actions, libs + headers,
                         default_package_file = "netcdf-3.6.0-p1.tar.gz")

    def setupBuild(self, env):
        installs = Package.setupBuild(self, env)
        env.AddGlobalTarget('libnetcdf', installs[0])
        env.AddGlobalTarget('libnetcdfpp', installs[1])

    def require(self, env):
        "Need to add both c and c++ libraries to the environment."
        Package.checkBuild(self, env)
        prefix = env['OPT_PREFIX']
        env.AppendUnique(CPPPATH=[os.path.join(prefix,'include'),])
        if self.building:
            env.Append(LIBS=env.GetGlobalTarget('libnetcdfpp'))
            env.Append(LIBS=env.GetGlobalTarget('libnetcdf'))
        else:
            env.Append(LIBS=['netcdf_c++', 'netcdf'])

The 'default_package_file' argument to the constructor is used by the Package base class to find the external package. If the netcdf package needs to be installed, then the Package base class looks for the name of package file in the SCons construction variable NETCDF_PACKAGE_FILE, and it looks for that file in the path given by the construction variable PACKAGE_DIRECTORY. The default package directory is "/net/ftp/pub/archive/aeros/packages", and each package usually passes a default package file name to the Package constructor. However, both of these can be overridden for a particular project in the global setup function. For example, the raddx project could add these lines to the RaddSetup() function in the raddx/SConstruct file:

        env['PACKAGE_DIRECTORY'] = "/net/ftp/pub/archive/spol/packages"
        env['NETCDF_PACKAGE_FILE'] = "netcdf-3.6.1.tar.gz"

Eventually a URL will be allowed for the package directory, and the python url module can be used to download the archive file from anywhere on the net.

This final code in pkg_netcdf.py creates the singleton NetcdfPackage instance, and uses that instance to implement the require() functionality of the exported PKG_NETCDF() function.

netcdf_package = NetcdfPackage()

def PKG_NETCDF(env):
    netcdf_package.require(env)

Export("PKG_NETCDF")

Any SConscript file which requires PKG_NETCDF will have its environment setup with the correct library and include paths for netcdf, as usual. However, if package builds are enabled, then the netcdf_package instance will check if the netcdf requirements are already installed or not. If not, then it will add to the environment the targets and builders necessary to automatically extract and install the netcdf source.

Package builds can be enabled or disabled through the atd_package module's sole scons option: packagebuilds. That option can be set to enable, disable, or force. By default automatic package builds are disabled. They can be enabled with the enable option. The force option does not force every package to be rebuilt, instead it forces the nodes for building the package to be added to the environment, whether the package needed to be built or not. This may not result in any attempt to build the package if the built package is already up-to-date.

So far these packages have been updated to use the Package framework and allow automatic installs:

Initial Setup

   ln -s ../raddx/config .

The source modules will be subdirectories of the top-level pickler. If there are modules, such as libraries or programs, which will be shared with other projects, also link them into the pickler directory:

   ln -s ../raddx/logx .
   ln -s ../mapr2/rtf_threads .

New modules specific to this project need to have their own SConscript files, and the toplevel SConstruct file needs to load them with SConscript(). If a module will be used elsewhere in the source tree, then it should define a PKG_ function for dependent modules to apply to get the right libraries and include paths added to the environment.

History

The result is basically very similar to the framework I put in place for the MAPR software tree, which due to the KDevelop IDE used automake and autoconf and started out divided into several smaller projects. That work in turn led me to borrow some things from the KDE development framework. I wished there were something better to base this on than the autoconf tools, but I did not know what that would be. Qmake seems to be difficult to extend to a larger project or with new make variables and rules. Perhaps JAM or ANT would have been better, but autoconf seemed to be the more widely used, and there are huge projects like KDE and GCC rich with working examples from which techniques can be pulled. Given that recent projects like RADD have the chance to run on both Windows and UNIX, it would be useful if the build system supported compiles on both platforms. I don't know how/if automake/autoconf could handle that.

Comparison Between SCons and Autoconf

So the advantage of scons is that all of the same funcationality can be self-contained in a single, portable scripting language--python. The build dependencies are not described in yet another static syntax, but instead they are assembled through calls to a standard and mostly intuitive scripting API built within python. The assembly of the dependencies and build rules on any particular build invocation can be very dynamic and runtime configurable. No preformatting of Makefiles or preconfiguration of the build environment is required. I think this makes scons slower by pushing all of that processing back to each and every build run, but it is also what allows scons to be more thorough and dynamic. Ant got the portability idea right by encapsulating build rules and their specification with XML and Java, but I think it went wrong by adding yet another static dependency format in the use of XML. I think Ant would have been better to follow the scons model of using Java for both build phases: first describe the rules and dependencies for a build environment using portable Java scripting like beanscript, then run the build engine on that environment using Java. The idea is the same: use a portable but powerful runtime like Java or python to provide both dynamic dependencies and rule execution. Describe the dependencies and write the rules in the same language.

On the other hand, the self-containment of scons lends to some of its current drawbacks, from my perspective. For example, automatic dependency scanning is built-in but primitive. It scans source files for include dependencies using regular expression matching, so it fails to catch dependencies which only exist with certain preprocessor definitions. However, that could be become more accurate as scons develops. Also, scons is very strict about thoroughly describing and checking all dependencies and only executing build actions when dependencies are outdated. This makes it harder to use scons to run auxiliary build actions which do not have such clear cut notions of dependencies, such as generating documentation (doxygen), cleaning, and installing. Scons does scale well to managing the dependencies of a large multi-directory project like the raddx tree, but only in dependency complexity and not in performance. I'm still trying to get it to scale back down to building small subsets of the source tree separately and quickly, which was always easy and fast with hierarchical makefiles.

For both autoconf and scons I've ended up grafting some of my own desired extensions on top of them to support more modular build environments. One way or another I'll be able to get scons to do what I want. So as for me, I don't see any reason to go back to autoconf or Makefiles.

Software Configuration Framework

Table of Contents

Overview

SCons Implementation