Next: , Previous: Using Build Properties, Up: Build Steps

4.12.3 Source Checkout

The first step of any build is typically to acquire the source code from which the build will be performed. There are several classes to handle this, one for each of the different source control system that Buildbot knows about. For a description of how Buildbot treats source control in general, see Version Control Systems.

All source checkout steps accept some common parameters to control how they get the sources and where they should be placed. The remaining per-VC-system parameters are mostly to specify where exactly the sources are coming from.

mode
a string describing the kind of VC operation that is desired. Defaults to update.
update
specifies that the CVS checkout/update should be performed directly into the workdir. Each build is performed in the same directory, allowing for incremental builds. This minimizes disk space, bandwidth, and CPU time. However, it may encounter problems if the build process does not handle dependencies properly (sometimes you must do a “clean build” to make sure everything gets compiled), or if source files are deleted but generated files can influence test behavior (e.g. python's .pyc files), or when source directories are deleted but generated files prevent CVS from removing them. Builds ought to be correct regardless of whether they are done “from scratch” or incrementally, but it is useful to test both kinds: this mode exercises the incremental-build style.
copy
specifies that the CVS workspace should be maintained in a separate directory (called the 'copydir'), using checkout or update as necessary. For each build, a new workdir is created with a copy of the source tree (rm -rf workdir; cp -r copydir workdir). This doubles the disk space required, but keeps the bandwidth low (update instead of a full checkout). A full 'clean' build is performed each time. This avoids any generated-file build problems, but is still occasionally vulnerable to CVS problems such as a repository being manually rearranged, causing CVS errors on update which are not an issue with a full checkout.
clobber
specifies that the working directory should be deleted each time, necessitating a full checkout for each build. This insures a clean build off a complete checkout, avoiding any of the problems described above. This mode exercises the “from-scratch” build style.
export
this is like clobber, except that the 'cvs export' command is used to create the working directory. This command removes all CVS metadata files (the CVS/ directories) from the tree, which is sometimes useful for creating source tarballs (to avoid including the metadata in the tar file).

workdir
like all Steps, this indicates the directory where the build will take place. Source Steps are special in that they perform some operations outside of the workdir (like creating the workdir itself).
alwaysUseLatest
if True, bypass the usual “update to the last Change” behavior, and always update to the latest changes instead.
retry
If set, this specifies a tuple of (delay, repeats) which means that when a full VC checkout fails, it should be retried up to repeats times, waiting delay seconds between attempts. If you don't provide this, it defaults to None, which means VC operations should not be retried. This is provided to make life easier for buildslaves which are stuck behind poor network connections.
repository
The name of this parameter might vary depending on the Source step you are running. The concept explained here is common to all steps and applies to repourl as well as for baseURL (when applicable). Buildbot, now being aware of the repository name via the ChangeSource step might in some cases not need the repository URL. There are multiple way to pass it through to this step, corresponding to the type of the parameter given to this step:
None
In the case where no parameter is specified, the repository URL will be taken directly from the Change property. This value should be used if your ChangeSource step has all the information about how to reach the Change.
string
The parameter might be a string. In this case, this string will be used as the full repository URL. The value coming from the ChangeSource step will be ignored.
format string
If the parameter is a string containing %s, then the repository property from the Change will be substituted in place of the %s. This is usefull when the ChangeSource step knows where the repository resides locally, but doesn't know the scheme used to access it. For instance, ssh://server/%s makes sense if the repository property is the local path of the repository.
dict
In this case, the repository URL will be the value indexed by the repository property in the dict given as parameter.
callable
The callable given as parameter will take the repository property from the Change and its return value will be used as repository URL.

timeout
Specifies the timeout for slave-side operations, in seconds. If your repositories are particularly large, then you may need to increase this value from its default of 1200 (20 minutes).

Use of WithProperties with string, dict and callable is supported.

Note that this is quite similar to the mechanism used by the WebStatus for the changecommentlink, projects or repositories parameter.

My habit as a developer is to do a cvs update and make each morning. Problems can occur, either because of bad code being checked in, or by incomplete dependencies causing a partial rebuild to fail where a complete from-scratch build might succeed. A quick Builder which emulates this incremental-build behavior would use the mode='update' setting.

On the other hand, other kinds of dependency problems can cause a clean build to fail where a partial build might succeed. This frequently results from a link step that depends upon an object file that was removed from a later version of the tree: in the partial tree, the object file is still around (even though the Makefiles no longer know how to create it).

“official” builds (traceable builds performed from a known set of source revisions) are always done as clean builds, to make sure it is not influenced by any uncontrolled factors (like leftover files from a previous build). A “full” Builder which behaves this way would want to use the mode='clobber' setting.

Each VC system has a corresponding source checkout class: their arguments are described on the following pages.