Class Source

Parameters:

• workdir (string) - local directory (relative to the Builder's root) where the tree should be placed
• mode (string) - the kind of VC operation that is desired:
  • 'update': specifies that the checkout/update should be performed directly into the workdir. Each build is performed in the same directory, allowing for incremental builds. This minimizes disk space, bandwidth, and CPU time. However, it may encounter problems if the build process does not handle dependencies properly (if you must sometimes do a 'clean build' to make sure everything gets compiled), or if source files are deleted but generated files can influence test behavior (e.g. python's .pyc files), or when source directories are deleted but generated files prevent CVS from removing them. When used with a patched checkout, from a previous buildbot try for instance, it will try to "revert" the changes first and will do a clobber if it is unable to get a clean checkout. The behavior is SCM-dependent.
  • 'copy': specifies that the source-controlled workspace should be maintained in a separate directory (called the 'copydir'), using checkout or update as necessary. For each build, a new workdir is created with a copy of the source tree (rm -rf workdir; cp -R -P -p copydir workdir). This doubles the disk space required, but keeps the bandwidth low (update instead of a full checkout). A full 'clean' build is performed each time. This avoids any generated-file build problems, but is still occasionally vulnerable to problems such as a CVS repository being manually rearranged (causing CVS errors on update) which are not an issue with a full checkout.
  • 'clobber': specifies that the working directory should be deleted each time, necessitating a full checkout for each build. This insures a clean build off a complete checkout, avoiding any of the problems described above, but is bandwidth intensive, as the whole source tree must be pulled down for each build.
  • 'export': is like 'clobber', except that e.g. the 'cvs export' command is used to create the working directory. This command removes all VC metadata files (the CVS/.svn/{arch} directories) from the tree, which is sometimes useful for creating source tarballs (to avoid including the metadata in the tar file). Not all VC systems support export.
• alwaysUseLatest (boolean) - whether to always update to the most recent available sources for this build.

  Normally the Source step asks its Build for a list of all Changes that are supposed to go into the build, then computes a 'source stamp' (revision number or timestamp) that will cause exactly that set of changes to be present in the checked out tree. This is turned into, e.g., 'cvs update -D timestamp', or 'svn update -r revnum'. If alwaysUseLatest=True, bypass this computation and always update to the latest available sources for each build.

  The source stamp helps avoid a race condition in which someone commits a change after the master has decided to start a build but before the slave finishes checking out the sources. At best this results in a build which contains more changes than the buildmaster thinks it has (possibly resulting in the wrong person taking the blame for any problems that result), at worst is can result in an incoherent set of sources (splitting a non-atomic commit) which may not build at all.

• retry (tuple of ints (delay, repeats) (or None)) - if provided, VC update failures are re-attempted up to REPEATS times, with DELAY seconds between each attempt. Some users have slaves with poor connectivity to their VC repository, and they say that up to 80% of their build failures are due to transient network failures that could be handled by simply retrying a couple times.

Overrides: process.buildstep.BuildStep.__init__

Overrides: process.buildstep.BuildStep.setDefaultWorkdir

Overrides: process.buildstep.LoggingBuildStep.describe

Each subclass must implement this method to do something more precise than -rHEAD every time. For version control systems that use repository-wide change numbers (SVN, P4), this can simply take the maximum such number from all the changes involved in this build. For systems that do not (CVS), it needs to create a timestamp based upon the latest Change, the Build's treeStableTimer, and an optional self.checkoutDelay value.

Begin the step. Override this method and add code to do local processing, fire off remote commands, etc.

To spawn a command in the buildslave, create a RemoteCommand instance and run it with self.runCommand:

 c = RemoteCommandFoo(args)
 d = self.runCommand(c)
 d.addCallback(self.fooDone).addErrback(self.failed)

As the step runs, it should send status information to the BuildStepStatus:

 self.step_status.setText(['compile', 'failed'])
 self.step_status.setText2(['4', 'warnings'])

To have some code parse stdio (or other log stream) in realtime, add a LogObserver subclass. This observer can use self.step.setProgress() to provide better progress notification to the step.:

 self.addLogObserver('stdio', MyLogObserver())

To add a LogFile, use self.addLog. Make sure it gets closed when it finishes. When giving a Logfile to a RemoteShellCommand, just ask it to close the log when the command completes:

 log = self.addLog('output')
 cmd = RemoteShellCommand(args)
 cmd.useLog(log, closeWhenFinished=True)

You can also create complete Logfiles with generated text in a single step:

 self.addCompleteLog('warnings', text)

When the step is done, it should call self.finished(result). 'result' will be provided to the buildbot.process.base.Build, and should be one of the constants defined above: SUCCESS, WARNINGS, FAILURE, or SKIPPED.

If the step encounters an exception, it should call self.failed(why). 'why' should be a Failure object. This automatically fails the whole build with an exception. It is a good idea to add self.failed as an errback to any Deferreds you might obtain.

If the step decides it does not need to be run, start() can return the constant SKIPPED. This fires the callback immediately: it is not necessary to call .finished yourself. This can also indicate to the status-reporting mechanism that this step should not be displayed.

A step can be configured to only run under certain conditions. To do this, set the step's doStepIf to a boolean value, or to a function that returns a boolean value. If the value or function result is False, then the step will return SKIPPED without doing anything, otherwise the step will be executed normally. If you set doStepIf to a function, that function should accept one parameter, which will be the Step object itself.

Overrides: process.buildstep.BuildStep.start

(inherited documentation)

This is a general-purpose hook method for subclasses. It will be called after the remote command has finished, but before any of the other hook functions are called.

Overrides: process.buildstep.LoggingBuildStep.commandComplete

(inherited documentation)