Most of the work involved in a build will take place on the
buildslave. But occasionally it is useful to do some work on the
buildmaster side. The most basic way to involve the buildmaster is
simply to move a file from the slave to the master, or vice versa.
There are a pair of BuildSteps named FileUpload and
FileDownload to provide this functionality. FileUpload
moves a file up to the master, while FileDownload moves
a file down from the master.
As an example, let's assume that there is a step which produces an HTML file within the source tree that contains some sort of generated project documentation. We want to move this file to the buildmaster, into a ~/public_html directory, so it can be visible to developers. This file will wind up in the slave-side working directory under the name docs/reference.html. We want to put it into the master-side ~/public_html/ref.html.
from buildbot.steps.shell import ShellCommand
from buildbot.steps.transfer import FileUpload
f.addStep(ShellCommand(command=["make", "docs"]))
f.addStep(FileUpload(slavesrc="docs/reference.html",
masterdest="~/public_html/ref.html"))
The masterdest= argument will be passed to os.path.expanduser,
so things like “~” will be expanded properly. Non-absolute paths
will be interpreted relative to the buildmaster's base directory.
Likewise, the slavesrc= argument will be expanded and
interpreted relative to the builder's working directory.
To move a file from the master to the slave, use the
FileDownload command. For example, let's assume that some step
requires a configuration file that, for whatever reason, could not be
recorded in the source code repository or generated on the buildslave
side:
from buildbot.steps.shell import ShellCommand
from buildbot.steps.transfer import FileDownload
f.addStep(FileDownload(mastersrc="~/todays_build_config.txt",
slavedest="build_config.txt"))
f.addStep(ShellCommand(command=["make", "config"]))
Like FileUpload, the mastersrc= argument is interpreted
relative to the buildmaster's base directory, and the
slavedest= argument is relative to the builder's working
directory. If the buildslave is running in ~buildslave, and the
builder's “builddir” is something like tests-i386, then the
workdir is going to be ~buildslave/tests-i386/build, and a
slavedest= of foo/bar.html will get put in
~buildslave/tests-i386/build/foo/bar.html. Both of these commands
will create any missing intervening directories.
The maxsize= argument lets you set a maximum size for the file
to be transferred. This may help to avoid surprises: transferring a
100MB coredump when you were expecting to move a 10kB status file
might take an awfully long time. The blocksize= argument
controls how the file is sent over the network: larger blocksizes are
slightly more efficient but also consume more memory on each end, and
there is a hard-coded limit of about 640kB.
The mode= argument allows you to control the access permissions
of the target file, traditionally expressed as an octal integer. The
most common value is probably 0755, which sets the “x” executable
bit on the file (useful for shell scripts and the like). The default
value for mode= is None, which means the permission bits will
default to whatever the umask of the writing process is. The default
umask tends to be fairly restrictive, but at least on the buildslave
you can make it less restrictive with a –umask command-line option at
creation time (see Buildslave Options).
The keepstamp= argument is a boolean that, when True, forces
the modified and accessed time of the destination file to match the
times of the source file. When False, the modified and accessed times
of the destination file are set to the current time on the buildmaster.
The default is false.
To transfer complete directories from the buildslave to the master, there
is a BuildStep named DirectoryUpload. It works like FileUpload,
just for directories. As an example, let's assume a step has
generated project documentation, which consists of many files (like the output
of doxygen or epydoc). We want to move the entire documentation to the
buildmaster, into a ~/public_html/docs directory. On the slave-side
the directory can be found under docs:
from buildbot.steps.shell import ShellCommand
from buildbot.steps.transfer import DirectoryUpload
f.addStep(ShellCommand(command=["make", "docs"]))
f.addStep(DirectoryUpload(slavesrc="docs",
masterdest="~/public_html/docs"))
The DirectoryUpload step will create all necessary directories and transfers empty directories, too.
The maxsize and blocksize parameters are the same as for
FileUpload, although note that the size of the transferred data is
implementation-dependent, and probably much larger than you expect due to the
encoding used (currently tar).
The optional compress argument can be given as 'gz' or
'bz2' to compress the datastream.