3.3.14. Master-Worker API
This section describes the master-worker interface. It covers the communication protocol of the “classic” remote Worker. Notice there are other types of workers which behave a bit differently, such as Local Worker and Latent Worker.
3.3.14.1. Connection
The interface is based on Twisted’s Perspective Broker, which operates over TCP connections.
The worker connects to the master, using the parameters supplied to buildbot-worker create-worker. It uses a reconnecting process with an exponential backoff, and will automatically reconnect on disconnection.
Once connected, the worker authenticates with the Twisted Cred (newcred) mechanism, using the username and password supplied to buildbot-worker create-worker.
The mind behind the worker is the worker bot instance (class buildbot_worker.pb.BotPb
).
On the master side, the realm is implemented by buildbot.pbmanager.Dispatcher
, which examines the username of incoming avatar requests.
There are special cases for change
and debug
, which are not discussed here.
For all other usernames, the botmaster is consulted, and if a worker with that name is configured, its buildbot.worker.Worker
instance is returned as the perspective.
3.3.14.2. Workers
At this point, the master-side Worker object has a pointer to the remote
worker-side Bot object in its self.worker
, and the worker-side Bot object has
a reference to the master-side Worker object in its self.perspective
.
Bot methods
The worker-side Bot object has the following remote methods:
remote_getCommands
Returns a dictionary for all commands the worker recognizes: the key of the dictionary is the command name and the command version is the value.
remote_setBuilderList
Given a list of builders and their build directories, ensures that those builders, and only those builders, are running. This can be called after the initial connection is established, with a new list, to add or remove builders.
This method returns a dictionary of
WorkerForBuilder
objects - see below.remote_print
Adds a message to the worker logfile.
remote_getWorkerInfo
Returns a dictionary with the contents of the worker’s
info/
directory (i.e. file name is used as key and file contents as the value). This dictionary also contains the following keys:environ
copy of the workers environment
system
OS the worker is running (extracted from Python’s
os.name
)basedir
base directory where the worker is running
numcpus
number of CPUs on the worker, either as configured or as detected (since
buildbot-worker
version 0.9.0)version
worker’s version (same as the result of
remote_getVersion
call)worker_commands
worker supported commands (same as the result of
remote_getCommands
call)remote_getVersion
Returns the worker’s version.
remote_shutdown
Shuts down the worker cleanly.
Worker methods
The master-side object has the following method:
perspective_keepalive
Does nothing - used to keep traffic flowing over the TCP connection
3.3.14.3. Setup
After the initial connection and trading of a mind (buildbot_worker.pb.BotPb
) for an avatar
(Worker), the master calls the Bot’s setBuilderList
method to set
up the proper builders on the worker side. This method returns a
reference to each of the new worker-side WorkerForBuilderPb
objects, described below. Each of these is handed to the corresponding
master-side WorkerForBuilder
object.
This immediately calls the remote setMaster
method, and then the print
method.
3.3.14.4. Pinging
To ping a remote Worker, the master calls its print
method.
3.3.14.5. Building
When a build starts, the master calls the worker’s startBuild
method.
Each BuildStep instance will subsequently call the startCommand
method,
passing a reference to itself as the stepRef
parameter. The
startCommand
method returns immediately, and the end of the command is
signalled with a call to a method on the master-side BuildStep object.
3.3.14.6. Worker For Builders
Each worker has a set of builders which can run on it. These are represented by distinct classes on the master and worker, just like the Worker and Bot objects described above.
On the worker side, builders are represented as instances of the
buildbot_worker.pb.WorkerForBuilderPb
class. On the master side, they are
represented by the buildbot.process.workerforbuilder.WorkerForBuilder
class.
The identical names are a source of confusion. The following will refer to
these as the worker-side and master-side Worker For Builder classes. Each object
keeps a reference to its opposite in self.remote
.
Worker-Side WorkerForBuilderPb
Methods
remote_setMaster
Provides a reference to the master-side Worker For Builder
remote_print
Adds a message to the worker logfile; used to check round-trip connectivity
remote_startBuild
Indicates that a build is about to start, and that any subsequent commands are part of that build
remote_startCommand
Invokes a command on the worker side
remote_interruptCommand
Interrupts the currently-running command
Master-side WorkerForBuilder
Methods
The master side does not have any remotely-callable methods.
3.3.14.7. Commands
The actual work done by the worker is represented on the master side by a
buildbot.process.remotecommand.RemoteCommand
instance.
The command instance keeps a reference to the worker-side
buildbot_worker.pb.WorkerForBuilderPb
, and calls methods like
remote_startCommand
to start new commands.
Once that method is called, the WorkerForBuilderPb
instance
keeps a reference to the command, and calls the following methods on it:
Master-Side RemoteCommand Methods
remote_update
Update information about the running command. See below for the format.
remote_complete
Signal that the command is complete, either successfully or with a Twisted failure.
3.3.14.8. Updates
Updates from the worker, sent via
remote_update
, are a list of
individual update elements. Each update element is, in turn, a list of the
form [data, 0]
, where the 0 is present for historical reasons. The data is
a dictionary, with keys describing the contents. The updates are handled by
remote_update
.
Updates with different keys can be combined into a single dictionary or delivered sequentially as list elements, at the worker’s option.
To summarize, an updates
parameter to
remote_update
might look like
this:
[
[ { 'header' : 'running command..' }, 0 ],
[ { 'stdout' : 'abcd', 'stderr' : 'local modifications' }, 0 ],
[ { 'log' : ( 'cmd.log', 'cmd invoked at 12:33 pm\n' ) }, 0 ],
[ { 'rc' : 0 }, 0 ],
]
Defined Commands
The following commands are defined on the workers.
shell
Runs a shell command on the worker. This command takes the following arguments:
command
The command to run. If this is a string, it will be passed to the system shell as a string. Otherwise, it must be a list, which will be executed directly.
workdir
The directory in which to run the command, relative to the builder dir.
env
A dictionary of environment variables to augment or replace the existing environment on the worker. In this dictionary,
PYTHONPATH
is treated specially: it should be a list of path components, rather than a string, and will be prepended to the existing Python path.
initial_stdin
A string which will be written to the command’s standard input before it is closed.
want_stdout
If false, then no updates will be sent for stdout.
want_stderr
If false, then no updates will be sent for stderr.
usePTY
If true, the command should be run with a PTY (POSIX only). This defaults to False.
not_really
If true, skip execution and return an update with rc=0.
timeout
Maximum time without output before the command is killed.
maxTime
Maximum overall time from the start before the command is killed.
logfiles
A dictionary specifying logfiles other than stdio. Keys are the logfile names, and values give the workdir-relative filename of the logfile. Alternately, a value can be a dictionary; in this case, the dictionary must have a
filename
key specifying the filename, and can also have the following keys:
follow
Only follow the file from its current end-of-file, rather that starting from the beginning.
logEnviron
If false, the command’s environment will not be logged.
The shell
command sends the following updates:
stdout
The data is a bytestring which represents a continuation of the stdout stream. Note that the bytestring boundaries are not necessarily aligned with newlines.
stderr
Similar to
stdout
, but for the error stream.
header
Similar to
stdout
, but containing data for a stream of Buildbot-specific metadata.
rc
The exit status of the command, where – in keeping with UNIX tradition – 0 indicates success and any nonzero value is considered a failure. No further updates should be sent after an
rc
.
failure_reason
Value is a string and describes additional scenarios when a process failed. The value of the
failure_reason
key can be one of the following:
timeout
if the command timed out due to time specified by themaxTime
parameter being exceeded.
timeout_without_output
if the command timed out due to time specified by thetimeout
parameter being exceeded.
log
This update contains data for a logfile other than stdio. The data associated with the update is a tuple of the log name and the data for that log. Note that non-stdio logs do not distinguish output, error, and header streams.
uploadFile
Upload a file from the worker to the master. The arguments are
workdir
Base directory for the filename, relative to the builder’s basedir.
workersrc
Name of the filename to read from, relative to the workdir.
writer
A remote reference to a writer object, described below.
maxsize
Maximum size, in bytes, of the file to write. The operation will fail if the file exceeds this size.
blocksize
The block size with which to transfer the file.
keepstamp
If true, preserve the file modified and accessed times.
The worker calls a few remote methods on the writer object. First, the
write
method is called with a bytestring containing data, until all of the
data has been transmitted. Then, the worker calls the writer’s close
,
followed (if keepstamp
is true) by a call to upload(atime, mtime)
.
This command sends rc
and stderr
updates, as defined for the shell
command.
uploadDirectory
Similar to uploadFile
, this command will upload an entire directory to the
master, in the form of a tarball. It takes the following arguments:
workdir
workersrc
writer
maxsize
blocksize
See
uploadFile
for these arguments.
compress
Compression algorithm to use – one of
None
,'bz2'
, or'gz'
.
The writer object is treated similarly to the uploadFile
command, but after
the file is closed, the worker calls the master’s unpack
method with no
arguments to extract the tarball.
This command sends rc
and stderr
updates, as defined for the shell
command.
downloadFile
This command will download a file from the master to the worker. It takes the following arguments:
workdir
Base directory for the destination filename, relative to the builder basedir.
workerdest
Filename to write to, relative to the workdir.
reader
A remote reference to a reader object, described below.
maxsize
Maximum size of the file.
blocksize
The block size with which to transfer the file.
mode
Access mode for the new file.
The reader object’s read(maxsize)
method will be called with a maximum
size, which will return no more than that number of bytes as a bytestring. At
EOF, it will return an empty string. Once EOF is received, the worker will call
the remote close
method.
This command sends rc
and stderr
updates, as defined for the shell
command.
mkdir
This command will create a directory on the worker. It will also create any intervening directories required. It takes the following argument:
dir
Directory to create.
The mkdir
command produces the same updates as shell
.
rmdir
This command will remove a directory or file on the worker. It takes the following arguments:
dir
Directory to remove.
timeout
maxTime
See
shell
above.
The rmdir
command produces the same updates as shell
.
cpdir
This command will copy a directory from one place to another place on the worker. It takes the following arguments:
fromdir
Source directory for the copy operation, relative to the builder’s basedir.
todir
Destination directory for the copy operation, relative to the builder’s basedir.
timeout
maxTime
See
shell
above.
The cpdir
command produces the same updates as shell
.
stat
This command returns status information about a file or directory. It takes a
single parameter, file
, specifying the filename relative to the builder’s
basedir.
It produces two status updates:
stat
The return value from Python’s
os.stat
.
rc
0 if the file is found, otherwise 1.
glob
This command finds all pathnames matching a specified pattern that uses shell-style wildcards.
It takes a single parameter, path
, specifying the pattern to pass to Python’s
glob.glob
function.
It produces two status updates:
files
The list of matching files returned from
glob.glob
rc
0 if the
glob.glob
does not raise exception, otherwise 1.
listdir
This command reads the directory and returns the list with directory contents. It
takes a single parameter, dir
, specifying the directory relative to the builder’s basedir.
It produces two status updates:
files
The list of files in the directory returned from
os.listdir
rc
0 if the
os.listdir
does not raise exception, otherwise 1.
rmfile
This command removes the file in the worker base directory.
It takes a single parameter, path
, specifying the file path relative to the builder’s basedir.
It produces one status update:
rc
0 if the
os.remove
does not raise exception, otherwise the corresponding errno.