Command-line Tool

This section describes command-line tools available after buildbot installation. Since version 0.8 the one-for-all buildbot command-line tool was divided into two parts namely buildbot and buildslave. The last one was separated from main command-line tool to minimize dependencies required for running a buildslave while leaving all other functions to buildbot tool.

Every command-line tool has a list of global options and a set of commands which have their own options. One can run these tools in the following way:

buildbot [global options] command [command options]
buildslave [global options] command [command options]

The buildbot command is used on the master, while buildlsave is used on the slave. Global options are the same for both tools which perform the following actions:

--help	Print general help about available commands and global options and exit. All subsequent arguments are ignored.
--verbose	Set verbose output.
--version	Print current buildbot version and exit. All subsequent arguments are ignored.

You can get help on any command by specifying --help as a command option:

buildbot @var{command} --help

You can also use manual pages for buildbot and buildslave for quick reference on command-line options.

The remainder of this section describes each buildbot command. See Command Line Index for a full list.

buildbot

The buildbot command-line tool can be used to start or stop a buildmaster or buildbot, and to interact with a running buildmaster. Some of its subcommands are intended for buildmaster admins, while some are for developers who are editing the code that the buildbot is monitoring.

Administrator Tools

The following buildbot sub-commands are intended for buildmaster administrators:

create-master

This creates a new directory and populates it with files that allow it to be used as a buildmaster's base directory.

You will usually want to use the -r option to create a relocatable buildbot.tac. This allows you to move the master directory without editing this file.

buildbot create-master -r {BASEDIR}

start

This starts a buildmaster which was already created in the given base directory. The daemon is launched in the background, with events logged to a file named twistd.log.

stop

This terminates the daemon (either buildmaster or buildslave) running in the given directory.

buildbot stop {BASEDIR}

sighup

This sends a SIGHUP to the buildmaster running in the given directory, which causes it to re-read its master.cfg file.

buildbot sighup {BASEDIR}

Developer Tools

These tools are provided for use by the developers who are working on the code that the buildbot is monitoring.

statuslog

buildbot statuslog --master {MASTERHOST}:{PORT}

This command starts a simple text-based status client, one which just prints out a new line each time an event occurs on the buildmaster.

The --master option provides the location of the buildbot.status.client.PBListener status port, used to deliver build information to realtime status clients. The option is always in the form of a string, with hostname and port number separated by a colon (HOSTNAME:PORTNUM). Note that this port is not the same as the slaveport (although a future version may allow the same port number to be used for both purposes). If you get an error message to the effect of Failure: twisted.cred.error.UnauthorizedLogin:, this may indicate that you are connecting to the slaveport rather than a PBListener port.

The --master option can also be provided by the masterstatus name in .buildbot/options (see .buildbot config directory).

statusgui

If you have set up a PBListener, you will be able to monitor your Buildbot using a simple Gtk+ application invoked with the buildbot statusgui command:

buildbot statusgui --master {MASTERHOST}:{PORT}

This command starts a simple Gtk+-based status client, which contains a few boxes for each Builder that change color as events occur. It uses the same --master argument and masterstatus option as the buildbot statuslog command (statuslog).

try

This lets a developer to ask the question What would happen if I committed this patch right now?. It runs the unit test suite (across multiple build platforms) on the developer's current code, allowing them to make sure they will not break the tree when they finally commit their changes.

The buildbot try command is meant to be run from within a developer's local tree, and starts by figuring out the base revision of that tree (what revision was current the last time the tree was updated), and a patch that can be applied to that revision of the tree to make it match the developer's copy. This (revision, patch) pair is then sent to the buildmaster, which runs a build with that SourceStamp. If you want, the tool will emit status messages as the builds run, and will not terminate until the first failure has been detected (or the last success).

There is an alternate form which accepts a pre-made patch file (typically the output of a command like svn diff). This --diff form does not require a local tree to run from. See try --diff concerning the --diff command option.

For this command to work, several pieces must be in place: the Try_Jobdir or :Try_Userpass, as well as some client-side configuration.

Locating the master

The try command needs to be told how to connect to the try scheduler, and must know which of the authentication approaches described above is in use by the buildmaster. You specify the approach by using --connect=ssh or --connect=pb (or try_connect = 'ssh' or try_connect = 'pb' in .buildbot/options).

For the PB approach, the command must be given a --master argument (in the form HOST:PORT) that points to TCP port that you picked in the Try_Userpass scheduler. It also takes a --username and --passwd pair of arguments that match one of the entries in the buildmaster's userpass list. These arguments can also be provided as try_master, try_username, and try_password entries in the .buildbot/options file.

For the SSH approach, the command must be given --host and --username, to get to the buildmaster host. It must also be given --jobdir, which points to the inlet directory configured above. The jobdir can be relative to the user's home directory, but most of the time you will use an explicit path like ~buildbot/project/trydir. These arguments can be provided in .buildbot/options as try_host, try_username, try_password, and try_jobdir.

In addition, the SSH approach needs to connect to a PBListener status port, so it can retrieve and report the results of the build (the PB approach uses the existing connection to retrieve status information, so this step is not necessary). This requires a --masterstatus argument, or a try_masterstatus entry in .buildbot/options, in the form of a HOSTNAME:PORT string.

The following command line arguments are deprecated, but retained for backward compatibility:

--tryhost	is replaced by --host
--trydir	is replaced by --jobdir
--master	is replaced by --masterstatus

Likewise, the following .buildbot/options file entries are deprecated, but retained for backward compatibility:

• try_dir is replaced by try_jobdir
• masterstatus is replaced by try_masterstatus

Choosing the Builders

A trial build is performed on multiple Builders at the same time, and the developer gets to choose which Builders are used (limited to a set selected by the buildmaster admin with the TryScheduler's builderNames= argument). The set you choose will depend upon what your goals are: if you are concerned about cross-platform compatibility, you should use multiple Builders, one from each platform of interest. You might use just one builder if that platform has libraries or other facilities that allow better test coverage than what you can accomplish on your own machine, or faster test runs.

The set of Builders to use can be specified with multiple --builder arguments on the command line. It can also be specified with a single try_builders option in .buildbot/options that uses a list of strings to specify all the Builder names:

try_builders = ["full-OSX", "full-win32", "full-linux"]

If you are using the PB approach, you can get the names of the builders that are configured for the try scheduler using the get-builder-names argument:

buildbot try --get-builder-names --connect=pb --master=... --username=... --passwd=...

Specifying the VC system

The try command also needs to know how to take the developer's current tree and extract the (revision, patch) source-stamp pair. Each VC system uses a different process, so you start by telling the try command which VC system you are using, with an argument like --vc=cvs or --vc=git. This can also be provided as try_vc in .buildbot/options.

The following names are recognized: bzr cvs darcs hg git mtn` p4 svn

Finding the top of the tree

Some VC systems (notably CVS and SVN) track each directory more-or-less independently, which means the try command needs to move up to the top of the project tree before it will be able to construct a proper full-tree patch. To accomplish this, the try command will crawl up through the parent directories until it finds a marker file. The default name for this marker file is .buildbot-top, so when you are using CVS or SVN you should touch .buildbot-top from the top of your tree before running buildbot try. Alternatively, you can use a filename like ChangeLog or README, since many projects put one of these files in their top-most directory (and nowhere else). To set this filename, use --topfile=ChangeLog, or set it in the options file with try_topfile = 'ChangeLog'.

You can also manually set the top of the tree with --topdir=~/trees/mytree, or try_topdir = '~/trees/mytree'. If you use try_topdir, in a .buildbot/options file, you will need a separate options file for each tree you use, so it may be more convenient to use the try_topfile approach instead.

Other VC systems which work on full projects instead of individual directories (darcs, mercurial, git, monotone) do not require try to know the top directory, so the --try-topfile and --try-topdir arguments will be ignored.

If the try command cannot find the top directory, it will abort with an error message.

The following command line arguments are deprecated, but retained for backward compatibility:

• --try-topdir is replaced by --topdir
• --try-topfile is replaced by --topfile

Determining the branch name

Some VC systems record the branch information in a way that try can locate it. For the others, if you are using something other than the default branch, you will have to tell the buildbot which branch your tree is using. You can do this with either the --branch argument, or a try_branch entry in the .buildbot/options file.

Determining the revision and patch

Each VC system has a separate approach for determining the tree's base revision and computing a patch.

CVS

try pretends that the tree is up to date. It converts the current time into a -D time specification, uses it as the base revision, and computes the diff between the upstream tree as of that point in time versus the current contents. This works, more or less, but requires that the local clock be in reasonably good sync with the repository.

SVN

try does a svn status -u to find the latest repository revision number (emitted on the last line in the Status against revision: NN message). It then performs an svn diff -rNN to find out how your tree differs from the repository version, and sends the resulting patch to the buildmaster. If your tree is not up to date, this will result in the try tree being created with the latest revision, then backwards patches applied to bring it back to the version you actually checked out (plus your actual code changes), but this will still result in the correct tree being used for the build.

bzr

try does a bzr revision-info to find the base revision, then a bzr diff -r$base.. to obtain the patch.

Mercurial

hg identify --debug emits the full revision id (as opposed to the common 12-char truncated) which is a SHA1 hash of the current revision's contents. This is used as the base revision. hg diff then provides the patch relative to that revision. For try to work, your working directory must only have patches that are available from the same remotely-available repository that the build process' source.Mercurial will use.

Perforce

try does a p4 changes -m1 ... to determine the latest changelist and implicitly assumes that the local tree is synched to this revision. This is followed by a p4 diff -du to obtain the patch. A p4 patch differs sligtly from a normal diff. It contains full depot paths and must be converted to paths relative to the branch top. To convert the following restriction is imposed. The p4base (see P4Source) is assumed to be //depot

Darcs

try does a darcs changes --context to find the list of all patches back to and including the last tag that was made. This text file (plus the location of a repository that contains all these patches) is sufficient to re-create the tree. Therefore the contents of this context file are the revision stamp for a Darcs-controlled source tree. It then does a darcs diff -u to compute the patch relative to that revision.

Git

git branch -v lists all the branches available in the local repository along with the revision ID it points to and a short summary of the last commit. The line containing the currently checked out branch begins with *  (star and space) while all the others start with    (two spaces). try scans for this line and extracts the branch name and revision from it. Then it generates a diff against the base revision.

Monotone

mtn automate get_base_revision_id emits the full revision id which is a SHA1 hash of the current revision's contents. This is used as the base revision. mtn diff then provides the patch relative to that revision. For try to work, your working directory must only have patches that are available from the same remotely-available repository that the build process' source.Monotone will use.

patch information

You can provide the --who=dev to designate who is running the try build. This will add the dev to the Reason field on the try build's status web page. You can also set try_who = dev in the .buildbot/options file. Note that --who=dev will not work on version 0.8.3 or earlier masters.

Similarly, --comment=COMMENT will specify the comment for the patch, which is also displayed in the patch information. The corresponding config-file option is try_comment.

Waiting for results

If you provide the --wait option (or try_wait = True in .buildbot/options), the buildbot try command will wait until your changes have either been proven good or bad before exiting. Unless you use the --quiet option (or try_quiet=True), it will emit a progress message every 60 seconds until the builds have completed.

try --diff

Sometimes you might have a patch from someone else that you want to submit to the buildbot. For example, a user may have created a patch to fix some specific bug and sent it to you by email. You've inspected the patch and suspect that it might do the job (and have at least confirmed that it doesn't do anything evil). Now you want to test it out.

One approach would be to check out a new local tree, apply the patch, run your local tests, then use buildbot try to run the tests on other platforms. An alternate approach is to use the buildbot try --diff form to have the buildbot test the patch without using a local tree.

This form takes a --diff argument which points to a file that contains the patch you want to apply. By default this patch will be applied to the TRUNK revision, but if you give the optional --baserev argument, a tree of the given revision will be used as a starting point instead of TRUNK.

You can also use buildbot try --diff=- to read the patch from stdin.

Each patch has a patchlevel associated with it. This indicates the number of slashes (and preceding pathnames) that should be stripped before applying the diff. This exactly corresponds to the -p or --strip argument to the patch utility. By default buildbot try --diff uses a patchlevel of 0, but you can override this with the -p argument.

When you use --diff, you do not need to use any of the other options that relate to a local tree, specifically --vc, --try-topfile, or --try-topdir. These options will be ignored. Of course you must still specify how to get to the buildmaster (with --connect, --tryhost, etc).

Other Tools

These tools are generally used by buildmaster administrators.

sendchange

This command is used to tell the buildmaster about source changes. It is intended to be used from within a commit script, installed on the VC server. It requires that you have a PBChangeSource (PBChangeSource) running in the buildmaster (by being set in c['change_source']).

buildbot sendchange --master {MASTERHOST}:{PORT} --auth {USER}:{PASS}
        --who {USER} {FILENAMES..}

The auth option specifies the credentials to use to connect to the master, in the form user:pass. If the password is omitted, then sendchange will prompt for it. If both are omitted, the old default (username "change" and password "changepw") will be used. Note that this password is well-known, and should not be used on an internet-accessible port.

The master and username arguments can also be given in the options file (see .buildbot config directory). There are other (optional) arguments which can influence the Change that gets submitted:

--branch	(or option branch) This provides the (string) branch specifier. If omitted, it defaults to None, indicating the default branch. All files included in this Change must be on the same branch.
--category	(or option category) This provides the (string) category specifier. If omitted, it defaults to None, indicating no category. The category property can be used by Schedulers to filter what changes they listen to.
--project	(or option project) This provides the (string) project to which this change applies, and defaults to ''. The project can be used by schedulers to decide which builders should respond to a particular change.
--repository	(or option repository) This provides the repository from which this change came, and defaults to ''.
--revision	This provides a revision specifier, appropriate to the VC system in use.
--revision_file
	This provides a filename which will be opened and the contents used as the revision specifier. This is specifically for Darcs, which uses the output of darcs changes --context as a revision specifier. This context file can be a couple of kilobytes long, spanning a couple lines per patch, and would be a hassle to pass as a command-line argument.
--property	This parameter is used to set a property on the Change generated by sendchange. Properties are specified as a name:value pair, separated by a colon. You may specify many properties by passing this parameter multiple times.
--comments	This provides the change comments as a single argument. You may want to use --logfile instead.
--logfile	This instructs the tool to read the change comments from the given file. If you use - as the filename, the tool will read the change comments from stdin.
--encoding	Specifies the character encoding for all other parameters, defaulting to 'utf8'.
--vc	Specifies which VC system the Change is coming from, one of: cvs, svn, darcs, hg, bzr, git, mtn, or p4. Defaults to None.

debugclient

buildbot debugclient --master {MASTERHOST}:{PORT} --passwd {DEBUGPW}

This launches a small Gtk+/Glade-based debug tool, connecting to the buildmaster's debug port. This debug port shares the same port number as the slaveport (see Setting the PB Port for Slaves), but the debugPort is only enabled if you set a debug password in the buildmaster's config file (see Debug Options). The --passwd option must match the c['debugPassword'] value.

--master can also be provided in .debug/options by the master key. --passwd can be provided by the debugPassword key. See .buildbot config directory.

The Connect button must be pressed before any of the other buttons will be active. This establishes the connection to the buildmaster. The other sections of the tool are as follows:

Reload .cfg

Forces the buildmaster to reload its master.cfg file. This is equivalent to sending a SIGHUP to the buildmaster, but can be done remotely through the debug port. Note that it is a good idea to be watching the buildmaster's twistd.log as you reload the config file, as any errors which are detected in the config file will be announced there.

Rebuild .py

(not yet implemented). The idea here is to use Twisted's rebuild facilities to replace the buildmaster's running code with a new version. Even if this worked, it would only be used by buildbot developers.

poke IRC

This locates a words.IRC status target and causes it to emit a message on all the channels to which it is currently connected. This was used to debug a problem in which the buildmaster lost the connection to the IRC server and did not attempt to reconnect.

Commit

This allows you to inject a Change, just as if a real one had been delivered by whatever VC hook you are using. You can set the name of the committed file and the name of the user who is doing the commit. Optionally, you can also set a revision for the change. If the revision you provide looks like a number, it will be sent as an integer, otherwise it will be sent as a string.

Force Build

This lets you force a Builder (selected by name) to start a build of the current source tree.

Currently

(obsolete). This was used to manually set the status of the given Builder, but the status-assignment code was changed in an incompatible way and these buttons are no longer meaningful.

user

Note that in order to use this command, you need to configure a CommandlineUserManager instance in your master.cfg file, which is explained in Users Options.

This command allows you to manage users in buildbot's database. No extra requirements are needed to use this command, aside from the Buildmaster running. For details on how Buildbot manages users, see Users.

--master	The user command can be run virtually anywhere provided a location of the running buildmaster. The master argument is of the form {MASTERHOST}:{PORT}.
--username	PB connection authentication that should match the arguments to CommandlineUserManager.
--passwd	PB connection authentication that should match the arguments to CommandlineUserManager.
--op	There are four supported values for the op argument: add, update, remove, and show. Each are described in full in the following sections.
--bb_username	Used with the update option, this sets the user's username for web authentication in the database. It requires bb_password to be set along with it.
--bb_password	Also used with the update option, this sets the password portion of a user's web authentication credentials into the database. The password is first encrypted prior to storage for security reasons.
--ids	When working with users, you need to be able to refer to them by unique identifiers to find particular users in the database. The ids option lets you specify a comma separated list of these identifiers for use with the user command. The ids option is used only when using remove or show.
--info	Users are known in buildbot as a collection of attributes tied together by some unique identifier (see Users). These attributes are specified in the form {TYPE}={VALUE} when using the info option. These {TYPE}={VALUE} pairs are specified in a comma separated list, so for example: --info=svn=jschmo,git='Joe Schmo <joe@schmo.com>' The info option can be specified multiple times in the user command, as each specified option will be interpreted as a new user. Note that info is only used with add or with update, and whenever you use update you need to specify the identifier of the user you want to update. This is done by prepending the info arguments with {ID:}. If we were to update 'jschmo' from the previous example, it would look like this: --info=jschmo:git='Joseph Schmo <joe@schmo.com>'

Note that --master, --username, --passwd, and --op are always required to issue the user command.

The --master, --username, and --passwd options can be specified in the option file with keywords user_master, user_username, and user_passwd, respectively. If user_master is not specified, then master from the options file will be used instead.

Below are examples of how each command should look. Whenever a user command is successful, results will be shown to whoever issued the command.

For add:

buildbot user --master={MASTERHOST} --op=add \
        --username={USER} --passwd={USERPW} \
        --info={TYPE}={VALUE},...

For update:

buildbot user --master={MASTERHOST} --op=update \
        --username={USER} --passwd={USERPW} \
        --info={ID}:{TYPE}={VALUE},...

For remove:

buildbot user --master={MASTERHOST} --op=remove \
        --username={USER} --passwd={USERPW} \
        --ids={ID1},{ID2},...

For show:

buildbot user --master={MASTERHOST} --op=show \
        --username={USER} --passwd={USERPW} \
        --ids={ID1},{ID2},...

A note on update: when updating the bb_username and bb_password, the info doesn't need to have additional {TYPE}={VALUE} pairs to update and can just take the {ID} portion.

.buildbot config directory

Many of the buildbot tools must be told how to contact the buildmaster that they interact with. This specification can be provided as a command-line argument, but most of the time it will be easier to set them in an options file. The buildbot command will look for a special directory named .buildbot, starting from the current directory (where the command was run) and crawling upwards, eventually looking in the user's home directory. It will look for a file named options in this directory, and will evaluate it as a python script, looking for certain names to be set. You can just put simple name = 'value' pairs in this file to set the options.

For a description of the names used in this file, please see the documentation for the individual buildbot sub-commands. The following is a brief sample of what this file's contents could be.

# for status-reading tools
masterstatus = 'buildbot.example.org:12345'
# for 'sendchange' or the debug port
master = 'buildbot.example.org:18990'
debugPassword = 'eiv7Po'

Note carefully that the names in the options file usually do not match the command-line option name.

masterstatus

Equivalent to --master for statuslog and statusgui, this gives the location of the client.PBListener status port.

master

Equivalent to --master for debugclient and sendchange. This option is used for two purposes. It is the location of the debugPort for debugclient and the location of the pb.PBChangeSource for `sendchange. Generally these are the same port.

debugPassword

Equivalent to --passwd for debugclient.

Important

This value must match the value of debugPassword, used to protect the debug port, for the debugclient command.

username

Equivalent to --username for the sendchange command.

branch

Equivalent to --branch for the sendchange command.

category

Equivalent to --category for the sendchange command.

try_connect

Equivalent to --connect, this specifies how the try command should deliver its request to the buildmaster. The currently accepted values are ssh and pb.

try_builders

Equivalent to --builders, specifies which builders should be used for the try build.

try_vc

Equivalent to --vc for try, this specifies the version control system being used.

try_branch

Equivlanent to --branch, this indicates that the current tree is on a non-trunk branch.

try_topdir

try_topfile

Use try_topdir, equivalent to --try-topdir, to explicitly indicate the top of your working tree, or try_topfile, equivalent to --try-topfile to name a file that will only be found in that top-most directory.

try_host

try_username

try_dir

When try_connect is ssh, the command will use try_host for --tryhost, try_username for --username, and try_dir for --trydir. Apologies for the confusing presence and absence of 'try'.

try_username

try_password

try_master

Similarly, when try_connect is pb, the command will pay attention to try_username for --username, try_password for --passwd, and try_master for --master.

try_wait

masterstatus

try_wait and masterstatus (equivalent to --wait and master, respectively) are used to ask the try command to wait for the requested build to complete.

buildslave

buildslave command-line tool is used for buildslave management only and does not provide any additional functionality. One can create, start, stop and restart the buildslave.

create-slave

This creates a new directory and populates it with files that let it be used as a buildslave's base directory. You must provide several arguments, which are used to create the initial buildbot.tac file.

The -r option is advisable here, just like for create-master.

buildslave create-slave -r {BASEDIR} {MASTERHOST}:{PORT} {SLAVENAME} {PASSWORD}

The create-slave options are described in Buildslave Options.

start

This starts a buildslave which was already created in the given base directory. The daemon is launched in the background, with events logged to a file named twistd.log.

buildbot start BASEDIR

stop

This terminates the daemon buildslave running in the given directory.

buildbot stop BASEDIR