This lets a developer to ask the question “What would happen if I committed this patch right now?”. It runs the unit test suite (across multiple build platforms) on the developer's current code, allowing them to make sure they will not break the tree when they finally commit their changes.
The buildbot try command is meant to be run from within a developer's local tree, and starts by figuring out the base revision of that tree (what revision was current the last time the tree was updated), and a patch that can be applied to that revision of the tree to make it match the developer's copy. This (revision, patch) pair is then sent to the buildmaster, which runs a build with that SourceStamp. If you want, the tool will emit status messages as the builds run, and will not terminate until the first failure has been detected (or the last success).
There is an alternate form which accepts a pre-made patch file (typically the output of a command like 'svn diff'). This “–diff” form does not require a local tree to run from. See See try –diff.
For this command to work, several pieces must be in place:
The buildmaster must have a scheduler.Try instance in
the config file's c['schedulers'] list. This lets the
administrator control who may initiate these “trial” builds, which
branches are eligible for trial builds, and which Builders should be
used for them.
The TryScheduler has various means to accept build requests:
all of them enforce more security than the usual buildmaster ports do.
Any source code being built can be used to compromise the buildslave
accounts, but in general that code must be checked out from the VC
repository first, so only people with commit privileges can get
control of the buildslaves. The usual force-build control channels can
waste buildslave time but do not allow arbitrary commands to be
executed by people who don't have those commit privileges. However,
the source code patch that is provided with the trial build does not
have to go through the VC system first, so it is important to make
sure these builds cannot be abused by a non-committer to acquire as
much control over the buildslaves as a committer has. Ideally, only
developers who have commit access to the VC repository would be able
to start trial builds, but unfortunately the buildmaster does not, in
general, have access to VC system's user list.
As a result, the TryScheduler requires a bit more
configuration. There are currently two ways to set this up:
buildbot try command creates
a special file containing the source stamp information and drops it in
the jobdir, just like a standard maildir. When the buildmaster notices
the new file, it unpacks the information inside and starts the builds.
The config file entries used by 'buildbot try' either specify a local queuedir (for which write and mv are used) or a remote one (using scp and ssh).
The advantage of this scheme is that it is quite secure, the disadvantage is that it requires fiddling outside the buildmaster config (to set the permissions on the jobdir correctly). If the buildmaster machine happens to also house the VC repository, then it can be fairly easy to keep the VC userlist in sync with the trial-build userlist. If they are on different machines, this will be much more of a hassle. It may also involve granting developer accounts on a machine that would not otherwise require them.
To implement this, the buildslave invokes 'ssh -l username host
buildbot tryserver ARGS', passing the patch contents over stdin. The
arguments must include the inlet directory and the revision
information.
buildbot try, their machine connects to the
buildmaster via PB and authenticates themselves using that username
and password, then sends a PB command to start the trial build.
The advantage of this scheme is that the entire configuration is performed inside the buildmaster's config file. The disadvantages are that it is less secure (while the “cred” authentication system does not expose the password in plaintext over the wire, it does not offer most of the other security properties that SSH does). In addition, the buildmaster admin is responsible for maintaining the username/password list, adding and deleting entries as developers come and go.
For example, to set up the “jobdir” style of trial build, using a
command queue directory of MASTERDIR/jobdir (and assuming that
all your project developers were members of the developers unix
group), you would first create that directory (with mkdir
MASTERDIR/jobdir MASTERDIR/jobdir/new MASTERDIR/jobdir/cur
MASTERDIR/jobdir/tmp; chgrp developers MASTERDIR/jobdir
MASTERDIR/jobdir/*; chmod g+rwx,o-rwx MASTERDIR/jobdir
MASTERDIR/jobdir/*), and then use the following scheduler in the
buildmaster's config file:
from buildbot.scheduler import Try_Jobdir
s = Try_Jobdir("try1", ["full-linux", "full-netbsd", "full-OSX"],
jobdir="jobdir")
c['schedulers'] = [s]
Note that you must create the jobdir before telling the buildmaster to use this configuration, otherwise you will get an error. Also remember that the buildmaster must be able to read and write to the jobdir as well. Be sure to watch the twistd.log file (see Logfiles) as you start using the jobdir, to make sure the buildmaster is happy with it.
To use the username/password form of authentication, create a
Try_Userpass instance instead. It takes the same
builderNames argument as the Try_Jobdir form, but
accepts an addtional port argument (to specify the TCP port to
listen on) and a userpass list of username/password pairs to
accept. Remember to use good passwords for this: the security of the
buildslave accounts depends upon it:
from buildbot.scheduler import Try_Userpass
s = Try_Userpass("try2", ["full-linux", "full-netbsd", "full-OSX"],
port=8031, userpass=[("alice","pw1"), ("bob", "pw2")] )
c['schedulers'] = [s]
Like most places in the buildbot, the port argument takes a
strports specification. See twisted.application.strports for
details.
The try command needs to be told how to connect to the
TryScheduler, and must know which of the authentication
approaches described above is in use by the buildmaster. You specify
the approach by using --connect=ssh or --connect=pb
(or try_connect = 'ssh' or try_connect = 'pb' in
.buildbot/options).
For the PB approach, the command must be given a --master
argument (in the form HOST:PORT) that points to TCP port that you
picked in the Try_Userpass scheduler. It also takes a
--username and --passwd pair of arguments that match
one of the entries in the buildmaster's userpass list. These
arguments can also be provided as try_master,
try_username, and try_password entries in the
.buildbot/options file.
For the SSH approach, the command must be given --tryhost,
--username, and optionally --password (TODO:
really?) to get to the buildmaster host. It must also be given
--trydir, which points to the inlet directory configured
above. The trydir can be relative to the user's home directory, but
most of the time you will use an explicit path like
~buildbot/project/trydir. These arguments can be provided in
.buildbot/options as try_host, try_username,
try_password, and try_dir.
In addition, the SSH approach needs to connect to a PBListener status
port, so it can retrieve and report the results of the build (the PB
approach uses the existing connection to retrieve status information,
so this step is not necessary). This requires a --master
argument, or a masterstatus entry in .buildbot/options,
in the form of a HOSTNAME:PORT string.
A trial build is performed on multiple Builders at the same time, and
the developer gets to choose which Builders are used (limited to a set
selected by the buildmaster admin with the TryScheduler's
builderNames= argument). The set you choose will depend upon
what your goals are: if you are concerned about cross-platform
compatibility, you should use multiple Builders, one from each
platform of interest. You might use just one builder if that platform
has libraries or other facilities that allow better test coverage than
what you can accomplish on your own machine, or faster test runs.
The set of Builders to use can be specified with multiple
--builder arguments on the command line. It can also be
specified with a single try_builders option in
.buildbot/options that uses a list of strings to specify all
the Builder names:
try_builders = ["full-OSX", "full-win32", "full-linux"]
If you are using the PB approach, you can get the names of the builders
that are configured for the try scheduler using the get-builder-names
argument:
buildbot try --get-builder-names --connect=pb --master=... --username=... --passwd=...
The try command also needs to know how to take the
developer's current tree and extract the (revision, patch)
source-stamp pair. Each VC system uses a different process, so you
start by telling the try command which VC system you are
using, with an argument like --vc=cvs or --vc=tla.
This can also be provided as try_vc in
.buildbot/options.
The following names are recognized: cvs svn baz
tla bzr hg darcs git p4
Some VC systems (notably CVS and SVN) track each directory
more-or-less independently, which means the try command
needs to move up to the top of the project tree before it will be able
to construct a proper full-tree patch. To accomplish this, the
try command will crawl up through the parent directories
until it finds a marker file. The default name for this marker file is
.buildbot-top, so when you are using CVS or SVN you should
touch .buildbot-top from the top of your tree before running
buildbot try. Alternatively, you can use a filename like
ChangeLog or README, since many projects put one of
these files in their top-most directory (and nowhere else). To set
this filename, use --try-topfile=ChangeLog, or set it in the
options file with try_topfile = 'ChangeLog'.
You can also manually set the top of the tree with
--try-topdir=~/trees/mytree, or try_topdir =
'~/trees/mytree'. If you use try_topdir, in a
.buildbot/options file, you will need a separate options file
for each tree you use, so it may be more convenient to use the
try_topfile approach instead.
Other VC systems which work on full projects instead of individual directories (tla, baz, darcs, monotone, mercurial, git) do not require try to know the top directory, so the --try-topfile and --try-topdir arguments will be ignored.
If the try command cannot find the top directory, it will abort with an error message.
Some VC systems record the branch information in a way that “try” can locate it, in particular Arch (both tla and baz). For the others, if you are using something other than the default branch, you will have to tell the buildbot which branch your tree is using. You can do this with either the --branch argument, or a try_branch entry in the .buildbot/options file.
Each VC system has a separate approach for determining the tree's base revision and computing a patch.
CVS-D time specification, uses it as the base
revision, and computes the diff between the upstream tree as of that
point in time versus the current contents. This works, more or less,
but requires that the local clock be in reasonably good sync with the
repository.
SVNsvn status -u to find the latest
repository revision number (emitted on the last line in the “Status
against revision: NN” message). It then performs an svn diff
-rNN to find out how your tree differs from the repository version,
and sends the resulting patch to the buildmaster. If your tree is not
up to date, this will result in the “try” tree being created with
the latest revision, then backwards patches applied to bring it
“back” to the version you actually checked out (plus your actual
code changes), but this will still result in the correct tree being
used for the build.
bazbaz tree-id to determine the
fully-qualified version and patch identifier for the tree
(ARCHIVE/VERSION–patch-NN), and uses the VERSION–patch-NN component
as the base revision. It then does a baz diff to obtain the
patch.
tlatla tree-version to get the
fully-qualified version identifier (ARCHIVE/VERSION), then takes the
first line of tla logs --reverse to figure out the base
revision. Then it does tla changes --diffs to obtain the patch.
bzrbzr revision-info to find the base revision,
then a bzr diff -r$base.. to obtain the patch.
Mercurialhg identify --debug emits the full revision id (as opposed to
the common 12-char truncated) which is a SHA1 hash of the current
revision's contents. This is used as the base revision.
hg diff then provides the patch relative to that
revision. For try to work, your working directory must only
have patches that are available from the same remotely-available
repository that the build process' source.Mercurial will use.
Perforcep4 changes -m1 ... to determine the latest
changelist and implicitly assumes that the local tree is synched to this
revision. This is followed by a p4 diff -du to obtain the patch.
A p4 patch differs sligtly from a normal diff. It contains full depot
paths and must be converted to paths relative to the branch top. To convert
the following restriction is imposed. The p4base (see see P4Source)
is assumed to be //depot
Darcsdarcs changes --context to find the list
of all patches back to and including the last tag that was made. This text
file (plus the location of a repository that contains all these
patches) is sufficient to re-create the tree. Therefore the contents
of this “context” file are the revision stamp for a
Darcs-controlled source tree. It then does a darcs diff
-u to compute the patch relative to that revision.
Gitgit branch -v lists all the branches available in the local
repository along with the revision ID it points to and a short summary
of the last commit. The line containing the currently checked out
branch begins with '* ' (star and space) while all the others start
with ' ' (two spaces). try scans for this line and extracts
the branch name and revision from it. Then it generates a diff against
the base revision.
If you provide the --wait option (or try_wait = True
in .buildbot/options), the buildbot try command will
wait until your changes have either been proven good or bad before
exiting. Unless you use the --quiet option (or
try_quiet=True), it will emit a progress message every 60
seconds until the builds have completed.