buildbot.changes.svnpoller.SVNPoller is a ChangeSource
which periodically polls a Subversion repository for new revisions, by running the
log command in a subshell. It can watch a single branch or multiple
SVNPoller accepts the following arguments:
http://divmod.org/svn/Divmod/, or even
file:///home/svn/Repository/ProjectA/branches/1.5/. This must include the access scheme, the location of the repository (both the hostname for remote ones, and any additional directory names necessary to get to the repository), and the sub-path within the repository's virtual filesystem for the project and branch of interest.
SVNPoller will only pay attention to files inside the
subdirectory specified by the complete svnurl.
SVNPoller. This function must accept a single string and return a two-entry tuple. There are a few utility functions in
buildbot.changes.svnpollerthat can be used as a
split_filefunction; see below for details.
The default value always returns (None, path), which indicates that all files are on the trunk.
SVNPoller can override the
method instead of using the
SVNPoller. This will then be set in any changes generated by the
SVNPoller, and can be used in a Change Filter for triggering particular builders.
--userargument will be added to all
svncommands. Use this if you have to authenticate to the svn server before you can do
svnuser, this will cause a
--passwordargument to be passed to all svn commands.
SVNPollerasks for the last HISTMAX changes and looks through them for any ones it does not already know about. If more than HISTMAX revisions have been committed since the last poll, older changes will be silently ignored. Larger values of histmax will cause more time and memory to be consumed on each poll attempt.
histmaxdefaults to 100.
svnexecutable to use. If subversion is installed in a weird place on your system (outside of the buildmaster's
$PATH), use this to tell
SVNPollerwhere to find it. The default value of “svn” will almost always be sufficient.
'http://myserver/websvn/revision.php?rev=%s'could be used to cause revision links to be created to a websvn repository viewer.
Each source file that is tracked by a Subversion repository has a
fully-qualified SVN URL in the following form:
(REPOURL)(PROJECT-plus-BRANCH)(FILEPATH). When you create the
SVNPoller, you give it a
svnurl value that includes all
of the REPOURL and possibly some portion of the PROJECT-plus-BRANCH
SVNPoller is responsible for producing Changes that
contain a branch name and a FILEPATH (which is relative to the top of
a checked-out tree). The details of how these strings are split up
depend upon how your repository names its branches.
One common layout is to have all the various projects that share a repository get a single top-level directory each. Then under a given project's directory, you get two subdirectories, one named “trunk” and another named “branches”. Under “branches” you have a bunch of other directories, one per branch, with names like “1.5.x” and “testing”. It is also common to see directories like “tags” and “releases” next to “branches” and “trunk”.
For example, the Twisted project has a subversion server on “svn.twistedmatrix.com” that hosts several sub-projects. The repository is available through a SCHEME of “svn:”. The primary sub-project is Twisted, of course, with a repository root of “svn://svn.twistedmatrix.com/svn/Twisted”. Another sub-project is Informant, with a root of “svn://svn.twistedmatrix.com/svn/Informant”, etc. Inside any checked-out Twisted tree, there is a file named bin/trial (which is used to run unit test suites).
The trunk for Twisted is in
“svn://svn.twistedmatrix.com/svn/Twisted/trunk”, and the
fully-qualified SVN URL for the trunk version of
trial would be
“svn://svn.twistedmatrix.com/svn/Twisted/trunk/bin/trial”. The same
SVNURL for that file on a branch named “1.5.x” would be
To set up a
SVNPoller that watches the Twisted trunk (and
nothing else), we would use the following:
from buildbot.changes.svnpoller import SVNPoller c['change_source'] = SVNPoller("svn://svn.twistedmatrix.com/svn/Twisted/trunk")
In this case, every Change that our
SVNPoller produces will
.branch=None, to indicate that the Change is on the trunk.
No other sub-projects or branches will be tracked.
If we want our ChangeSource to follow multiple branches, we have to do
two things. First we have to change our
svnurl= argument to
watch more than just “.../Twisted/trunk”. We will set it to
“.../Twisted” so that we'll see both the trunk and all the branches.
Second, we have to tell
SVNPoller how to split the
(PROJECT-plus-BRANCH)(FILEPATH) strings it gets from the repository
out into (BRANCH) and (FILEPATH) pairs.
We do the latter by providing a “split_file” function. This function
is responsible for splitting something like
filepath=”bin/trial”. This function is always given a string
that names a file relative to the subdirectory pointed to by the
svnurl= argument. It is expected to return a
(BRANCHNAME, FILEPATH) tuple (in which FILEPATH is relative to the
branch indicated), or None to indicate that the file is outside any
project of interest.
(note that we want to see “branches/1.5.x” rather than just “1.5.x” because when we perform the SVN checkout, we will probably append the branch name to the baseURL, which requires that we keep the “branches” component in there. Other VC schemes use a different approach towards branches and may not require this artifact.)
If your repository uses this same PROJECT/BRANCH/FILEPATH naming scheme, the following function will work:
def split_file_branches(path): pieces = path.split('/') if pieces == 'trunk': return (None, '/'.join(pieces[1:])) elif pieces == 'branches': return ('/'.join(pieces[0:2]), '/'.join(pieces[2:])) else: return None
This function is provided as
buildbot.changes.svnpoller.split_file_branches for your
convenience. So to have our Twisted-watching
multiple branches, we would use this:
from buildbot.changes.svnpoller import SVNPoller, split_file_branches c['change_source'] = SVNPoller("svn://svn.twistedmatrix.com/svn/Twisted", split_file=split_file_branches)
Changes for all sorts of branches (with names like “branches/1.5.x”, and None to indicate the trunk) will be delivered to the Schedulers. Each Scheduler is then free to use or ignore each branch as it sees fit.
Another common way to organize a Subversion repository is to put the branch name at the top, and the projects underneath. This is especially frequent when there are a number of related sub-projects that all get released in a group.
For example, Divmod.org hosts a project named “Nevow” as well as one named “Quotient”. In a checked-out Nevow tree there is a directory named “formless” that contains a python source file named “webform.py”. This repository is accessible via webdav (and thus uses an “http:” scheme) through the divmod.org hostname. There are many branches in this repository, and they use a (BRANCHNAME)/(PROJECT) naming policy.
The fully-qualified SVN URL for the trunk version of webform.py is
You can do an
svn co with that URL and get a copy of the latest
version. The 1.5.x branch version of this file would have a URL of
The whole Nevow trunk would be checked out with
http://divmod.org/svn/Divmod/trunk/Nevow, while the Quotient
trunk would be checked out using
Now suppose we want to have an
SVNPoller that only cares about
the Nevow trunk. This case looks just like the PROJECT/BRANCH layout
from buildbot.changes.svnpoller import SVNPoller c['change_source'] = SVNPoller("http://divmod.org/svn/Divmod/trunk/Nevow")
But what happens when we want to track multiple Nevow branches? We
have to point our
svnurl= high enough to see all those
branches, but we also don't want to include Quotient changes (since
we're only building Nevow). To accomplish this, we must rely upon the
split_file function to help us tell the difference between
files that belong to Nevow and those that belong to Quotient, as well
as figuring out which branch each one is on.
from buildbot.changes.svnpoller import SVNPoller c['change_source'] = SVNPoller("http://divmod.org/svn/Divmod", split_file=my_file_splitter)
my_file_splitter function will be called with
repository-relative pathnames like:
formless/webform.py", and a branch of None
The following definition for
my_file_splitter will do the job:
def my_file_splitter(path): pieces = path.split('/') if pieces == 'trunk': branch = None pieces.pop(0) # remove 'trunk' elif pieces == 'branches': pieces.pop(0) # remove 'branches' # grab branch name branch = 'branches/' + pieces.pop(0) else: return None # something weird projectname = pieces.pop(0) if projectname != 'Nevow': return None # wrong project return (branch, '/'.join(pieces))