| Home | Trees | Indices | Help |
|
|---|
|
|
process.buildstep.BuildStep --+
|
process.buildstep.LoggingBuildStep --+
|
shell.ShellCommand --+
|
Trial
I run a unit test suite using 'trial', a unittest-like testing framework that comes with Twisted. Trial is used to implement Twisted's own unit tests, and is the unittest-framework of choice for many projects that use Twisted internally.
Projects that use trial typically have all their test cases in a 'test' subdirectory of their top-level library directory. I.e. for my package 'petmail', the tests are in 'petmail/test/test_*.py'. More complicated packages (like Twisted itself) may have multiple test directories, like 'twisted/test/test_*.py' for the core functionality and 'twisted/mail/test/test_*.py' for the email-specific tests.
To run trial tests, you run the 'trial' executable and tell it where the test cases are located. The most common way of doing this is with a module name. For petmail, I would run 'trial petmail.test' and it would locate all the test_*.py files under petmail/test/, running every test case it could find in them. Unlike the unittest.py that comes with Python, you do not run the test_foo.py as a script; you always let trial do the importing and running. The 'tests' parameter controls which tests trial will run: it can be a string or a list of strings.
To find these test cases, you must set a PYTHONPATH that allows something like 'import petmail.test' to work. For packages that don't use a separate top-level 'lib' directory, PYTHONPATH=. will work, and will use the test cases (and the code they are testing) in-place. PYTHONPATH=build/lib or PYTHONPATH=build/lib.$ARCH are also useful when you do a'setup.py build' step first. The 'testpath' attribute of this class controls what PYTHONPATH= is set to.
Trial has the ability (through the --testmodule flag) to run only the set of test cases named by special 'test-case-name' tags in source files. We can get the list of changed source files from our parent Build and provide them to trial, thus running the minimal set of test cases needed to cover the Changes. This is useful for quick builds, especially in trees with a lot of test cases. The 'testChanges' parameter controls this feature: if set, it will override 'tests'.
The trial executable itself is typically just 'trial' (which is usually found on your $PATH as /usr/bin/trial), but it can be overridden with the 'trial' parameter. This is useful for Twisted's own unittests, which want to use the copy of bin/trial that comes with the sources. (when bin/trial discovers that it is living in a subdirectory named 'Twisted', it assumes it is being run from the source tree and adds that parent directory to PYTHONPATH. Therefore the canonical way to run Twisted's own unittest suite is './bin/trial twisted.test' rather than 'PYTHONPATH=. /usr/bin/trial twisted.test', especially handy when /usr/bin/trial has not yet been installed).
To influence the version of python being used for the tests, or to add flags to the command, set the 'python' parameter. This can be a string (like 'python2.2') or a list (like ['python2.3', '-Wall']).
Trial creates and switches into a directory named _trial_temp/ before running the tests, and sends the twisted log (which includes all exceptions) to a file named test.log . This file will be pulled up to the master where it can be seen as part of the status output.
There are some class attributes which may be usefully overridden by subclasses. 'trialMode' and 'trialArgs' can influence the trial command line.
| Instance Methods | |||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
Inherited from Inherited from Inherited from |
|||
| Class Variables | |
name = |
|
progressMetrics = |
|
logfiles = a dict mapping log NAMEs to workdir-relative FILENAMEs of their corresponding logfiles. |
|
flunkOnFailure = True
|
|
python = None
|
|
trial = |
|
trialMode = |
|
trialArgs = |
|
testpath = |
|
testChanges = False
|
|
recurse = False
|
|
reactor = None
|
|
randomly = False
|
|
tests = None
|
|
|
Inherited from Inherited from Inherited from |
|
| Instance Variables | |
|
Inherited from Inherited from |
| Method Details |
|
|
Begin the step. Override this method and add code to do local processing, fire off remote commands, etc. To spawn a command in the buildslave, create a RemoteCommand instance and run it with self.runCommand: c = RemoteCommandFoo(args) d = self.runCommand(c) d.addCallback(self.fooDone).addErrback(self.failed) As the step runs, it should send status information to the BuildStepStatus: self.step_status.setText(['compile', 'failed']) self.step_status.setText2(['4', 'warnings']) To have some code parse stdio (or other log stream) in realtime, add a LogObserver subclass. This observer can use self.step.setProgress() to provide better progress notification to the step.:
self.addLogObserver('stdio', MyLogObserver())
To add a LogFile, use self.addLog. Make sure it gets closed when it finishes. When giving a Logfile to a RemoteShellCommand, just ask it to close the log when the command completes:
log = self.addLog('output')
cmd = RemoteShellCommand(args)
cmd.useLog(log, closeWhenFinished=True)
You can also create complete Logfiles with generated text in a single step:
self.addCompleteLog('warnings', text)
When the step is done, it should call self.finished(result). 'result' will be provided to the buildbot.process.base.Build, and should be one of the constants defined above: SUCCESS, WARNINGS, FAILURE, or SKIPPED. If the step encounters an exception, it should call self.failed(why). 'why' should be a Failure object. This automatically fails the whole build with an exception. It is a good idea to add self.failed as an errback to any Deferreds you might obtain. If the step decides it does not need to be run, start() can return the constant SKIPPED. This fires the callback immediately: it is not necessary to call .finished yourself. This can also indicate to the status-reporting mechanism that this step should not be displayed. A step can be configured to only run under certain conditions. To do this, set the step's doStepIf to a boolean value, or to a function that returns a boolean value. If the value or function result is False, then the step will return SKIPPED without doing anything, otherwise the step will be executed normally. If you set doStepIf to a function, that function should accept one parameter, which will be the Step object itself.
|
This is a general-purpose hook method for subclasses. It will be called after the remote command has finished, but before any of the other hook functions are called.
|
To create summary logs, do something like this: warnings = grep('^Warning:', log.getText()) self.addCompleteLog('warnings', warnings)
|
Decide whether the command was SUCCESS, WARNINGS, or FAILURE. Override this to, say, declare WARNINGS if there is any stderr activity, or to say that rc!=0 is not actually an error.
|
|
We have decided to add a short note about ourselves to the overall build description, probably because something went wrong. Return a short list of short strings. If your subclass counts test failures or warnings of some sort, this is a good place to announce the count.
|
| Home | Trees | Indices | Help |
|
|---|
| Generated by Epydoc 3.0.1 on Tue May 25 17:52:56 2010 | http://epydoc.sourceforge.net |