Release Notes for Buildbot 0.8.9 ================================ .. Any change that adds a feature or fixes a bug should have an entry here. Most simply need an additional bulleted list item, but more significant changes can be given a subsection of their own. The following are the release notes for Buildbot 0.8.9. Buildbot 0.8.9 was released on 14 June, 2014. Master ------ Features ~~~~~~~~ * The following optional parameters have been added to :py:class:`EC2LatentBuildSlave` * Boolean parameter ``spot_instance``, default False, creates a spot instance. * Float parameter ``max_spot_price`` defines the maximum bid for a spot instance. * List parameter ``volumes``, takes a list of (volume_id, mount_point) tuples. * String parameter ``placement`` is appended to the ``region`` parameter, e.g. ``region='us-west-2', placement='b'`` will result in the spot request being placed in us-west-2b. * Float parameter ``price_multiplier`` specifies the percentage bid above the 24-hour average spot price. * Dict parameter ``tags`` specifies AWS tags as key/value pairs to be applied to new instances. With ``spot_instance=True``, an ``EC2LatentBuildSlave`` will attempt to create a spot instance with the provided spot price, placement, and so on. * The web hooks now include support for Bitbucket, GitLab and Gitorious. * The GitHub webhook has been updated to work with v3 of the GitHub webhook API. * The GitHub webhook can now optionally ignore non-distinct commits (:bug:`1861`). * The :bb:chsrc:`HgPoller` and :bb:chsrc:`GitPoller` now split filenames on newlines, rather than whitespace, so files containing whitespace are handled correctly. * Add 'pollAtLaunch' flag for polling change sources. This allows a poller to poll immediately on launch and get changes that occurred while it was down. * Added the :bb:chsrc:`BitbucketPullrequestPoller` changesource. * The :bb:chsrc:`GitPoller` can now be configured to poll all available branches (:pull:`1010`). * The :bb:chsrc:`P4Source` changesource now supports Perforce servers in a different timezone than the buildbot master (:pull:`728`). * Each Scheduler type can now take a 'reason' argument to customize the reason it uses for triggered builds. * A new argument ``createAbsoluteSourceStamps`` has been added to :bb:sched:`SingleBranchScheduler` for use with multiple codebases. * A new argument ``createAbsoluteSourceStamps`` has been added to :bb:sched:`Nightly` for use with multiple codebases. * The :bb:sched:`Periodic` scheduler now supports codebases. * The :bb:sched:`ForceScheduler` now takes a ``buttonName`` argument to specify the name of the button on the force-build form. * Master side source checkout steps now support patches (:bug:`2098`). The :bb:step:`Git` and :bb:step:`Mercurial` steps use their inbuilt commands to apply patches (:bug:`2563`). * Master side source checkout steps now support retry option (:bug:`2465`). * Master-side source checkout steps now respond to the "stop build" button (:bug:`2356`). * :bb:step:`Git` source checkout step now supports reference repositories. * The :bb:step:`Git` step now uses the `git clean` option `-f` twice, to also remove untracked directories managed by another git repository. See :bug:`2560`. * The ``branch`` and ``codebase`` arguments to the :bb:step:`Git` step are now renderable. * Gerrit integration with :bb:step:`Git` Source step on master side (:bug:`2485`). * :bb:step:`P4` source step now supports more advanced options. * The master-side :bb:step:`SVN` step now supports authentication for mode=export, fixing :bug:`2463`. * The :bb:step:`SVN` step will now canonicalize URL's before matching them for better accuracy. * The :bb:step:`SVN` step now obfuscates the password in status logs, fixing :bug:`2468`. * :bb:step:`SVN` source step and ShellCommand now support password obfuscation. (:bug:`2468` and :bug:`1748`). * :bb:step:`CVS` source step now checks for "sticky dates" from a previous checkout before updating an existing source directory. * ::bb:step:`Repo` now supports a ``depth`` flag when initializing the repo. This controls the amount of git history to download. * The ``manifestBranch`` of the bb:step:`Repo` step is now renderable * New source step :bb:step:`Monotone` added on master side. * New source step :bb:step:`Darcs` added on master side. * A new :bb:step:`Robocopy` step is available for Windows builders (:pull:`728`). * The attributes ``description``, ``descriptionDone`` and ``descriptionSuffix`` have been moved from :py:class:`ShellCommand` to its superclass :py:class:`BuildStep` so that any class that inherits from :py:class:`BuildStep` can provide a suitable description of itself. * A new :py:class:`FlattenList` Renderable has been added which can flatten nested lists. * Added new build steps for :bb:step:`VC12`, :bb:step:`VS2013` and :bb:step:`MsBuild12`. * The ``mode`` parameter of the VS steps is now renderable (:bug:`2592`). * The :bb:step:`HTTPStep` step can make arbitrary HTTP requests from the master, allowing communication with external APIs. This new feature requires the optional ``txrequests`` and ``requests`` Python packages. * A new :bb:step:`MultipleFileUpload` step was added to allow uploading several files (or directories) in a single step. * Information about the buildslaves (admin, host, etc) is now persisted in the database and available even if the slave is not connected. * Buildslave info can now be retrieved via :ref:`Interpolate` and a new ``SetSlaveInfo`` buildstep. * The ``GNUAutotools`` factory now has a reconf option to run autoreconf before ``./configure``. * Builder configurations can now include a ``description``, which will appear in the web UI to help humans figure out what the builder does. * The WebStatus builder page can now filter pending/current/finished builds by property parameters of the form ``?property.=``. * The WebStatus ``StatusResourceBuilder`` page can now take the ``maxsearch`` argument * The WebStatus has a new authz "view" action that allows you to require users to logged in to view the WebStatus. * The WebStatus now shows revisions (+ codebase) where it used to simply say "multiple rev". * The Console view now supports codebases. * The web UI for Builders has been updated: * shows the build 'reason' and 'interested users' * shows sourcestamp information for builders that use multiple codebases (instead of the generic "multiple rev" placeholder that was shown before). * The waterfall and atom/rss feeds can be filtered with the ``project`` url parameter. * The WebStatus ``Authorization`` support now includes a ``view`` action which can be used to restrict read-only access to the Buildbot instance. * The web status now has options to cancel some or all pending builds. * The WebStatus now interprets ANSI color codes in stdio output. * It is now possible to select categories to show in the waterfall help * The web status now automatically scrolls output logs (:pull:`1078`). * The web UI now supports a PNG Status Resource that can be accessed publicly from for example README.md files or wikis or whatever other resource. This view produces an image in PNG format with information about the last build for the given builder name or whatever other build number if is passed as an argument to the view. * Revision links for commits on SouceForge (Allura) are now automatically generated. * The 'Rebuild' button on the web pages for builds features a dropdown to choose whether to rebuild from exact revisions or from the same sourcestamps (ie, update branch references) * Build status can be sent to GitHub. Depends on txgithub package. See :bb:reporter:`GitHubStatusPush` and `GitHub Commit Status `_. * The IRC bot of :bb:reporter:`IRC` will, unless useRevisions is set, shorten long lists of revisions printed when a build starts; it will only show two, and the number of additional revisions included in the build. * A new argument ``summaryCB`` has been added to ``GerritStatusPush``, to allow sending one review per buildset. Sending a single "summary" review per buildset is now the default if neither ``summaryCB`` nor ``reviewCB`` are specified. * The ``comments`` field of changes is no longer limited to 1024 characters on MySQL and Postgres. See :bug:`2367` and :pull:`736`. * HTML log files are no longer stored in status pickles (:pull:`1077`) * Builds are now retried after a slave is lost (:pull:`1049`). * The buildbot status client can now access a build properties via the ``getProperties`` call. * The ``start``, ``restart``, and ``reconfig`` commands will now wait for longer than 10 seconds as long as the master continues producing log lines indicating that the configuration is progressing. * Added new config option ``protocols`` which allows to configure multiple protocols on single master. * RemoteShellCommands can be killed by SIGTERM with the sigtermTime parameter before resorting to SIGKILL (:bug:`751`). If the slave's version is less than 0.8.9, the slave will kill the process with SIGKILL regardless of whether sigtermTime is supplied. * Introduce an alternative way to deploy Buildbot and try the pyflakes tutorial using :ref:`Docker `. * Added zsh and bash tab-completions support for 'buildbot' command. * An example of a declarative configuration is included in ``master/contrib/SimpleConfig.py``, with copious comments. * Systemd unit files for Buildbot are available in the :contrib-src:`master/contrib/` directory. * We've added some extra checking to make sure that you have a valid locale before starting buildbot (#2608). Forward Compatibility ~~~~~~~~~~~~~~~~~~~~~ In preparation for a more asynchronous implementation of build steps in Buildbot 0.9.0, this version introduces support for new-style steps. Existing old-style steps will continue to function correctly in Buildbot 0.8.x releases and in Buildbot 0.9.0, but support will be dropped soon afterward. See :ref:`New-Style-Build-Steps`, below, for guidance on rewriting existing steps in this new style. To eliminate ambiguity, the documentation for this version only reflects support for new-style steps. Refer to the documentation for previous versions for infrormation on old-style steps. Fixes ~~~~~ * Fixes an issue where :bb:chsrc:`GitPoller` sets the change branch to ``refs/heads/master`` - which isn't compatible with :bb:step:`Git` (:pull:`1069`). * Fixed an issue where the :bb:step:`Git` and :bb:step:`CVS` source steps silently changed the ``workdir`` to ``'build'`` when the 'copy' method is used. * The :bb:step:`CVS` source step now respects the timeout parameter. * The :bb:step:`Git` step now uses the `git submodule update` option `--init` when updating the submodules of an existing repository, so that it will receive any newly added submodules. * The web status no longer relies on the current working directory, which is not set correctly by some initscripts, to find the ``templates/`` directory (:bug:`2586`). * The Perforce source step uses the correct path separator when the master is on Windows and the build slave is on a POSIX OS (:pull:`1114`). * The source steps now correctly interpolate properties in ``env``. * ``GerritStatusPush`` now supports setting scores with Gerrit 2.6 and newer * The change hook no longer fails when passing unicode to ``change_hook_auth`` (:pull:`996`). * The source steps now correctly interpolate properties in ``env``. * Whitespace is properly handled for StringParameter, so that appropriate validation errors are raised for ``required`` parameters (:pull:`1084`). * Fix a rare case where a buildtep might fail from a GeneratorExit exception (:pull:`1063`). * Fixed an issue where UTF-8 data in logs caused RSS feed exceptions (:bug:`951`). * Fix an issue with unescaped author names causing invalid RSS feeds (:bug:`2596`). * Fixed an issue with pubDate format in feeds. * Fixed an issue where the step text value could cause a ``TypeError`` in the build detail page (:pull:`1061`). * Fix failures where ``git clean`` fails but could be clobbered (:pull:`1058`). * Build step now correctly fails when the git clone step fails (:pull:`1057`). * Fixed a race condition in slave shutdown (:pull:`1019`). * Now correctly unsubscribes StatusPush from status updates when reconfiguring (:pull:`997`). * Fixes parsing git commit messages that are blank. * :bb:step:`Git` no longer fails when work dir exists but isn't a checkout (:bug:`2531`). * The `haltOnFailure` and `flunkOnFailure` attributes of :bb:step:`ShellCommand` are now renderable. (:bug:`2486`). * The `rotateLength` and `maxRotatedFile` arguments are no longer treated as strings in :file:`buildbot.tac`. This fixes log rotation. The upgrade_master command will notify users if they have this problem. * Buildbot no longer specifies a revision when pulling from a mercurial (:bug:`438`). * The WebStatus no longer incorrectly refers to fields that might not be visible. * The GerritChangeSource now sets a default author, fixing an exception that occurred when Gerrit didn't report an owner name/email. * Respects the ``RETRY`` status when an interrupt occurs. * Fixes an off-by-one error when the tryclient is finding the current git branch. * Improve the Mercurial source stamp extraction in the try client. * Fixes some edge cases in timezone handling for python < ``2.7.4`` (:bug:`2522`). * The ``EC2LatentBuildSlave`` will now only consider available AMI's. * Fixes a case where the first build runs on an old slave instead of a new one after reconfig (:bug:`2507`). * The e-mail address validation for the MailNotifier status receiver has been improved. * The ``--db`` parameter of ``buildbot create-master`` is now validated. * No longer ignores default choice for ForceScheduler list parameters * Now correctly handles ``BuilderConfig(..., mergeRequests=False)`` (:bug:`2555`). * Now excludes changes from sourcestamps when they aren't in the DB (:bug:`2554`). * Fixes a compatibility issue with HPCloud in the OpenStack latent slave. * Allow ``_`` as a valid character in JSONP callback names. * Fix build start time retrieval in the WebStatus grid view. * Increase the length of the DB fields ``changes.comments`` and ``buildset_properties.property_value``. Deprecations, Removals, and Non-Compatible Changes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * The slave-side source steps are deprecated in this version of Buildbot, and master-side support will be removed in a future version. Please convert any use of slave-side steps (imported directly from ``buildbot.steps.source``, rather than from a specific module like ``buildbot.steps.source.svn``) to use master-side steps. * Both old-style and new-style steps are supported in this version of Buildbot. Upgrade your steps to new-style now, as support for old-style steps will be dropped after Buildbot-0.9.0. See :ref:`New-Style-Build-Steps` for details. * The ``LoggingBuildStep`` class has been deprecated, and support will be removed along with support for old-style steps after the Buildbot-0.9.0 release. Instead, subclass :class:`~buildbot.process.buildstep.BuildStep` and mix in :class:`~buildbot.process.buildstep.ShellMixin` to get similar behavior. * ``slavePortnum`` option deprecated, please use ``c['protocols']['pb']['port']`` to set up PB port * The ``buildbot.process.mtrlogobserver`` module have been renamed to :py:mod:`buildbot.steps.mtrlogobserver`. * The buildmaster now requires at least Twisted-11.0.0. * The buildmaster now requires at least sqlalchemy-migrate 0.6.1. * The ``hgbuildbot`` Mercurial hook has been moved to ``contrib/``, and does not work with recent versions of Mercurial and Twisted. The runtimes for these two tools are incompatible, yet ``hgbuildbot`` attempts to run both in the same Python interpreter. Mayhem ensues. * The try scheduler's ``--connect=ssh`` method no longer supports waiting for results (``--wait``). * The former ``buildbot.process.buildstep.RemoteCommand`` class and its subclasses are now in :py:mod:`buildbot.process.remotecommand`, although imports from the previous path will continue to work. Similarly, the former ``buildbot.process.buildstep.LogObserver`` class and its subclasses are now in :py:mod:`buildbot.process.logobserver`, although imports from the previous path will continue to work. * The undocumented BuildStep method ``checkDisconnect`` is deprecated and now does nothing as the handling of disconnects is now handled in the ``failed`` method. Any custom steps adding this method as a callback or errback should no longer do so. * The build step ``MsBuild`` is now called ``MsBuild4`` as multiple versions are now supported. An alias is provided so existing setups will continue to work, but this will be removed in a future release. Changes for Developers ~~~~~~~~~~~~~~~~~~~~~~ * The :py:class:`CompositeStepMixin` now provides a ``runGlob`` method to check for files on the slave that match a given shell-style pattern. * The :py:class:`BuilderStatus` now allows you to pass a ``filter_fn`` argument to ``generateBuilds``. Slave ----- Features ~~~~~~~~ * Added zsh and bash tab-completions support for 'buildslave' command. * RemoteShellCommands accept the new sigtermTime parameter from master. This allows processes to be killed by SIGTERM before resorting to SIGKILL (:bug:`751`) * Commands will now throw a ``ValueError`` if mandatory args are not present. * Added a new remote command :py:class:`GlobPath` that can be used to call Python's ``glob.glob`` on the slave. Fixes ~~~~~ * Fixed an issue when buildstep stop() was raising an exception incorrectly if timeout for buildstep wasn't set or was None (see :pull:`753`) thus keeping watched logfiles open (this prevented their removal on Windows in subsequent builds). * Fixed a bug in P4 source step where the ``timeout`` parameter was ignored. * Fixed a bug in P4 source step where using a custom view-spec could result in failed syncs due to incorrectly generated command-lines. * The logwatcher will use ``/usr/xpg4/bin/tail`` on Solaris, it if is available (:pull:`1065`). Deprecations, Removals, and Non-Compatible Changes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Details ------- For a more detailed description of the changes made in this version, see the git log itself: .. code-block:: bash git log v0.8.8..v0.8.9