3.10.5. RemoteCommands

Most of the action in build steps consists of performing operations on the worker. This is accomplished via RemoteCommand and its subclasses. Each represents a single operation on the worker.

Most data is returned to a command via updates. These updates are described in detail in Updates.

3.10.5.1. RemoteCommand

class buildbot.process.remotecommand.RemoteCommand(remote_command, args, collectStdout=False, ignore_updates=False, decodeRC=dict(0), stdioLogName='stdio')
Parameters:

  • remote_command (string) – command to run on the worker

  • args (dictionary) – arguments to pass to the command

  • collectStdout – if True, collect the command’s stdout

  • ignore_updates – true to ignore remote updates

  • decodeRC – dictionary associating rc values to buildstep results constants (e.g. SUCCESS, FAILURE, WARNINGS)

  • stdioLogName – name of the log to which to write the command’s stdio

This class handles running commands, consisting of a command name and a dictionary of arguments. If true, ignore_updates will suppress any updates sent from the worker.

This class handles updates for stdout, stderr, and header by appending them to a stdio logfile named by the stdioLogName parameter. Steps that run multiple commands and want to separate those commands’ stdio streams can use this parameter.

It handles updates for rc by recording the value in its rc attribute.

Most worker-side commands, even those which do not spawn a new process on the worker, generate logs and an rc, requiring this class or one of its subclasses. See Updates for the updates that each command may send.

active

True if the command is currently running

run(step, remote)
Parameters:

  • step – the buildstep invoking this command

  • remote – a reference to the remote WorkerForBuilder instance

Returns:

Deferred

Run the command. Call this method to initiate the command; the returned Deferred will fire when the command is complete. The Deferred fires with the RemoteCommand instance as its value.

interrupt(why)
Parameters:

why (Twisted Failure) – reason for interrupt

Returns:

Deferred

This method attempts to stop the running command early. The Deferred it returns will fire when the interrupt request is received by the worker; this may be a long time before the command itself completes, at which time the Deferred returned from run will fire.

results()
Returns:

results constant

This method checks the rc against the decodeRC dictionary, and returns a results constant.

didFail()
Returns:

bool

This method returns True if the results() function returns FAILURE.

The following methods are invoked from the worker. They should not be called directly.

remote_update(updates)
Parameters:

updates – new information from the worker

Handles updates from the worker on the running command. See Updates for the content of the updates. This class splits the updates out, and handles the ignore_updates option, then calls remoteUpdate to process the update.

remote_complete(failure=None)
Parameters:

failure – the failure that caused the step to complete, or None for success

Called by the worker to indicate that the command is complete. Normal completion (even with a nonzero rc) will finish with no failure; if failure is set, then the step should finish with status EXCEPTION.

These methods are hooks for subclasses to add functionality.

remoteUpdate(update)
Parameters:

update – the update to handle

Handle a single update. Subclasses must override this method.

remoteComplete(failure)
Parameters:

failure – the failure that caused the step to complete, or None for success

Returns:

Deferred

Handle command completion, performing any necessary cleanup. Subclasses should override this method. If failure is not None, it should be returned to ensure proper processing.

logs

A dictionary of Log instances representing active logs. Do not modify this directly – use useLog instead.

rc

Set to the return code of the command, after the command has completed. For compatibility with shell commands, 0 is taken to indicate success, while nonzero return codes indicate failure.

stdout

If the collectStdout constructor argument is true, then this attribute will contain all data from stdout, as a single string. This is helpful when running informational commands (e.g., svnversion), but is not appropriate for commands that will produce a large amount of output, as that output is held in memory.

To set up logging, use useLog or useLogDelayed before starting the command:

useLog(log, closeWhenFinished=False, logfileName=None)
Parameters:

  • log – the Log instance to add to

  • closeWhenFinished – if true, call finish when the command is finished

  • logfileName – the name of the logfile, as given to the worker. This is stdio for standard streams

Route log-related updates to the given logfile. Note that stdio is not included by default, and must be added explicitly. The logfileName must match the name given by the worker in any log updates.

useLogDelayed(logfileName, activateCallback, closeWhenFinished=False)
Parameters:

  • logfileName – the name of the logfile, as given to the worker. This is stdio for standard streams

  • activateCallback – callback for when the log is added; see below

  • closeWhenFinished – if true, call finish when the command is finished

Similar to useLog, but the logfile is only actually added when an update arrives for it. The callback, activateCallback, will be called with the RemoteCommand instance when the first update for the log is delivered. It should return the desired log instance, optionally via a Deferred.

With that finished, run the command using the inherited run method. During the run, you can inject data into the logfiles with any of these methods:

addStdout(data)
Parameters:

data – data to add to the logfile

Returns:

Deferred

Add stdout data to the stdio log.

addStderr(data)
Parameters:

data – data to add to the logfile

Returns:

Deferred

Add stderr data to the stdio log.

addHeader(data)
Parameters:

data – data to add to the logfile

Returns:

Deferred

Add header data to the stdio log.

addToLog(logname, data)
Parameters:

  • logname – the logfile to receive the data

  • data – data to add to the logfile

Returns:

Deferred

Add data to a logfile other than stdio.

class buildbot.process.remotecommand.RemoteShellCommand(workdir, command, env=None, want_stdout=True, want_stderr=True, timeout=20 * 60, maxTime=None, sigtermTime=None, logfiles={}, usePTY=None, logEnviron=True, collectStdio=False, collectStderr=False, interruptSignal=None, initialStdin=None, decodeRC=None, stdioLogName='stdio')
Parameters:

  • workdir – directory in which the command should be executed, relative to the builder’s basedir

  • command (string or list) – shell command to run

  • want_stdout – If false, then no updates will be sent for stdout

  • want_stderr – If false, then no updates will be sent for stderr

  • timeout – Maximum time without output before the command is killed

  • maxTime – Maximum overall time from the start before the command is killed

  • sigtermTime – Try to kill the command with SIGTERM and wait for sigtermTime seconds before firing interruptSignal or SIGKILL if it’s not defined. If None, SIGTERM will not be fired

  • env – A dictionary of environment variables to augment or replace the existing environment on the worker

  • logfiles – Additional logfiles to request from the worker

  • usePTY – True to use a PTY, false to not use a PTY; the default value is False

  • logEnviron – If false, do not log the environment on the worker

  • collectStdout – If True, collect the command’s stdout

  • collectStderr – If True, collect the command’s stderr

  • interruptSignal – The signal to send to interrupt the command, e.g. KILL or TERM. If None, SIGKILL is used

  • initialStdin – The input to supply the command via stdin

  • decodeRC – dictionary associating rc values to buildstep results constants (e.g. SUCCESS, FAILURE, WARNINGS)

  • stdioLogName – name of the log to which to write the command’s stdio

Most of the constructor arguments are sent directly to the worker; see shell for the details of the formats. The collectStdout, decodeRC and stdioLogName parameters are as described for the parent class.

If a shell command contains passwords, they can be hidden from log files by using Secret Management. This is the recommended procedure for new-style build steps. For legacy build steps passwords were hidden from the log file by passing them as tuples in command arguments. Eg. ['print', ('obfuscated', 'password', 'dummytext')] is logged as ['print', 'dummytext'].

This class is used by the ShellCommand step, and by steps that run multiple customized shell commands.