Caution
This page documents the latest, unreleased version of Buildbot. For documentation for released versions, see https://docs.buildbot.net/current/.
3.10.5. RemoteCommands
Most of the action in build steps consists of performing operations on the worker.
This is accomplished via RemoteCommand
and its subclasses.
Each represents a single operation on the worker.
Most data is returned to a command via updates. These updates are described in detail in Updates.
3.10.5.1. RemoteCommand
- class buildbot.process.remotecommand.RemoteCommand(remote_command, args, collectStdout=False, ignore_updates=False, decodeRC=dict(0), stdioLogName='stdio')
- Parameters:
remote_command (string) – command to run on the worker
args (dictionary) – arguments to pass to the command
collectStdout – if True, collect the command’s stdout
ignore_updates – true to ignore remote updates
decodeRC – dictionary associating
rc
values to buildstep results constants (e.g.SUCCESS
,FAILURE
,WARNINGS
)stdioLogName – name of the log to which to write the command’s stdio
This class handles running commands, consisting of a command name and a dictionary of arguments. If true,
ignore_updates
will suppress any updates sent from the worker.This class handles updates for
stdout
,stderr
, andheader
by appending them to a stdio logfile named by thestdioLogName
parameter. Steps that run multiple commands and want to separate those commands’ stdio streams can use this parameter.It handles updates for
rc
by recording the value in itsrc
attribute.Most worker-side commands, even those which do not spawn a new process on the worker, generate logs and an
rc
, requiring this class or one of its subclasses. See Updates for the updates that each command may send.- active
True if the command is currently running
- run(step, remote)
- Parameters:
step – the buildstep invoking this command
remote – a reference to the remote
WorkerForBuilder
instance
- Returns:
Deferred
Run the command. Call this method to initiate the command; the returned Deferred will fire when the command is complete. The Deferred fires with the
RemoteCommand
instance as its value.
- interrupt(why)
- Parameters:
why (Twisted Failure) – reason for interrupt
- Returns:
Deferred
This method attempts to stop the running command early. The Deferred it returns will fire when the interrupt request is received by the worker; this may be a long time before the command itself completes, at which time the Deferred returned from
run
will fire.
- results()
- Returns:
results constant
This method checks the
rc
against the decodeRC dictionary, and returns a results constant.
- didFail()
- Returns:
bool
This method returns True if the results() function returns FAILURE.
The following methods are invoked from the worker. They should not be called directly.
- remote_update(updates)
- Parameters:
updates – new information from the worker
Handles updates from the worker on the running command. See Updates for the content of the updates. This class splits the updates out, and handles the
ignore_updates
option, then callsremoteUpdate
to process the update.
- remote_complete(failure=None)
- Parameters:
failure – the failure that caused the step to complete, or None for success
Called by the worker to indicate that the command is complete. Normal completion (even with a nonzero
rc
) will finish with no failure; iffailure
is set, then the step should finish with statusEXCEPTION
.
These methods are hooks for subclasses to add functionality.
- remoteUpdate(update)
- Parameters:
update – the update to handle
Handle a single update. Subclasses must override this method.
- remoteComplete(failure)
- Parameters:
failure – the failure that caused the step to complete, or None for success
- Returns:
Deferred
Handle command completion, performing any necessary cleanup. Subclasses should override this method. If
failure
is not None, it should be returned to ensure proper processing.
- logs
A dictionary of
Log
instances representing active logs. Do not modify this directly – useuseLog
instead.
- rc
Set to the return code of the command, after the command has completed. For compatibility with shell commands, 0 is taken to indicate success, while nonzero return codes indicate failure.
- stdout
If the
collectStdout
constructor argument is true, then this attribute will contain all data from stdout, as a single string. This is helpful when running informational commands (e.g.,svnversion
), but is not appropriate for commands that will produce a large amount of output, as that output is held in memory.
To set up logging, use
useLog
oruseLogDelayed
before starting the command:- useLog(log, closeWhenFinished=False, logfileName=None)
- Parameters:
Route log-related updates to the given logfile. Note that
stdio
is not included by default, and must be added explicitly. ThelogfileName
must match the name given by the worker in anylog
updates.
- useLogDelayed(logfileName, activateCallback, closeWhenFinished=False)
- Parameters:
logfileName – the name of the logfile, as given to the worker. This is
stdio
for standard streamsactivateCallback – callback for when the log is added; see below
closeWhenFinished – if true, call
finish
when the command is finished
Similar to
useLog
, but the logfile is only actually added when an update arrives for it. The callback,activateCallback
, will be called with theRemoteCommand
instance when the first update for the log is delivered. It should return the desired log instance, optionally via a Deferred.
With that finished, run the command using the inherited
run
method. During the run, you can inject data into the logfiles with any of these methods:- addStdout(data)
- Parameters:
data – data to add to the logfile
- Returns:
Deferred
Add stdout data to the
stdio
log.
- addStderr(data)
- Parameters:
data – data to add to the logfile
- Returns:
Deferred
Add stderr data to the
stdio
log.
- addHeader(data)
- Parameters:
data – data to add to the logfile
- Returns:
Deferred
Add header data to the
stdio
log.
- addToLog(logname, data)
- Parameters:
logname – the logfile to receive the data
data – data to add to the logfile
- Returns:
Deferred
Add data to a logfile other than
stdio
.
- class buildbot.process.remotecommand.RemoteShellCommand(workdir, command, env=None, want_stdout=True, want_stderr=True, timeout=20 * 60, maxTime=None, max_lines=None, sigtermTime=None, logfiles={}, usePTY=None, logEnviron=True, collectStdio=False, collectStderr=False, interruptSignal=None, initialStdin=None, decodeRC=None, stdioLogName='stdio')
- Parameters:
workdir – directory in which the command should be executed, relative to the builder’s basedir
command (string or list) – shell command to run
want_stdout – If false, then no updates will be sent for stdout
want_stderr – If false, then no updates will be sent for stderr
timeout – Maximum time without output before the command is killed
maxTime – Maximum overall time from the start before the command is killed
max_lines – Maximum lines command can produce to stdout, then command is killed.
sigtermTime – Try to kill the command with SIGTERM and wait for sigtermTime seconds before firing
interruptSignal
or SIGKILL if it’s not defined. If None, SIGTERM will not be firedenv – A dictionary of environment variables to augment or replace the existing environment on the worker
logfiles – Additional logfiles to request from the worker
usePTY – True to use a PTY, false to not use a PTY; the default value is False
logEnviron – If false, do not log the environment on the worker
collectStdout – If True, collect the command’s stdout
collectStderr – If True, collect the command’s stderr
interruptSignal – The signal to send to interrupt the command, e.g.
KILL
orTERM
. If None, SIGKILL is usedinitialStdin – The input to supply the command via stdin
decodeRC – dictionary associating
rc
values to buildstep results constants (e.g.SUCCESS
,FAILURE
,WARNINGS
)stdioLogName – name of the log to which to write the command’s stdio
Most of the constructor arguments are sent directly to the worker; see shell for the details of the formats. The
collectStdout
,decodeRC
andstdioLogName
parameters are as described for the parent class.If a shell command contains passwords, they can be hidden from log files by using Secret Management. This is the recommended procedure for new-style build steps. For legacy build steps passwords were hidden from the log file by passing them as tuples in command arguments. Eg.
['print', ('obfuscated', 'password', 'dummytext')]
is logged as['print', 'dummytext']
.This class is used by the
ShellCommand
step, and by steps that run multiple customized shell commands.