3.4.18. LogObservers

class buildbot.process.logobserver.LogObserver

This is a base class for objects which receive logs from worker commands as they are produced. It does not provide an interface for reading logs - such access should occur directly through the Data API.

See Adding LogObservers for help creating and using a custom log observer.

The three methods that subclasses may override follow. None of these methods may return a Deferred. It is up to the callee to handle any asynchronous operations. Subclasses may also override the constructor, with no need to call LogObserver's constructor.

outReceived(data):
Parameters:data (unicode) -- received data

This method is invoked when a "chunk" of data arrives in the log. The chunk contains one or more newline-terminated unicode lines. For stream logs (e.g., stdio), output to stderr generates a call to errReceived, instead.

errReceived(data):
Parameters:data (unicode) -- received data

This method is similar to outReceived, but is called for output to stderr.

headerReceived(data):
Parameters:data (unicode) -- received data

This method is similar to outReceived, but is called for header output.

finishReceived()

This method is invoked when the observed log is finished.

class buildbot.process.logobserver.LogLineObserver

This subclass of LogObserver calls its subclass methods once for each line, instead of once per chunk.

outLineReceived(line):
Parameters:line (unicode) -- received line, without newline

Like outReceived, this is called once for each line of output received. The argument does not contain the trailing newline character.

errLineReceived(line):
Parameters:line (unicode) -- received line, without newline

Similar to outLineReceived, but for stderr.

headerLineReceived(line):
Parameters:line (unicode) -- received line, without newline

Similar to outLineReceived, but for header output..

finishReceived()

This method, inherited from LogObserver, is invoked when the observed log is finished.

class buildbot.process.logobserver.LineConsumerLogObserver

This subclass of LogObserver takes a generator function and "sends" each line to that function. This allows consumers to be written as stateful Python functions, e.g.,

def logConsumer(self):
    while True:
        stream, line = yield
        if stream == 'o' and line.startswith('W'):
            self.warnings.append(line[1:])

def __init__(self):
    ...
    self.warnings = []
    self.addLogObserver('stdio', logobserver.LineConsumerLogObserver(self.logConsumer))

Each yield expression evaluates to a tuple of (stream, line), where the stream is one of 'o', 'e', or 'h' for stdout, stderr, and header, respectively. As with any generator function, the yield expression will raise a GeneratorExit exception when the generator is complete. To do something after the log is finished, just catch this exception (but then re-raise it or return)

def logConsumer(self):
    while True:
        try:
            stream, line = yield
            if stream == 'o' and line.startswith('W'):
                self.warnings.append(line[1:])
        except GeneratorExit:
            self.warnings.sort()
            return

Warning

This use of generator functions is a simple Python idiom first described in PEP 342. It is unrelated to the generators used in inlineCallbacks. In fact, consumers of this type are incompatible with asynchronous programming, as each line must be processed immediately.

class buildbot.process.logobserver.BufferLogObserver(wantStdout=True, wantStderr=False)
Parameters:
  • wantStdout (boolean) -- true if stdout should be buffered
  • wantStderr (boolean) -- true if stderr should be buffered

This subclass of LogObserver buffers stdout and/or stderr for analysis after the step is complete. This can cause excessive memory consumption if the output is large.

getStdout()
Returns:unicode string

Return the accumulated stdout.

getStderr()
Returns:unicode string

Return the accumulated stderr.