Caution

This page documents the latest, unreleased version of Buildbot. For documentation for released versions, see https://docs.buildbot.net/current/.

3.8.7. Logs connector

class buildbot.db.logs.LogsConnectorComponent

This class handles log data. Build steps can have zero or more logs. Logs are uniquely identified by name within a step.

Information about a log, apart from its contents, is represented as a LogModel dataclass with the following fields:

  • id (log ID, globally unique)

  • stepid (step ID, indicating the containing step)

  • name free-form name of this log

  • slug (50-identifier for the log, unique within the step)

  • complete (true if the log is complete and will not receive more lines)

  • num_lines (number of lines in the log)

  • type (log type; see below)

Each log has a type that describes how to interpret its contents. See the logchunk resource type for details.

A log contains a sequence of newline-separated lines of unicode. Log line numbering is zero-based.

Each line must be less than 64k when encoded in UTF-8. Longer lines will be truncated, and a warning will be logged.

Lines are stored internally in “chunks”, and optionally compressed, but the implementation hides these details from callers.

getLog(logid)
Parameters:

logid (integer) – ID of the requested log

Returns:

LogModel or None, via Deferred

Get a log, identified by logid.

getLogBySlug(stepid, slug)
Parameters:

  • stepid (integer) – ID of the step containing this log

  • slug – slug of the logfile to retrieve

Returns:

LogModel or None, via Deferred

Get a log, identified by name within the given step.

getLogs(stepid)
Parameters:

stepid (integer) – ID of the step containing the desired logs

Returns:

list of LogModel via Deferred

Get all logs within the given step.

iter_log_lines(logid, first_line, last_line)
Parameters:

  • logid (integer) – ID of the log

  • first_line – first line to return

  • last_line – last line to return

Returns:

an AsyncGenerator of the chunks content (as str)

Get a subset of lines for a logfile.

yield lines (including line-ending).

getLogLines(logid, first_line, last_line)
Parameters:

  • logid (integer) – ID of the log

  • first_line – first line to return

  • last_line – last line to return

Returns:

see below

Get a subset of lines for a logfile.

The return value, via Deferred, is a concatenation of newline-terminated strings. If the requested last line is beyond the end of the logfile, only existing lines will be included. If the log does not exist, or has no associated lines, this method returns an empty string.

addLog(stepid, name, type)
Parameters:

  • stepid (integer) – ID of the step containing this log

  • name (string) – name of the logfile

  • slug (50-character identifier) – slug (unique identifier) of the logfile

  • type (string) – log type (see above)

Raises:

KeyError – if a log with the given slug already exists in the step

Returns:

ID of the new log, via Deferred

Add a new log file to the given step.

appendLog(logid, content)
Parameters:

  • logid (integer) – ID of the requested log

  • content (string) – new content to be appended to the log

Returns:

tuple of first and last line numbers in the new chunk, via Deferred

Append content to an existing log. The content must end with a newline. If the given log does not exist, the method will silently do nothing.

It is not safe to call this method more than once simultaneously for the same logid.

finishLog(logid)
Parameters:

logid (integer) – ID of the log to mark complete

Returns:

Deferred

Mark a log as complete.

Note that no checking for completeness is performed when appending to a log. It is up to the caller to avoid further calls to appendLog after finishLog.

compressLog(logid)
Parameters:

logid (integer) – ID of the log to compress

Returns:

Deferred

Compress the given log. This method performs internal optimizations on a log’s chunks to reduce the space used and make read operations more efficient. It should only be called for finished logs. This method may take some time to complete.

deleteOldLogChunks(older_than_timestamp)
Parameters:

older_than_timestamp (integer) – the logs whose step’s started_at is older than older_than_timestamp will be deleted.

Returns:

Deferred

Delete old logchunks (helper for the logHorizon policy). Old logs have their logchunks deleted from the database, but they keep their num_lines metadata. They have their types changed to ‘d’, so that the UI can display something meaningful.