buildbot.status.web.baseweb

51 implements(IStatusReceiver) 52 # TODO: IStatusReceiver is really about things which subscribe to hear 53 # about buildbot events. We need a different interface (perhaps a parent 54 # of IStatusReceiver) for status targets that don't subscribe, like the 55 # WebStatus class. buildbot.master.BuildMaster.loadConfig:737 asserts 56 # that everything in c['status'] provides IStatusReceiver, but really it 57 # should check that they provide IStatusTarget instead. 58 59 """ 60 The webserver provided by this class has the following resources: 61 62 /waterfall : the big time-oriented 'waterfall' display, with links 63 to individual changes, builders, builds, steps, and logs. 64 A number of query-arguments can be added to influence 65 the display. 66 /rss : a rss feed summarizing all failed builds. The same 67 query-arguments used by 'waterfall' can be added to 68 influence the feed output. 69 /atom : an atom feed summarizing all failed builds. The same 70 query-arguments used by 'waterfall' can be added to 71 influence the feed output. 72 /grid : another summary display that shows a grid of builds, with 73 sourcestamps on the x axis, and builders on the y. Query 74 arguments similar to those for the waterfall can be added. 75 /tgrid : similar to the grid display, but the commits are down the 76 left side, and the build hosts are across the top. 77 /builders/BUILDERNAME: a page summarizing the builder. This includes 78 references to the Schedulers that feed it, 79 any builds currently in the queue, which 80 buildslaves are designated or attached, and a 81 summary of the build process it uses. 82 /builders/BUILDERNAME/builds/NUM: a page describing a single Build 83 /builders/BUILDERNAME/builds/NUM/steps/STEPNAME: describes a single step 84 /builders/BUILDERNAME/builds/NUM/steps/STEPNAME/logs/LOGNAME: a StatusLog 85 /builders/_all/{force,stop}: force a build/stop building on all builders. 86 /buildstatus?builder=...&number=...: an embedded iframe for the console 87 /changes : summarize all ChangeSources 88 /changes/CHANGENUM: a page describing a single Change 89 /buildslaves : list all BuildSlaves 90 /buildslaves/SLAVENAME : describe a single BuildSlave 91 /one_line_per_build : summarize the last few builds, one line each 92 /one_line_per_build/BUILDERNAME : same, but only for a single builder 93 /about : describe this buildmaster (Buildbot and support library versions) 94 /change_hook[/DIALECT] : accepts changes from external sources, optionally 95 choosing the dialect that will be permitted 96 (i.e. github format, etc..) 97 98 and more! see the manual. 99 100 101 All URLs for pages which are not defined here are used to look 102 for files in PUBLIC_HTML, which defaults to BASEDIR/public_html. 103 This means that /robots.txt or /favicon.ico can be placed in 104 that directory 105 106 This webserver uses the jinja2 template system to generate the web pages 107 (see http://jinja.pocoo.org/2/) and by default loads pages from the 108 buildbot.status.web.templates package. Any file here can be overridden by placing 109 a corresponding file in the master's 'templates' directory. 110 111 The main customization points are layout.html which loads style sheet 112 (css) and provides header and footer content, and root.html, which 113 generates the root page. 114 115 All of the resources provided by this service use relative URLs to reach 116 each other. The only absolute links are the c['titleURL'] links at the 117 top and bottom of the page, and the buildbot home-page link at the 118 bottom. 119 120 Buildbot uses some generic classes to identify the type of object, and 121 some more specific classes for the various kinds of those types. It does 122 this by specifying both in the class attributes where applicable, 123 separated by a space. It is important that in your CSS you declare the 124 more generic class styles above the more specific ones. For example, 125 first define a style for .Event, and below that for .SUCCESS 126 127 The following CSS class names are used: 128 - Activity, Event, BuildStep, LastBuild: general classes 129 - waiting, interlocked, building, offline, idle: Activity states 130 - start, running, success, failure, warnings, skipped, exception: 131 LastBuild and BuildStep states 132 - Change: box with change 133 - Builder: box for builder name (at top) 134 - Project 135 - Time 136 137 """ 138 139 # we are not a ComparableMixin, and therefore the webserver will be 140 # rebuilt every time we reconfig. This is because WebStatus.putChild() 141 # makes it too difficult to tell whether two instances are the same or 142 # not (we'd have to do a recursive traversal of all children to discover 143 # all the changes). 144

145 - def __init__(self, http_port=None, distrib_port=None, allowForce=None, 146 public_html="public_html", site=None, numbuilds=20, 147 num_events=200, num_events_max=None, auth=None, 148 order_console_by_time=False, changecommentlink=None, 149 revlink=None, projects=None, repositories=None, 150 authz=None, logRotateLength=None, maxRotatedFiles=None, 151 change_hook_dialects = {}, provide_feeds=None):

152 """Run a web server that provides Buildbot status. 153 154 @type http_port: int or L{twisted.application.strports} string 155 @param http_port: a strports specification describing which port the 156 buildbot should use for its web server, with the 157 Waterfall display as the root page. For backwards 158 compatibility this can also be an int. Use 159 'tcp:8000' to listen on that port, or 160 'tcp:12345:interface=127.0.0.1' if you only want 161 local processes to connect to it (perhaps because 162 you are using an HTTP reverse proxy to make the 163 buildbot available to the outside world, and do not 164 want to make the raw port visible). 165 166 @type distrib_port: int or L{twisted.application.strports} string 167 @param distrib_port: Use this if you want to publish the Waterfall 168 page using web.distrib instead. The most common 169 case is to provide a string that is an absolute 170 pathname to the unix socket on which the 171 publisher should listen 172 (C{os.path.expanduser(~/.twistd-web-pb)} will 173 match the default settings of a standard 174 twisted.web 'personal web server'). Another 175 possibility is to pass an integer, which means 176 the publisher should listen on a TCP socket, 177 allowing the web server to be on a different 178 machine entirely. Both forms are provided for 179 backwards compatibility; the preferred form is a 180 strports specification like 181 'unix:/home/buildbot/.twistd-web-pb'. Providing 182 a non-absolute pathname will probably confuse 183 the strports parser. 184 185 @param allowForce: deprecated; use authz instead 186 @param auth: deprecated; use with authz 187 188 @param authz: a buildbot.status.web.authz.Authz instance giving the authorization 189 parameters for this view 190 191 @param public_html: the path to the public_html directory for this display, 192 either absolute or relative to the basedir. The default 193 is 'public_html', which selects BASEDIR/public_html. 194 195 @type site: None or L{twisted.web.server.Site} 196 @param site: Use this if you want to define your own object instead of 197 using the default.` 198 199 @type numbuilds: int 200 @param numbuilds: Default number of entries in lists at the /one_line_per_build 201 and /builders/FOO URLs. This default can be overriden both programatically --- 202 by passing the equally named argument to constructors of OneLinePerBuildOneBuilder 203 and OneLinePerBuild --- and via the UI, by tacking ?numbuilds=xy onto the URL. 204 205 @type num_events: int 206 @param num_events: Default number of events to show in the waterfall. 207 208 @type num_events_max: int 209 @param num_events_max: The maximum number of events that are allowed to be 210 shown in the waterfall. The default value of C{None} will disable this 211 check 212 213 @type auth: a L{status.web.auth.IAuth} or C{None} 214 @param auth: an object that performs authentication to restrict access 215 to the C{allowForce} features. Ignored if C{allowForce} 216 is not C{True}. If C{auth} is C{None}, people can force or 217 stop builds without auth. 218 219 @type order_console_by_time: bool 220 @param order_console_by_time: Whether to order changes (commits) in the console 221 view according to the time they were created (for VCS like Git) or 222 according to their integer revision numbers (for VCS like SVN). 223 224 @type changecommentlink: callable, dict, tuple (2 or 3 strings) or C{None} 225 @param changecommentlink: adds links to ticket/bug ids in change comments, 226 see buildbot.status.web.base.changecommentlink for details 227 228 @type revlink: callable, dict, string or C{None} 229 @param revlink: decorations revision ids with links to a web-view, 230 see buildbot.status.web.base.revlink for details 231 232 @type projects: callable, dict or c{None} 233 @param projects: maps project identifiers to URLs, so that any project listed 234 is automatically decorated with a link to it's front page. 235 see buildbot.status.web.base.dictlink for details 236 237 @type repositories: callable, dict or c{None} 238 @param repositories: maps repository identifiers to URLs, so that any project listed 239 is automatically decorated with a link to it's web view. 240 see buildbot.status.web.base.dictlink for details 241 242 @type logRotateLength: None or int 243 @param logRotateLength: file size at which the http.log is rotated/reset. 244 If not set, the value set in the buildbot.tac will be used, 245 falling back to the BuildMaster's default value (1 Mb). 246 247 @type maxRotatedFiles: None or int 248 @param maxRotatedFiles: number of old http.log files to keep during log rotation. 249 If not set, the value set in the buildbot.tac will be used, 250 falling back to the BuildMaster's default value (10 files). 251 252 @type change_hook_dialects: None or dict 253 @param change_hook_dialects: If empty, disables change_hook support, otherwise 254 whitelists valid dialects. In the format of 255 {"dialect1": "Option1", "dialect2", None} 256 Where the values are options that will be passed 257 to the dialect 258 259 To enable the DEFAULT handler, use a key of DEFAULT 260 261 262 263 264 @type provide_feeds: None or list 265 @param provide_feeds: If empty, provides atom, json, and rss feeds. 266 Otherwise, a dictionary of strings of 267 the type of feeds provided. Current 268 possibilities are "atom", "json", and "rss" 269 """ 270 271 service.MultiService.__init__(self) 272 if type(http_port) is int: 273 http_port = "tcp:%d" % http_port 274 self.http_port = http_port 275 if distrib_port is not None: 276 if type(distrib_port) is int: 277 distrib_port = "tcp:%d" % distrib_port 278 if distrib_port[0] in "/~.": # pathnames 279 distrib_port = "unix:%s" % distrib_port 280 self.distrib_port = distrib_port 281 self.num_events = num_events 282 if num_events_max: 283 if num_events_max < num_events: 284 config.error( 285 "num_events_max must be greater than num_events") 286 self.num_events_max = num_events_max 287 self.public_html = public_html 288 289 # make up an authz if allowForce was given 290 if authz: 291 if allowForce is not None: 292 config.error( 293 "cannot use both allowForce and authz parameters") 294 if auth: 295 config.error( 296 "cannot use both auth and authz parameters (pass " + 297 "auth as an Authz parameter)") 298 else: 299 # invent an authz 300 if allowForce and auth: 301 authz = Authz(auth=auth, default_action="auth") 302 elif allowForce: 303 authz = Authz(default_action=True) 304 else: 305 if auth: 306 log.msg("Warning: Ignoring authentication. Search for 'authorization'" 307 " in the manual") 308 authz = Authz() # no authorization for anything 309 310 self.authz = authz 311 312 self.orderConsoleByTime = order_console_by_time 313 314 # If we were given a site object, go ahead and use it. (if not, we add one later) 315 self.site = site 316 317 # store the log settings until we create the site object 318 self.logRotateLength = logRotateLength 319 self.maxRotatedFiles = maxRotatedFiles 320 321 # create the web site page structure 322 self.childrenToBeAdded = {} 323 self.setupUsualPages(numbuilds=numbuilds, num_events=num_events, 324 num_events_max=num_events_max) 325 326 self.revlink = revlink 327 self.changecommentlink = changecommentlink 328 self.repositories = repositories 329 self.projects = projects 330 331 # keep track of cached connections so we can break them when we shut 332 # down. See ticket #102 for more details. 333 self.channels = weakref.WeakKeyDictionary() 334 335 # do we want to allow change_hook 336 self.change_hook_dialects = {} 337 if change_hook_dialects: 338 self.change_hook_dialects = change_hook_dialects 339 self.putChild("change_hook", ChangeHookResource(dialects = self.change_hook_dialects)) 340 341 # Set default feeds 342 if provide_feeds is None: 343 self.provide_feeds = ["atom", "json", "rss"] 344 else: 345 self.provide_feeds = provide_feeds

346

347 - def setupUsualPages(self, numbuilds, num_events, num_events_max):

348 #self.putChild("", IndexOrWaterfallRedirection()) 349 self.putChild("waterfall", WaterfallStatusResource(num_events=num_events, 350 num_events_max=num_events_max)) 351 self.putChild("grid", GridStatusResource()) 352 self.putChild("console", ConsoleStatusResource( 353 orderByTime=self.orderConsoleByTime)) 354 self.putChild("tgrid", TransposedGridStatusResource()) 355 self.putChild("builders", BuildersResource()) # has builds/steps/logs 356 self.putChild("one_box_per_builder", Redirect("builders")) 357 self.putChild("changes", ChangesResource()) 358 self.putChild("buildslaves", BuildSlavesResource()) 359 self.putChild("buildstatus", BuildStatusStatusResource()) 360 self.putChild("one_line_per_build", 361 OneLinePerBuild(numbuilds=numbuilds)) 362 self.putChild("about", AboutBuildbot()) 363 self.putChild("authfail", AuthFailResource()) 364 self.putChild("authzfail", AuthzFailResource()) 365 self.putChild("users", UsersResource()) 366 self.putChild("login", LoginResource()) 367 self.putChild("logout", LogoutResource())

368

369 - def __repr__(self):

370 if self.http_port is None: 371 return "<WebStatus on path %s at %s>" % (self.distrib_port, 372 hex(id(self))) 373 if self.distrib_port is None: 374 return "<WebStatus on port %s at %s>" % (self.http_port, 375 hex(id(self))) 376 return ("<WebStatus on port %s and path %s at %s>" % 377 (self.http_port, self.distrib_port, hex(id(self))))

378

379 - def setServiceParent(self, parent):

380 service.MultiService.setServiceParent(self, parent) 381 382 # this class keeps a *separate* link to the buildmaster, rather than 383 # just using self.parent, so that when we are "disowned" (and thus 384 # parent=None), any remaining HTTP clients of this WebStatus will still 385 # be able to get reasonable results. 386 self.master = parent.master 387 388 # set master in IAuth instance 389 if self.authz.auth: 390 self.authz.auth.master = self.master 391 392 def either(a,b): # a if a else b for py2.4 393 if a: 394 return a 395 else: 396 return b

397 398 rotateLength = either(self.logRotateLength, self.master.log_rotation.rotateLength) 399 maxRotatedFiles = either(self.maxRotatedFiles, self.master.log_rotation.maxRotatedFiles) 400 401 # Set up the jinja templating engine. 402 if self.revlink: 403 revlink = self.revlink 404 else: 405 revlink = self.master.config.revlink 406 self.templates = createJinjaEnv(revlink, self.changecommentlink, 407 self.repositories, self.projects) 408 409 if not self.site: 410 411 class RotateLogSite(server.Site): 412 def _openLogFile(self, path): 413 try: 414 from twisted.python.logfile import LogFile 415 log.msg("Setting up http.log rotating %s files of %s bytes each" % 416 (maxRotatedFiles, rotateLength)) 417 if hasattr(LogFile, "fromFullPath"): # not present in Twisted-2.5.0 418 return LogFile.fromFullPath(path, rotateLength=rotateLength, maxRotatedFiles=maxRotatedFiles) 419 else: 420 log.msg("WebStatus: rotated http logs are not supported on this version of Twisted") 421 except ImportError, e: 422 log.msg("WebStatus: Unable to set up rotating http.log: %s" % e) 423 424 # if all else fails, just call the parent method 425 return server.Site._openLogFile(self, path)

Source Code for Module buildbot.status.web.baseweb