Search code examples
pythonfreebsdbuildbotpoudriere

Buildbot how to handle multiple builders


I have followed this tutorial to setup my buildbot server on FreeBSD 12.1:

https://www.digitalocean.com/community/tutorials/how-to-build-and-deploy-packages-for-your-freebsd-servers-using-buildbot-and-poudriere

Server is working and building with poudriere, after I change how to the tutorial handle the array of workers.

I adapted the tutorial instructions for my scenario as follows on file /var/buildbot-master/master.cfg:

. . .
####### SCHEDULERS

c['schedulers'] = []

# Forceful scheduler allowed for all builders
c['schedulers'].append(schedulers.ForceScheduler(
    name='force',
    builderNames=[builder.name for builder in c['builders']]))

# Watch ports tree for changes on given branch
c['schedulers'].append(schedulers.SingleBranchScheduler(
    name='sched-bulk-121amd64-2020Q2',
    change_filter=util.ChangeFilter(project='freebsd-ports', branch='branches/2020Q2'),
    builderNames=['bulk-121amd64-2020Q2']))
. . .

This line give error on startup:

builderNames=[builder.name for builder in c['builders']]))

I don't know pyhton, but sees to me it's an for to get all builder names in an array...

The error message is:

2020-09-16 15:33:18-0300 [-] twistd 20.3.0 (/usr/local/bin/python3.7 3.7.9) starting up.
2020-09-16 15:33:18-0300 [-] reactor class: twisted.internet.pollreactor.PollReactor.
2020-09-16 15:33:18-0300 [-] Starting BuildMaster -- buildbot.version: 2.7.0
2020-09-16 15:33:18-0300 [-] Loading configuration from '/var/buildbot-master/master.cfg'
2020-09-16 15:33:18-0300 [-] error while parsing config file:
        Traceback (most recent call last):
          File "/usr/local/lib/python3.7/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
            inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
          File "/usr/local/lib/python3.7/site-packages/twisted/python/context.py", line 122, in callWithContext
            return self.currentContext().callWithContext(ctx, func, *args, **kw)
          File "/usr/local/lib/python3.7/site-packages/twisted/python/context.py", line 85, in callWithContext
            return func(*args,**kw)
          File "/usr/local/lib/python3.7/site-packages/buildbot/config.py", line 170, in loadConfig
            self.basedir, self.configFileName)
        --- <exception caught here> ---
          File "/usr/local/lib/python3.7/site-packages/buildbot/config.py", line 128, in loadConfigDict
            execfile(filename, localDict)
          File "/usr/local/lib/python3.7/site-packages/twisted/python/compat.py", line 247, in execfile
            exec(code, globals, locals)
          File "/var/buildbot-master/master.cfg", line 103, in <module>
            builderNames=[builder.name for builder in c['builders']]))
        builtins.KeyError: 'builders'

2020-09-16 15:33:18-0300 [-] Configuration Errors:
2020-09-16 15:33:18-0300 [-]   error while parsing config file: 'builders' (traceback in logfile)
2020-09-16 15:33:18-0300 [-] Halting master.
2020-09-16 15:33:18-0300 [-] BuildMaster startup failed
2020-09-16 15:33:18-0300 [-] BuildMaster is stopped
2020-09-16 15:33:18-0300 [-] Main loop terminated.
2020-09-16 15:33:18-0300 [-] Server Shut Down.

The following part point me where make the change:

          File "/var/buildbot-master/master.cfg", line 103, in <module>
            builderNames=[builder.name for builder in c['builders']]))
        builtins.KeyError: 'builders'

2020-09-16 15:33:18-0300 [-] Configuration Errors:
2020-09-16 15:33:18-0300 [-]   error while parsing config file: 'builders' (traceback in logfile)
2020-09-16 15:33:18-0300 [-] Halting master.

So, instead check the array, I have hard coded the parameter:

builderNames=[builder.name for builder in c['builders']]))

To:

builderNames=['bulk-121amd64-2020Q2']))

Then buildbot start wihtout erros, and can run poudriere.

My question is:

What should be the correct syntax to use that for checking the array c['builders'] letting it dynamic instead hard coded?

Have tried a lot of examples from python tutorials without success.


Solution

  • In default master.cfg.sample, structure c['builders'] is creating after c['schedulers'], that is why it is undefined in your code.

    Try to put (move) BUILDERS block before SCHEDULERS block.