I wish to trigger builders dynamically (depending of list of supported platforms of incoming project). There are virtual builders for this purposes. It's really works but there is a critical problem: when I trigger more virtual builds then number of free workers, other builds are skipped.
On image below 4 builds triggered with one active worker
On success we can see only two executed builds. Other are lost (not executed at all)
Again, for 1 free (active, not paused) worker only 2 (of 4) triggered builds was executed. For N free workers it will be N+1 executed builds. Reproduction 100%.
The code is pretty clear
class MultiplePlatformsBuilder(Trigger):
def __init__(self, *args, **kwargs):
...
def getSchedulersAndProperties(self):
sch = self.schedulerNames[0]
triggered_schedulers = []
counter = 0
# trigger scheduler for each requested platform
platforms_list = self.set_properties.get('buildable_platforms_list')
for pl in platforms_list:
counter += 1
props = dict()
props["virtual_builder_name"] = "virtual-" + pl
props["virtual_builder_description"] = "This is dynamic build " + str(counter)
triggered_schedulers.append([sch, props])
return triggered_schedulers
# trigger configuration
c['schedulers'].append(schedulers.Triggerable(name='dynamic_scheduler', builderNames=['factory1']))
factory.addStep(MultiplePlatformsBuilder(
schedulerNames=['dynamic_scheduler'],
set_properties={
'buildable_platforms_list': ["platform1", "platform2", "platform3", "platform4"]
# platforms_list_var: Property(platforms_list_var),
},
))
The problem took me several days. It turned out this is not a bug but feature named Collapsing Build Requests
All build requests with the same codebases will collapse if there are no free workers.
All I need is just disable this feature:
c['collapseRequests'] = False
I'm sure this should be mentioned in context of virtual builders because they all have same codebases but shouldn't be collapsed - this makes no sense