Search code examples
buildbot

How can you "source" an environment inside a Buildbot step?


Within Buildbot I need to be able to "source" an environment before doing a compilation step.

If I was building the application from command line using bash I would have to do:

. envrionment-set-up-script
build_command

Within the build bot master.cfg file I have tried the following:

factory.addStep(ShellCommand(command=["source","environment-set-up-script"])
factory.addStep(ShellCommand(command=[".","environment-set-up-script"]))
factory.addStep(Configure(command=["source","environment-set-up-script"]))
factory.addStep(Configure(command=[".","environment-set-up-script"]))

All of which fail, this is because the command cannot be found, which makes sense as it is a bash builtin.

Also I do not think that this is the correct approach as the environment would not necessarily be used when the next step of the factory is called.


Solution

  • After some experimenting I have found a way in which to achieve this. You need to:

    • run a bash sub-shell setting the environment that should be used for that shell, i.e. call bash with the environment variable BASH_ENV set to the file that should be sourced into the environment.
    • run the env command in bash to capture the environment
    • parse the result of the env command into a property (using a SetProperty step)
    • use the property within further steps as the env parameter

    Note: that the environment should be parsed as a dictionary that can be used as an env parameter

        from buildbot.process.factory import BuildFactory
        from buildbot.steps.shell import ShellCommand, SetProperty
        from buildbot.process.properties import Property  
    
        def glob2list(rc, stdout, stderr):
            ''' Function used as the extrat_fn function for SetProperty class
                This takes the output from env command and creates a dictionary of 
                the environment, the result of which is stored in a property names
                env'''
            if not rc:
                env_list = [ l.strip() for l in stdout.split('\n') ]
                env_dict={ l.split('=',1)[0]:l.split('=',1)[1] for l in 
                              env_list if len(l.split('=',1))==2}
                return {'env':env_dict}
    
        #This is the equivalent of running source MyWorkdir/my-shell-script then  
        #capturing the environment afterwords.
        factory.addStep(SetProperty(command="bash -c env",
                    extract_fn=glob2list,       
                    workdir='MyWorkdir',
                    env={BASH_ENV':'my-shell-script' }))
    
        #Do another step with the environment that was sourced from 
        #MyWorkdir/my-shell-script
        factory.addStep(ShellCommand(command=["env"],
                    workdir="MyWorkdir",
                    env=Property('env')))