Octopus deploy process: share data between steps

Question

Given a runbook/process with 3 steps

• step 1 and step 2 both write some JSON data to an output variable (JSON for an object)

Set-OctopusVariable -name "SharedData" -value ($sharedObject | ConvertTo-Json)

• step 2 and step 3 need to read and update the data from the previous steps

$OctopusParameters["Octopus.Action[StepA].Output.SharedData"]

$OctopusParameters["Octopus.Action[StepB].Output.SharedData"]

Assume that any secondary step can use the shared data object from any previous step. It's just an object that is manipulated by multiple steps along the way.

If I choose to skip step 2, then step 3 won't see the step 2 output var value because the read instruction requires the name of the step (StepA or StepB).

Is there a way for the output var read syntax to just get the value from the previous step instead of an explicitly named step? E.g.:

$OctopusParameters["Octopus.Action[previous step alias].Output.SharedData"]

I already tried doing this using the $OctopusParameters dictionary directly.

In one step:

$OctopusParameters["SharedData"] = ($sharedObject | ConvertTo-Json)

Then this in a subsequent step:

$sharedObject = $OctopusParameters["SharedData"] | ConvertFrom-Json

But it doesn't work. The dictionary read returns a null. The raw dictionary assignment isn't persisted between the steps. It only works using the provided Set-OctopusVariable helper or other prescribed methods, but those lock you into knowing the previous step name.

Alternatively, is there a way to store data more "globally" to a process execution for use later without the need to tie it specific to the output of another step of a process?

Answer 1

The way I approached this problem is by considering the use of the Dictionary $OctopusParameters to your advantage. As it's a dictionary, it has keys you can inspect. If you want to get the last variable with the same name, just iterate the keys, and get the last one.

e.g., Suppose you have a deployment process like this:

Step A has code like this:

$sharedObject = [PSCustomObject]@{
    StepName = "Step A";
    Value = "Value from Step A";
    Message = "Step A says Hello!";
};

Set-OctopusVariable -name "SharedData" -value ($sharedObject | ConvertTo-Json)

Whilst Step B has code like this:

$sharedObject = [PSCustomObject]@{
    StepName = "Step B";
    Value = "Value from Step B";
    Message = "Step B says Hello!";
};

Set-OctopusVariable -name "SharedData" -value ($sharedObject | ConvertTo-Json)

Finally, the last step checks for the existence of any Output variable ending in SharedData and then just iterate over each one to print the values to the log.

It then selects the last one, which is the important part. It does this so no matter which of Step A or Step B was skipped, it will always get the last one where the variable was set (you can obviously change this logic to suit your requirements)

$MatchingKeys = $OctopusParameters.Keys | Where-Object { $_ -match "^Octopus\.Action.*\.Output.SharedData$" }
Write-Highlight "Found $($MatchingKeys.Count) matching output variables"

foreach($matchingKey in $matchingKeys) {
    $OutputVariableValue = $OctopusParameters[$matchingKey]
    
    Write-Host "$matchingKey value: $OutputVariableValue"
}

Write-Host "Finding last value..."
$lastKey = $matchingKeys | Select-Object -Last 1

Write-Highlight "Last Match: $($OctopusParameters[$lastKey])"

You can also turn the above into a one-liner:

$JsonSharedData = $($OctopusParameters.Keys | Where-Object { $_ -match "^Octopus\.Action.*\.Output.SharedData$" } | Select-Object -Last 1 | ForEach-Object {$OctopusParameters[$_]})