How to deal with IaC code (Infrastructure part of build pipeline) when the pipeline fails

Question

This is a general question that I have been having for couple of days now and after hours of searching google I am still not sure how it works.

Say I have a single pipeline to look for my IaC code change, deploy if there are any changes, and also then build the code and then deploy to the same infrastructure created in the step before.

So, it will look something like: Pipeline

Step1/stage 1: Look for changes in the IaC code (Terraform) and then deploy if there are any changes to .tf files

step2/stage2: Build the npm application

step3/stage3: Run the tests

step4/stage4: deploy the built application to the Infrastructure.

Now let's say the if the application fails to build (step2) or if the tests (step3) fail, how do we deal with the infrastructure rollback?

Answer 1

1. You can always deploy previous versions of your application in different release or build

2. You should have a quality ansurance environment before production environment so as to check if new changes will work

3. If you want to combine rollback deployment inside the same build you can use stage conditions to add new stage which will run only if previous stages fail

Check failed() condition and combine it with 'and', 'or' keywords

https://learn.microsoft.com/en-us/azure/devops/pipelines/process/stages?view=azure-devops&tabs=yaml#conditions

# stage B runs if A fails
- stage: B
  condition: failed()