How to limit concurrent pipeline job execution on an Azure agent?

Question

EDIT: It turns out I was experiencing rapid failures from an instances that still had a docker container running from the previous runs. It looked as though a parallel problem to me when it was in fact a dirty environment. My initial question is misleading in that it implies multiple jobs are run on agents. I no longer believe this to be the case.

I have the following needs in developing a pipeline on Azure Devops with our own instances running as agents:

1. I need stages to execute concurrently
2. Each stage needs to be guaranteed to execute on its own agent

I have #1 going. The problem is that I am running into resource and port contentions. I am outputting the build ID on stage runs and they often report the same agent:

steps:
- task: Bash@3
  displayName: Debug
  inputs:
    targetType: 'inline'
    script: |
      echo "Agent ID: $(Agent.Id)"

How do I make sure that each agent only runs one stage and/or job at a time?

Answer 1

My problem was that a docker container was being run manually in the pipeline. Since the machines persist, the container continued to run on the next pipeline. This caused port and name conflicts.

The solution is to manually stop the container at the end of the pipeline and set the step to always run regardless of failure of previous steps. I also defensively set a step that stops and removes the container if necessary before starting up the container.