How can I run GITLAB CI automation tests in parallel using a dynamic array of tickers?

Question

so I am currently running some GITLAB CI automation tests that involve doing a bunch of calculations given the ticker information. I am running the tests individually since there is a lot of information that is outputted, so I don't want to have it all run within one GITLAB step. In addition, some of these tickers may fail if there is some outlier issue that may occur with the way I handle the data. The following is a snippet of some of the gitlab_ci.yml configurations that I have created so far.

automation_test1:
stage: test
tags:
  - market_cap
variables:
  - COSE_BASE=python3

script:
  - python stock_data_org.py AAPL

automation_test2:
stage: test
tags:
  - market_cap
variables:
  - COSE_BASE=python3

script:
  - python stock_data_org.py GOOG

automation_test3:
stage: test
tags:
  - market_cap
variables:
  - COSE_BASE=python3

script:
  - python stock_data_org.py BRK.A

automation_test4:
stage: test
tags:
  - market_cap
variables:
  - COSE_BASE=python3

script:
  - python stock_data_org.py AMZN

automation_test5:
stage: test
tags:
  - market_cap
variables:
  - COSE_BASE=python3

script:
  - python stock_data_org.py MSFT

I don't want to keep following this convention, since it requires me to write so many GITLAB steps for each ticker. In addition, I want to be able to use some sort of array, or a way that I can call these tests SEPARATELY from a ticker list. That way, I keep the result sets separate for all tickers, and I can look at them individually to see what data/errors were produced for that individual ticker alone. My question is how can I pass an array to run a test in parallel, and NOT to fail based on one of them failing. I was looking at parallel:matrix, but I am not sure how to pass a dynamic array that it would read through each time to call the GITLAB step. Any information would be greatly appreciated.

I was looking to parallel:matrix for GITLAB configurations but I am not sure of the syntax to be able to create what I am looking for.

Essentially I want something in the nature of this below:

tickers:
stage: test
tags:
  - market_cap
variables:
  - TICKERS=$(cat tickers.txt)

script:
  - |+
    for ticker in $TICKERS
    do 
       automation_test $ticker
    done
    

automation_test:
stage: test
depends:
 - ticker
tags:
  - market_cap
variables:
  - COSE_BASE=python3

script:
  - python stock_data_org.py $ticker

where the automation_test produces an individual output for each of the tickers, and it does NOT fail and exit on one of them throwing an error.

Answer 1

Using parallel:matrix: you can do something like this:

automation_tests:
  variables:
    COSE_BASE: python3
  parallel:
    matrix:
      TICKER: ["GOOG", "AAPL", "AMZN", "MSFT"] # etc.
  script:
    - python stock_data_org.py "$TICKER"

This will create several jobs that all run independently and in parallel. Outputs for each ticker are separate and the failure of one job will not affect the status or output of any of the other jobs.

---

As a side note, you may want to consider orienting your testing to use a test framework such as the builtin unittest library and/or pytest. Pytest, for example, can support parallel execution via the pytest-xdist .

For example, to test a Python CLI entrypoint, you might just write a simple stock_tests.py file like so:

from stock_data_org import main
from unittest.mock import patch
import pytest

TICKERS = ["AAPL", "AMZN", ] # etc... or generate this list dynamically

@pytest.mark.parametrize("ticker", TICKERS)
def test_main(ticker):
    with patch('sys.argv', ['stock_data_org.py', ticker]):
        result = main()
        assert result == 0

Then running pytest -v stock_tests.py will execute an independent test for each ticker and will provide success/failure independently. If you install the pytest-xdist plugin the test-cases can run in parallel.

So your gitlab yaml can just be something like this:

unittests:
  before_script:
    - python -m pip install --upgrade pytest pytest-xdist psutil
    # optionally also install any pytest plugins you want to use
  script:
    - python -m pytest -n auto stock_tests.py