In Robot framework, how can I execute multiple of test cases in data driven method

Question

I have set of Test scenarios (say 10) which I would like to execute against different countries (say 3).

1. for loop is not preferred as execution time per scenario will be longer and each scenario Pass/Fail will have to be managed.
2. Create Keyword for each Test Scenario and call them per country.
   • this leads to 3 different robot file one per country with 10 Testcases for each scenario
   • Any new add/remove scenarios, will have to update 3 files
3. robot data driver template-based approach appears to support one Test scenario per robot file. Uses data file and dynamically execute one data entry as one testcase
   • This leads 10 robot file one per Test Scenario
   • Any new Test Scenario will be new robot file
   • Any way to include more Test scenario in robot data-driven approach
4. Any other approach you would suggest for iterative execution of scenario against data set where each iteration results are captured separately.

Answer 1

My first recommendation would be Templates with for loops. This way you do not have to manage failures, each iterations will be independent from the other. Every data set will be executed with the template. Note that if one iteration fails the whole test case will be marked as failed, but you will be able to check which iteration has failed.

Here is the code for the above example:

*** Variables ***
@{COUNTRIES}    USA    UK

*** Test Cases ***
Test Scenario 1
    [Template]    Test Scenario 1 Template
    FOR    ${country}    IN    @{COUNTRIES}
        ${country}
    END

Test Scenario 2
    [Template]    Test Scenario 2 Template
    FOR    ${country}    IN    @{COUNTRIES}
        ${country}
    END

Test Scenario 3
    [Template]    Test Scenario 3 Template
    FOR    ${country}    IN    @{COUNTRIES}
        ${country}
    END
    
*** Keywords ***
Test Scenario 1 Template
    [Arguments]    ${country}
    Log    ${country}
    Run Keyword If    $country == 'UK'    Fail    Simulate failure.

Test Scenario 2 Template
    [Arguments]    ${country}
    Log    ${country}
    Run Keyword If    $country == 'USA'   Fail    Simulate failure.

Test Scenario 3 Template
    [Arguments]    ${country}
    Log    ${country}

---

The other option is based on this answer and it generates test cases dynamically during run time. Only a small library that also acts as a listener is needed. This library can have a start_suite method that will be invoked and it will get the suite(s) as Python object(s), robot.running.model.TestSuite. Then you could use this object along with Robot Framework's API to create new test cases programmatically.

DynamicTestLibrary.py:

from robot.running.model import TestSuite

class DynamicTestLibrary(object):
    ROBOT_LISTENER_API_VERSION = 3
    ROBOT_LIBRARY_SCOPE = 'GLOBAL'
    ROBOT_LIBRARY_VERSION = 0.1

    def __init__(self):
        self.ROBOT_LIBRARY_LISTENER = self
        self.top_suite = None

    def _start_suite(self, suite, result):
        self.top_suite = suite
        self.top_suite.tests.clear() # remove placeholder test

    def add_test_case(self, name, keyword, *args):
        tc = self.top_suite.tests.create(name=name)
        tc.keywords.create(name=keyword, args=args)
        
    def add_test_matrix(self, data_set, test_scenarios):
        for data in data_set:
            for test_scenario in test_scenarios:
                self.add_test_case(f'{test_scenario} - {data}', test_scenario, data)

globals()[__name__] = DynamicTestLibrary

UPDATE for Robot Framework 4.0

Due to the backward incompatible changes (Running and result models have been changed) made in the 4.0 release the add_test_case function should be change as below if you are using version above 4.0.

def add_test_case(self, name, keyword, *args):
    tc = self.top_suite.tests.create(name=name)
     tc.body.create_keyword(name=keyword, args=args)

---

In the robot file add a Suite Setup in which you can call the Add Test Matrix keyword with the list of countries and test scenarios to generate a test case for each combination. This way there will be an individual test case for each country - test scenario pair while having everything in one single file.

test.robot:

*** Settings ***
Library        DynamicTestLibrary
Suite Setup    Generate Test Matrix

*** Variables ***
@{COUNTRIES}    USA    UK

*** Test Cases ***
Placeholder test
    [Documentation]    Placeholder test to prevent empty suite error. It will be removed from execution during the run.
    No Operation
    
*** Keywords ***
Generate Test Matrix
    ${test scenarios}=    Create List    Test Scenario 1    Test Scenario 2    Test Scenario 3
    DynamicTestLibrary.Add Test Matrix    ${COUNTRIES}    ${test scenarios}

Test Scenario 1
    [Arguments]    ${country}
    Log    ${country}
    Run Keyword If    $country == 'UK'    Fail    Simulate failure.

Test Scenario 2
    [Arguments]    ${country}
    Log    ${country}
    Run Keyword If    $country == 'USA'   Fail    Simulate failure.

Test Scenario 3
    [Arguments]    ${country}
    Log    ${country}

This will be the output on the console:

# robot --pythonpath . test.robot
==============================================================================
Test
==============================================================================
Test Scenario 1 - USA                                                 | PASS |
------------------------------------------------------------------------------
Test Scenario 2 - USA                                                 | FAIL |
Simulate failure.
------------------------------------------------------------------------------
Test Scenario 3 - USA                                                 | PASS |
------------------------------------------------------------------------------
Test Scenario 1 - UK                                                  | FAIL |
Simulate failure.
------------------------------------------------------------------------------
Test Scenario 2 - UK                                                  | PASS |
------------------------------------------------------------------------------
Test Scenario 3 - UK                                                  | PASS |
------------------------------------------------------------------------------
Test                                                                  | FAIL |
6 critical tests, 4 passed, 2 failed
6 tests total, 4 passed, 2 failed
==============================================================================