Search code examples
jmetertaurus

taurus tests running out of order?

I am new to taurus testing. I have a set of tests inside of a taurus project. enter image description here

Within the scenario 1300_Azuresubscriptions.yaml I have the following list of labels:

0100-Authorization
1300_ListAzureSubscriptions
1310_CreateAzureSubscriptions
1320_UpdateAzureSubscriptions
1330_FetchAzureSubscription
1340_CreateAVWGateway
1341_CreateVirtualWanSite
1342_ListVirtualWanSites
1343_UpdateVirtualWanSite
1344_FetchVirtualWanSite
1345_DeleteVirtualWanSite
1346_DeleteAVWGateway
1350_DeleteAzureSubscription
1351_ListADSubscription
1352_CreateAzureSubscriptions
1353_FetchADSubscription
1354_ValidateADSubscriptions
1355_GetADGroups
1356_ADSyncConfigurations
1357_ADSync
1358_CheckADCLientCreation
1359_DeleteADSubscription
1360_CheckADCLientDeletion
1361_ListAzureSubscriptionsWithInvalidAuthHeader
1362_ListAzureSubscriptionsWithNoAuthHeader
1363_CreateAzureSubscriptionsWithInvalidAuthHeader
1364_CreateAzureSubscriptionsWithNoAuthHeader
1365_UpdateAzureSubscriptionsWithInvalidAuthHeader
1366_UpdateAzureSubscriptionsWithNoAuthHeader
1367_FetchAzureSubscriptionWithInvalidAuthHeader
1368_FetchAzureSubscriptionWithNoAuthHeader
1369_CreateAVWGatewayWithInvalidAuthHeader
1370_CreateAVWGatewayWithNoAuthHeader

However when I run bzt test-cases/1300_AzureSubscriptions.yaml, only the following list of labels are actually getting tested: 

0100-Authorization
1300_ListAzureSubscriptions
1310_CreateAzureSubscriptions
1320_UpdateAzureSubscriptions
1330_FetchAzureSubscription
1340_CreateAVWGateway
1341_CreateVirtualWanSite
1342_ListVirtualWanSites
1343_UpdateVirtualWanSite
1344_FetchVirtualWanSite
1345_DeleteVirtualWanSite
1346_DeleteAVWGateway
1350_DeleteAzureSubscription
1351_ListADSubscription
1352_CreateAzureSubscriptions
1353_FetchADSubscription
1354_ValidateADSubscriptions
1355_GetADGroups
1356_ADSyncConfigurations
1357_ADSync
1358_CheckADCLientCreation
1359_DeleteADSubscription
1363_CreateAzureSubscriptionsWithInvalidAuthHeader
1364_CreateAzureSubscriptionsWithNoAuthHeader
1365_UpdateAzureSubscriptionsWithInvalidAuthHeader
1366_UpdateAzureSubscriptionsWithNoAuthHeader
1367_FetchAzureSubscriptionWithInvalidAuthHeader
1368_FetchAzureSubscriptionWithNoAuthHeader
1369_CreateAVWGatewayWithInvalidAuthHeader
1370_CreateAVWGatewayWithNoAuthHeader

Why isn't it running the labels 1360-1362? I don't understand the problem. Any help would be greatly appreciated. Thanks!

Solution

  • The issue was the indentation in the yaml. I would recommend that if you're running these tests you'd better install a linter to check your yaml formatting. One tab off and the error is invisible. It just ignores the tests.