Search code examples
junitcirclecipython-behave

CircleCI is unable to read the JUnit xml generated by Behave while splitting the test by timings

Ok, so I am trying to split my Appium tests by their timing in CircleCI for running the test in parallel. My tests are written in behave(Python) and I am generating the JUnit XML file. Here is my config.yml file

version: 2.1
orbs:
  macos: circleci/[email protected]

jobs:
  example-job:      
macos:
  xcode: 13.4.1

parallelism: 4  

resource_class: large

steps:
  - checkout
  - run:
      name: Install appium server
      command: |
        sudo npm update -g
        sudo npm install -g appium
        sudo npm install -g wd
  - run:
      name: Start appium server
      command: appium --address localhost --port 4723
      background: true
       
  - run:
      name: Installing Dependencies
      command: pip3 install -r requirements.txt
  
  - run:
      name: Test application
      command: |
                TEST=$(circleci tests glob "features/featurefiles/*.feature" | circleci tests split --split-by=timings --timings-type=classname)
                echo $TEST
                behave $TEST --junit     
  
  - store_test_results:
      path: reports
      
  - store_artifacts:
      path: ./Logs
      destination: logs-file
  
  - store_artifacts:
      path: ./screenshots

workflows:
  example-workflow:
jobs:
  - example-job

When I am running the test, I am getting the error "No timing found for "features/featurefiles/XXX.feature" and it is splitting the test by the filename. It runs well but the split is not happening by the timing.

When the execution is done, I can see the data in the TESTS tab, also in the Timing Tab

enter image description here

enter image description here

I believe CircleCI is not able to read the JUnit file generated by Behave, it is searching for the different JUnit XML file. How can we make CircleCI read the JUnit file generated by Behave?

Solution

  • If anyone face such issue, please take a look at the classname in the JUnit report. There is a format in which CircleCI read the classname. In my case, the classname in the JUnit report was mentioned as

    features.featurefiles.Login.feature

    But the CircleCI was looking for the classname in below format

    features/featurefiles/Login.feature

    I had to write a utility to change the classname in the report once the execution was completed.Once it was done, CircleCI was able to read the timings.

    Hope it help someone :)