Search code examples
jmetertest-kitchenserverspec

TestKichen, Serverspec and out-of-order command execution

Inside TestKitchen describe blocks I'm running a command, loading its output into a variable then running multiple expect statements over that output validating different parts of it. The end goal is using this as part of CI builds to do blackbox testing.

In this instance I'm calling Jmeter (using it to run a remote agent to perform off-DUT tests) then running through the results that it returns checking each test (yeah yeah... it's a little nasty but it works a treat):

describe "Test Transparent Proxy (JMeter)" do
    $jmeter_run = command("/usr/local/apache-jmeter-2.13/bin/jmeter -n -t /root/jmx/mytest.jmx -r -Jremote_hosts=192.168.7.252 -Gdut_ip=#$internal_ip -X -l /dev/stdout 2>&1").stdout

    it 'test1' do
        expect($jmeter_run).to match /text_to_match/
    end
    it 'test2' do
        expect($jmeter_run).to match /more_text to match/
    end
end

The tests themselves run fine, but I'm finding that I'm getting multiple jmeter runs (different test sets) being run out-of-order as to how they're defined in the test spec. I have other blocks that are being executed around the Jmeter tests. Here is my flow:

block 1
block 2
block 3 (Jmeter1)
block 4
block 5 (Jmeter2)

What I'm getting though is this:

block5
block3
block1
block2
block4

None of the documentation I've found seems to give me any clues as to how to avoid this. I don't want to put the command execution inside a should/expect chunk of its own as I want/need to be able to tell if an individual test has failed. I would also like to avoid running 50-odd individual Jmeter tests (they're about 5 secs each even with an avg of 20 tests in each run).

Help? :D

Solution

  • Well I managed to resolve this issue myself.

    After a lot of tinkering I ended up running the command inside a test:

      it 'JMeter executed correctly' do
    $jmeter_run1 = command("/usr/local/apache-jmeter-2.13/bin/jmeter -n -t /root/jmx/mytest.jmx -r -Jremote_hosts=192.168.7.252 -Gdut_ip=#$internal_ip -X -l /dev/stdout 2>&1").stdout
    expect($jmeter_run1).not_to be_empty
  end

    Everything now runs nicely in order like it is supposed to and everything is happy.