Search code examples
javajmetercommand-line-interfacemicroserviceslatency

JMeter giving Error percentage in CLI but not in GUI

I have a small microservice running in a docker container that I am trying to obtain the latency of using JMeter CLI. I made the test using the JMeter GUI and ran the test from it first. And the error percentage then was 0. But when I run the same test using the CLI I get a value for error percentage. And I get the below warnings:

Creating summariser <summary>
Created the tree successfully using /Users/sulekahelmini/Documents/fyp/jmeter_scripts/factorial.jmx
Starting standalone test @ Tue Feb 18 02:20:35 IST 2020 (1581972635848)
Waiting for possible Shutdown/StopTestNow/HeapDump/ThreadDump message on port 4445
summary =     50 in 00:00:02 =   25.0/s Avg:    50 Min:     4 Max:   323 Err:     0 (0.00%)
Tidying up ...    @ Tue Feb 18 02:20:38 IST 2020 (1581972638095)
... end of run
2020-02-18 02:20:39,012 WARN o.a.j.u.JMeterUtils: Exception 'null' occurred when fetching String property:'sampleresult.default.encoding', defaulting to: ISO-8859-1
2020-02-18 02:20:39,022 WARN o.a.j.u.JMeterUtils: Exception 'null' occurred when fetching String property:'jmeterPlugin.prefixPlugins'
2020-02-18 02:20:39,022 INFO k.a.j.PluginsCMDWorker: Using JMeterPluginsCMD v. N/A
2020-02-18 02:20:39,025 INFO o.a.j.u.JMeterUtils: Setting Locale to en_LK
2020-02-18 02:20:39,030 INFO k.a.j.JMeterPluginsUtils: Loading user properties from: /Users/sulekahelmini/Documents/fyp/apache-jmeter-5.2.1/bin/user.properties
2020-02-18 02:20:39,030 INFO k.a.j.JMeterPluginsUtils: Loading system properties from: /Users/sulekahelmini/Documents/fyp/apache-jmeter-5.2.1/bin/system.properties
2020-02-18 02:20:40,083 WARN o.a.j.g.ObjectTableModel: Header count=13 but classes count=11
2020-02-18 02:20:40,083 WARN o.a.j.g.ObjectTableModel: Header count=13 but writeFunctor count=11
2020-02-18 02:20:40,210 INFO o.a.j.s.SaveService: Testplan (JMX) version: 2.2. Testlog (JTL) version: 2.2
2020-02-18 02:20:40,215 INFO o.a.j.s.SaveService: Using SaveService properties file encoding UTF-8
2020-02-18 02:20:40,217 INFO o.a.j.s.SaveService: Using SaveService properties version 5.0
2020-02-18 02:20:40,224 INFO o.a.j.s.SampleResult: Note: Sample TimeStamps are START times
2020-02-18 02:20:40,224 INFO o.a.j.s.SampleResult: sampleresult.default.encoding is set to ISO-8859-1
2020-02-18 02:20:40,225 INFO o.a.j.s.SampleResult: sampleresult.useNanoTime=true
2020-02-18 02:20:40,225 INFO o.a.j.s.SampleResult: sampleresult.nanoThreadSleep=5000
2020-02-18 02:20:40,226 INFO o.a.j.s.SampleEvent: List of sample_variables: []
2020-02-18 02:20:40,450 INFO k.a.j.v.AggregateReportGui: Saving CSV to /Users/sulekahelmini/Documents/fyp/fyp_work/MLscripts/agg_test.csv

This is the code that I used to run the test and convert the result to csv:

#run test
  cd /Users/sulekahelmini/Documents/fyp/apache-jmeter-5.2.1/bin && sh jmeter -n -t /Users/sulekahelmini/Documents/fyp/jmeter_scripts/factorial.jmx -l /Users/sulekahelmini/Documents/fyp/fyp_work/MLscripts/jmeter_results.jtl
  #convert result to csv
  cd /Users/sulekahelmini/Documents/fyp/apache-jmeter-5.2.1/bin && ./JMeterPluginsCMD.sh --generate-csv /Users/sulekahelmini/Documents/fyp/fyp_work/MLscripts/agg_test.csv --input-jtl /Users/sulekahelmini/Documents/fyp/fyp_work/MLscripts/jmeter_results.jtl --plugin-type AggregateReport

Shown below is the generated aggregated report (using the jtl file). Note that there is an error percentage there.

enter image description here

Also, note that my JMeter is not inside a container. What am I doing wrong here? How do I not get an error percentage when running in GUI?

Solution

  • I fail to see where did you get a value for error percentage as it seems to be zero:

    summary = 50 in 00:00:02 = 25.0/s Avg: 50 Min: 4 Max: 323 Err: 0 (0.00%)

    If you're seeing errors in the generated /Users/sulekahelmini/Documents/fyp/fyp_work/MLscripts/agg_test.csv it might be the case that the file existed previously and contains "old" results with failures.

    Moreover I think you're writing something into that file using Aggregate Report listener and later on appending the last results into the same file using JMeter Plugins Command Line Tool.

    In order to get "clean" and valid results:

    1. Remove /Users/sulekahelmini/Documents/fyp/fyp_work/MLscripts/agg_test.csv file
    2. Remove /Users/sulekahelmini/Documents/fyp/fyp_work/MLscripts/jmeter_results.jtl file
    3. Remove Aggregate Report listener (and all other listeners) from your Test Plan as listeners don't add any value and only consume resources
    4. Repeat your 2 commands in the terminal
    5. You should see the agg_test.csv file without any errors