Search code examples
performance-testingrptrational-performance-test

RPT error: fail verdict roll up

I am a newbie for RPT and performance testing.

I am using RPT trial version 8.7.1. i have recorded a simple test. What this recorded test is doing is that it simply logging in an eccommerce site, then browsing it to a item, then adding that item to shopping cart and then logging out.

I played back this test in a schedule having 5 users. Also i have attached a datapool having 5 rows(loginids and passwords) with the test.

After playback i opened the ecommerce site with the login ids mentioned in datapool manually to check whether test has added the items in shopping cart of respective login ids or not. The test has successfully added the items in shopping cart of each login id mentioned in datapool.

But when i saw the TEST LOG of rpt in eclipse. it shows '2 FAIL verdicts reported' (see atached figure 1.jpeg).

I have googled it to know the meaning of this but i could not find any useful reference. Also i checked in the IBM knowledge center site for this error but again i could not find any useful description.

Please see the attached screenhots.

Any help will be appreciated.

enter image description here enter image description here enter image description here

Solution

  • Firstly, "FAIL verdicts reported" is a generic summary message which indicates that a playback error occurred on at least one of the elements (HTTP request-response pair) in a page/transaction/test container. An example for a playback error is a response code mismatch for a specific element such as a request which received a 200 status code in the test, but got a 500 status code while playing back. So, the message "FAIL verdicts reported" can't tell us anything more about problems in the playback.

    To know more details on the requests that failed, you would have to look into the test log in detail. You would have to expand the test, followed by pages and elements to reach the failing element. You can examine the element's request, response and a browser view of the response using the Protocol Data view. (Protocol Data view is open by default, if you don't find it use Window->Show View->Protocol Data). You'll be able to get clues on what went wrong by comparing the content of the requests in the playback with the corresponding element in the test. If any of the dynamic parameters have to be correlated/substituted, they may need to be done so. RPT provides a "Comparison" view which could be handy for your analysis. To open this, right-click on the problematic element and say compare and select all among the options. Also, for user's convenience, each element in the Test Log contains a link that takes you to the corresponding element in the test.

    Based on your screenshot of the Execution event console, it appears that you have 2 errors for each of the virtual users. One is a failure to extract a reference variable and the other is a timeout error. The first step to rectify this would be to compare the request data of the requests where these happened with the corresponding element in the test.

    It is likely that you won't have all the required content in the test log because RPT doesn't log all events for all users in a schedule by default. It has to bumped up explicitly.

    Lastly, as general guidelines, it is recommended to playback the test itself first, preferably with the login id that was used for recording (no datapool) and iron out issues there. Next, substitute the login id from a datapool and try again. If that too goes fine, try playing back the test in a schedule with a minimal number of users, say 5-10. While we are in this mode of "test debugging", it would be a good idea to set the test log level to ALL/ALL/ALL so that we would have adequate information for resolving test issues. (Test log settings are in the Schedule options)