Search code examples
androidfirebase-test-lab

Why Robo tests gets marked as passed so quickly?


TL;DR

The app has tons of flows, but sometimes runs get passed faster than 2 mins...


Is there any way to keep it running until the timeout period (e.g. 1hr) is almost consumed? Attached a screenshot for a quick termination e.g.

Fast passed e.g.

Although the app is very big with tons of flows, sometimes runs get passed after 2min, 5mins but what is the critieria which decides that the running robo test should terminate now with a passed result? any idea what makes the recorded graph decides to go to this node? n.b. I assumed it's the terminal node

terminated


Solution

  • Why Robo tests get marked as passed so quickly?

    It turns out that due to having a varying b.e. responses, the app journey gets changed. If there're 3 disconnected components (as in gif), what happens is sth like the app can start in any of the 3 flows resembling the 3 components. Which implies how long the journey will be

    Is there any way to keep it running until the timeout period (e.g. 1hr) is almost consumed

    Guiding the robo tests as explained here is a promising way to let journey bigger by following some sequence of actions which make the graph bigger

    What is the criteria which decides that the running robo test should terminate now with a passed result?

    Robo tests are simply applying flood fill on the app (as in gif). Where the graph nodes are represented by screens, e.g. onboarding screen, and edges are represented by actions, e.g. click on next button

    Recursive flood fill with 4 directions