How to download file from Firebase Test Lab device

Question

I am trying to download files from a Google Firebase Test Lab device. These files are generated from running instrumented tests. The files are screenshots. I need to download the screenshots because subsequent runs of the tests will compare screens to these screenshots. Thus the screenshots are a baseline for visual change detection.

I've tried looking for a way to connect to the Firebase Test Lab device's Device File Explorer. But there does not seem to be a way to access it remotely.

Since Firebase Test Lab cleans up the device after the test, I assume the only way I can achieve what I want to do is to add code to the test to push the files someplace.

One place I'd like to push the files to is the Google Cloud storage bucket where the test results are stored. From the documentation below, it seems like that is an intended purpose of the storage bucket:

After Test Lab completes testing of your app, you can review test results in the Firebase console or in a Google Cloud storage bucket in your project. You can also add a gsutil command to the shell command shown above to copy the test results data to your local computer. To learn more, see Analyzing Firebase Test Lab Results. (Firebase documentation)

For this approach, how would my instrumented tests push the files to the bucket? How does it know what bucket to use? What is the API to perform the file transfer?

Another approach I could take is Take Screenshots from Firebase Test Lab Instrumentation Tests That would get the screenshots into the bucket. But I see two potential problems.

The first problem is the screenshot library's screenshots might differ from the screenshots taken from UiAutomation.takeScreenshot(). In other words, the baseline screenshot is taken with screenshot library but on subsequent runs I would take the screenshot with UiAutomation.takeScreenshot() and compare them. Since the libraries are different, their screenshots might differ. A possible solution would be to take both screenshots with the screenshot library. But then I would need to figure out how to read the screenshot back into memory so I can perform my comparison operations. Today I read them in as Bitmaps.

For this approach, how would my instrumented tests load screenshot library's screenshots as Bitmaps?

A second problem is figuring out how to download the screenshots from the bucket to my local computer. The user interface for the bucket does not have a download button. It only has an upload button:

For this approach, how would I download files from the bucket to my local computer so I can add them to the resources directory that my instrumented test can read?

Answer 1

Alright, I hope I captured all your questions below:

By default all files from /sdcard/Screenshots are copied from the device to the GCS bucket you have access to. You can write your own files during the test into the same folder on the device. Test Lab will pick them up and copy them to the bucket.

If you want to download additional folders and download them locally, I can recommend flank which has an option called "directories-to-pull" to specify multiple directories. It already provides a lot of what you try to achieve by hand and more.

In case you don't want to use flank, gcloud has this option too, but puts the file into your GCS bucket. No download to local machine.

The Cloud Storage UI seems fairly limited. It allows you to download individual files, but not folders as a zip. Use gsutil to download files recursively:

gsutil -m cp -r gs://path/to/bucket/folder path/to/local/folder

By default Test Lab provides a GCS bucket location for you where it copies the files to, but you can use your own bucket and location e.g. using the gcloud client:

gcloud firebase test android run --results-bucket=gs://my-bucket --results-dir=path/to/folder