How to get text of a toaster message in mobile app using Appium

Question

I am trying to verify a toaster message in Android Mobile app but not able to get text of toaster message as it doesn't show in uiautomatorviewer. Got some information that by the help of OCR it can be done taking screenshots and fetching the text from that screenshots Can anyone help me out how to do this step by step using java in Appium project?

Answer 1

You can follow the information on the below links to install the Tesseract on your machine:

For Mac: http://emop.tamu.edu/Installing-Tesseract-Mac

For Windows: http://emop.tamu.edu/Installing-Tesseract-Windows8

After installing the TessEract on your machine you need to add the dependency of TessEract Java library in your project. If you are using Maven for it, adding below dependency will work:

    <dependency>
        <groupId>org.bytedeco.javacpp-presets</groupId>
        <artifactId>tesseract</artifactId>
        <version>3.04-1.1</version>
    </dependency>

Also the 'Step 3' which is mentioned by Ivan need not to be followed.

If you are using 'TestNG' the TessEract API needs to be initialised only once so instead of initialising it every time, as per your framework you can initialise it either in the 'BeforeTest' or 'BeforeSuite' or 'BeforeClass' method and accordingly close the API either in 'AfterTest' or 'AfterSuite' or 'AfterClass' method.

Below is the code that I have written to achieve it.

import static org.bytedeco.javacpp.lept.pixDestroy;
import static org.bytedeco.javacpp.lept.pixRead;
import java.io.File;
import java.io.IOException;
import org.apache.commons.io.FileUtils;
import org.bytedeco.javacpp.lept.PIX;
import org.bytedeco.javacpp.tesseract.TessBaseAPI;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.BeforeSuite;

    public class BaseTest {
    
    static TessBaseAPI api = new TessBaseAPI();
    
    @BeforeSuite
        public void beforeSuit() throws IOException {
    
            File screenshotsDirec = new File("target/screenshots");
    
            if (screenshotsDirec.exists())
                FileUtils.forceDelete(screenshotsDirec);
    
            FileUtils.forceMkdir(screenshotsDirec);
    
            System.out.println("Initializing TessEract library");
    
            if (api.Init("/opt/local/share", "eng") != 0) {
                System.err.println("Could not initialize tesseract.");
            }
    
        }
    
    public synchronized boolean verifyToastMessage(String msg)
                throws IOException {
            TakesScreenshot takeScreenshot = ((TakesScreenshot) driver);
    
            File[] screenshots = new File[5];
    
            for (int i = 0; i < screenshots.length; i++) {
                screenshots[i] = takeScreenshot.getScreenshotAs(OutputType.FILE);
            }
    
            String outText;
    
            Boolean isMsgContains = false;
    
            for (int i = 0; i < screenshots.length; i++) {
                PIX image = pixRead(screenshots[i].getAbsolutePath());
                api.SetImage(image);
                outText = api.GetUTF8Text().getString().replaceAll("\\s", "");
                System.out.println(outText);
                isMsgContains = outText.contains(msg);
                pixDestroy(image);
                if (isMsgContains) {
                    break;
                }
            }
    
            return isMsgContains;
    
        }
    
    @AfterSuite()
        public void afterTest() {
    
            try {
                api.close();
            } catch (Exception e) {
                api.End();
                e.printStackTrace();
            }
    
        }
    }

I would also like to add that writing tests to read and verify the Toast messages in this way is not very much reliable as in one of my tests this code successfully captures the Toast message while in another test it fails to capture the toast message because the capturing of the screenshots starts when the toast message disappears. That was the reason I tried to write this code very much efficiently. However that also does not serve the purpose.