how to get a rectangle of the element (text) located inside the picture using the latest Tesseract / Tess4j since getBoxText() is not working?
update :
getBoxText() was deprecated in v5.0 and cannot find an alternative according to the documentation Documentation
String extractedText = tesseract.doOCR(screenShot);
System.out.println("current snapshot from screen has the following test : " + extractedText);
// Check if target string exists
if (extractedText.contains(textElement)) {
System.out.println("Targeted element by Text found ! ");
BufferedImage bufferedImage = ImageIO.read(screenShot);
Rectangle textRectangle = tesseract.getBoxText(textElement); // => here ****
// Calculate midpoint coordinates
int x = textRectangle.x + textRectangle.width / 2;
int y = textRectangle.y + textRectangle.height / 2;
System.out.println("clicked on element location (x,y) coordinate in snapshot is : + " + x + "," + y);
Actions actions = new Actions(driver);
actions.moveToLocation(x, y).click().perform();
} else {
System.out.println("Targeted element by Text Not found !");
}
here is the code that works, using getSegmentedRegions() method
//take a Screenshot by emulator
File screenShot = this.driver.getScreenshotAs(OutputType.FILE);
BufferedImage bufferedImage = ImageIO.read(screenShot);
//get all text from the image snapshot
String extractedText = tesseract.doOCR(bufferedImage);
System.out.println("current snapshot from screen has the following test : " + extractedText);
// Check if target string exists if yes get it's coordinates
if (extractedText.contains(textElement)) {
System.out.println("Targeted element by Text found ! ");
List<java.awt.Rectangle> textRectangles = tesseract.getSegmentedRegions(bufferedImage, 2);
for (java.awt.Rectangle rectangle : textRectangles) {
// Extract recognized text for this region
String regionText = tesseract.doOCR(bufferedImage, rectangle);// Implement extraction logic
if (regionText.contains(textElement)) {
// Calculate midpoint coordinates
int x = rectangle.x + rectangle.width / 2;
int y = rectangle.y + rectangle.height / 2;
Actions actions = new Actions(driver);
actions.moveToLocation(x, y).click().perform();
System.out.println("clicked on element location (x,y) coordinate in snapshot is : + " + x + "," + y);
Thread.sleep(2000);
break;
}
}
} else {
System.out.println("Targeted element by Text Not found !");
}