Search code examples
javaautomationsikuli

Click on specific drop down item which are used multiple times using sikuli

I am facing a problem using sikuli.

In the attached toolbar image, i have same drop down menu for three time for different purpose. Using sikuli i want to click on the second dropdown menu.

I am using the below code, but the problem is while running the code, it only click on the first drop down.

My code is:

Screen screen = new Screen();

    // Create object of Pattern class and specify the images path

    Pattern image = new Pattern(AppConstant.IMAGE_DIR+"toolbar.png");
    Pattern image2 = new Pattern(AppConstant.IMAGE_DIR+"import-button.png");
    Pattern image3 = new Pattern(AppConstant.IMAGE_DIR+"dropdown.png");
    //screen.wait(image.exact(), 10000);
    screen.find(image);
    screen.find(image2);
    screen.find(image3);

enter image description here

enter image description here

Any suggestion how to do this?

Thanks

Solution

  • In scenarios with multiple similar patterns, the best practice is to use the surrounding elements as a pivot. In your case, if you know that you have another unique element in the same area of the element you wish to click on, you can find that unique element first and then search for the element you actually need around the unique element.

    For example, in your case, you have the blue down arrow just next to the drop down menu button you need. So you can do something like this:

    ImagePath.setBundlePath("C:\\someDir\\sikulipatterns"); //This is to avoid supplying directory for each pattern
Screen screen = new Screen();

Pattern bigBlueArrowPattern = new Pattern("bigBlueArrow.png");
Pattern dropDownPattern = new Pattern("dropDownArrow.png");

Region bigBlueArrowPatternRegion = screen.find(bigBlueArrowPattern);
bigBlueArrowPatternRegion.grow(50).find(dropDownPattern).highlight(1);

    Here, 50 is the margin to be added around the region, so basically extending the region around the blue arrow. I included highlight(1) just to emphasize that the correct element has been located indeed but you should remove it and do whatever you wish with the found element.