Search code examples
macosautomationui-automationsikulisikuli-ide

How can I create and run SikuliX scripts on MacOS?


I'm trying to use Sikuli on MacOS to do some automation. I've downloaded the SikuliX IDE 2.0.4 from here, and Jython 2.7.2 from here, and put the two of them in the same directory as per instructions in the docs.

I'm running MacOS Catalina 10.15.7 and this is the Java version:

java version "1.8.0_281" 
Java(TM) SE Runtime Environment (build 1.8.0_281-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.281-b09, mixed mode)

I'm able to open the SikuliX IDE, however when I click on the "Take Screenshot" button all applications are minimized, including the ones I want to take a screenshot of, and that happens even if I open the application that I want to automate and use the SikuliX IDE shortcut to take a screenshot.

When this happens only the SikuliX menu bar, background and docker are visible. And even then, if I try to take a screenshot of a icon in the docker the image that opens up on the SikuliX IDE contains only the background, as if the docker is transparent.

I tried to take a screenshot using MacOS's default tools, but in this case when I try to run an instruction as simple as click() the script crashes.

This is the output of an attempt to click on the Finder icon in the docker:

enter image description here

[error] script [ create_new_folder ] stopped with error in line 1
[error] FindFailed ( finder_icon.png: (86x78) in R[0,0 1680x1050]@S(0) )
[error] --- Traceback --- error source first
line: module ( function ) statement 
1: main (  <module> )     click("finder_icon.png")
[error] --- Traceback --- end --------------

I wasn't able to find much stuff about SikuliX on MacOS. Any suggestions on how I might be able to take screenshots and automate things on MacOS with SikuliX?


Solution

  • Reason behind: in the standard macOS now blocks taking screenshots.

    Solution: This has to be allowed in the Prefs -> Security&Privacy -> Privacy -> ScreenRecording.