Blueprism-like spying and bot development

Question

Blueprism gives the possibility to spy elements (like buttons and textboxes) in both web-browsers and windows applications. How can I spy (windows-based only) applications using Python, R, Java, C++, C# or other, anything but not Blueprism, preferrably opensource.

1. For web-browsers, I know how to do this, without being an expert. Using Python or R, for example, I can use Selenium or RSelenium, to spy elements of a website using different ways such as CSS selector, xpath, ID, Class Name, Tag, Text etc.
2. But for Applications, I have no clue. BluePrism has mainly two different App spying modes which are WIN32 and Active Accessibility. How can I do this type of spying and interacting with an application outside of Blueprism, preferrably using an opensource language?

(only interested in windows-based apps for now)

The aim is of course to create robots able to navigate the apps as a human would do.

Answer 1

I guess you are using selenium for web-browsers

There is also some projects for windows based applications working with a Windows Driver.

Take a look to the project on github, it may be what you are looking for.

https://github.com/2gis/Winium

https://github.com/microsoft/WinAppDriver