Search code examples
c#graphicsscreen-scrapinggraphical-interaction

Screen scraping an application window and interacting with the mouse and keyboard


The other day I found myself addicted to a Flash game and frustrated by the thing at the same time. In a moment of frustration with the game I thought I would make a 'bot' to beat it for me. Well, I really wouldn't, but it made me realize: I don't know how to interact with another application in a way to do this. Which brings me to the question, how would one take screenshots of another running application and interact with it with the keyboard and mouse. Ideally the solution would be in a managed language like c#.

When doing the background reading the net was drowning with articles on scraping HTML. There were not many articles on actually screen scraping an application.

I'm looking for a way to interface with another application rather than script/macro another application.

Could something like Xming be used to redirect the interface? http://www.straightrunning.com/XmingNotes/

Perhaps a Terminal Services client? http://www.codeproject.com/KB/cs/RemoteDesktop_CSharpNET.aspx


Solution

  • Check out Sikuli, it is basically what you are looking for. It is written in Java however.

    http://groups.csail.mit.edu/uid/sikuli/