Recognizing colors/patterns in webpage

Question

I want to try to create a learning chess application as a school project. My first plan was to simply pit this AI against itself, but to really show if it has been succesful it needs to be able to show how well it progresses. In order to do this, i want it to play rated games on sites such as chess.com. However, they do not (yet) have a public API, i believe.

Therefore, i wanted to make a program in java that recognizes colors and images. It keeps an internal 2-dimensional array of all the positions, and recognizes the pieces on the board. I think i have found a way to do this in a window using something like the Java Robot Class.

What i would like it to do, however, is to open this webpage in an internal window and keep doing this in the background. Is there a way to recognize colors within the own window, without needing to be in the foreground?

Edit: I'm planning on using this browser component i just found. I noticed that it is possible to create a full-page snapshot of the page and save it as a BufferedImage(?). Would this make it easier to do this?

Edit 2: I just read that 'Outside assistance from other people, computers/chess engines, or endgame tablebases is entirely prohibited'. I suppose letting a computer do all the playing does certainly include in that. So i might try using another site, so answers that are specific for chess.com won't cut it!

Answer 1

I don't know it it helps but may be you can have a look at the Sikuli project. http://sikuli.org/

Sikuli is a program (and an API) to handle the interactions with the User Interface. For instance, you can write a script to click on an image or a button in certain conditions.

Especially interesting for you, there is a Java integration: http://sikuli.org/docx/faq/030-java-dev.html

Here is an extract of the website to give you an idea of the code you can write.

EDIT: in this code it is important to notice that you are defining new Patterns with the images. Sikuli will be able to find matching patterns.

import org.sikuli.script.*;

public class TestSikuli {

    public static void main(String[] args) {
            Screen s = new Screen();
            try{
                    s.click("imgs/spotlight.png", 0);
                    s.wait("imgs/spotlight-input.png");
                    s.type(null, "hello world\n", 0);
            }
            catch(FindFailed e){
                    e.printStackTrace();
            }
    }
}