Detect Graphical Regions in an Image on a Web Page

I've been beating around the bush a lot, so I'll explain my problem here and hope with the whole picture, somebody has some ideas. With the following image:

Model with Regions

I need to detect a mouseover on the blobs over her eyes and mouth, and solve this problem in a general form. The model and blobs are on two different layers, so I can produce one image with only the blobs, and one with only the model, and somehow synchronise a virtual cursor over the blobs while it actually hovers over the model.

I can also make the blobs polygons, for hit testing, but I think a colour hit test would be much easier. If I hit blue, I am on her mouth and I show lipstick images; if I hit pink, I'm over her eyes, and display eye makeup images.

What are the suggestions and conversation of the learned ones here?

Solution

The simpler way to do it would be to load the layer image in a canvas, then get all its pixel data. When the mouse is hovering the model image, find out what color is currently selected and if it is different from a previous one, trigger an event to indicate that the selection has changed.

Here is an example, feel free to toy with it; but be aware that it doesn't handle all cases:

what if the layer and the model image are not the same size
what if the layer and the model image are not the same width/height ratio
what if you want to use some alpha channel (the example doesn't take it into account)

$(function() {

  /* we load all the image data first */
  var imageData = null;
  var layerImage = new Image();
  layerImage.onload = function() {
    var canvas = document.createElement("canvas");
    canvas.width = this.width;
    canvas.height = this.height;
    context = canvas.getContext('2d');
    context.drawImage(this, 0, 0, canvas.width, canvas.height);
    imageData = context.getImageData(0, 0, canvas.width, canvas.height).data;
  };

  /* it's easier to set the image data for example as base64 data */
  layerImage.src = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEAAAABACAYAAACqaXHeAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAOwgAADsIBFShKgAAAABp0RVh0U29mdHdhcmUAUGFpbnQuTkVUIHYzLjUuMTFH80I3AAAA5klEQVR4Xu3WMQ7CMAwF0IxcgCtw/xsGdWApItWHKorLQ8qESe2HbWjNiwABAgQIECBAgAABAgQIECBAgMAUgd4e/ehMSeTMh4wK2j/nqPjt/TNzm3IXgEFb64CdgBG44hKcsmg8pKDAvbf+7SlY7nvK3xb/+lx5BAA/jMCGpwOqC/z9CFT/ApfP/9Z6T87yBaUJJsVvsen9y8cDMAJ2gCWY7IHll1qaYFL84a9AelnF+CFwxYLSnAGMBFLNivE6QAcMBCq2dJqzETACRuCzQDpPFePTv9riCRAgQIAAAQIECBC4nsATagY67TVyuhAAAAAASUVORK5CYII=";

  var pColor = null;

  /* on mouse over the model image */
  $("#model").mousemove(function(event) {
    /* we correct the offset */
    var offset = $(this).offset();
    var relX = event.pageX - offset.left;
    var relY = event.pageY - Math.round(offset.top);

    /* and get the pixel values at this place (note we are not keeping the alpha channel; it's your decision whether or not it is valuable */
    var pixelIndex = relY * layerImage.width + relX;
    var dataIndex = pixelIndex * 4;
    var color = [imageData[dataIndex], imageData[dataIndex + 1], imageData[dataIndex + 2]];

    if (pColor == null) {
      /* we trigger when first entering the image */
      $(this).trigger("newColor", {
        message: "Initial layer color",
        data: color
      });
    } else if (pColor[0] != color[0] || pColor[1] != color[1] || pColor[2] != color[2]) {
      /* we trigger if the new position is a new color in the layer image */
      $(this).trigger("newColor", {
        message: "Changed layer color",
        data: color
      });
    }
    pColor = color;
  });

  /* some small help to convert rgb to css colors */
  function rgb2hex(red, green, blue) {
    var rgb = blue | (green << 8) | (red << 16);
    return '#' + (0x1000000 + rgb).toString(16).slice(1)
  }

  /* there you have the new layer color event management; for the example sake we change the color of some text */
  $("#model").on("newColor", function(event, eventData) {
    $("#selector").css("color", rgb2hex(eventData.data[0], eventData.data[1], eventData.data[2]));
  });
});

img {
  border: 1px solid silver
}

<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.0/jquery.min.js"></script>

<body>
  <h4>Model image</h4>

  <img id="model" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAEAAAABACAIAAAAlC+aJAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAAadEVYdFNvZnR3YXJlAFBhaW50Lk5FVCB2My41LjExR/NCNwAAAtdJREFUaEPtmS2PwkAQhhGQEASEBIECFAJBgoCAxpDgsWBxSP4lPwGJRCLv5m4uk6Xt7s5He9dLFst2+z7vfG3bxsc//zX+uf6PBPDXEUwRSBEwOpBSyGig+fIUAbOFxg3KjMD1em3EfqvV6vl8GkW7l5cJsF6vY/p//i8RQwbwer22221U5W63K/T4fr93u1263LdMFB8ZAEc96BsOhwER5WIIAMB+NO92u4lMKlw8m80oFJaMEgCg/ZADdvW0A2GMRiPdtgKAEu13tUJG4c667sQFoBap8yl8FdiPDIqyZgFQ9oerU83mlrV0ExaAJfshdKfTiSMLg8BZKR5k4ewPS4TpNplMOLKqAqD88YngSwxjVAVgyR+O8bSmKoCKumeerVoAkZe6xQnA45vCGH7rfGuIFbVRBUDgvO2bCefzWXEj4I8PDtyXOYzQ0cVi4WMonAmWSR8HIDWbzUZ33grXNKifTqcA3Ov1FNUfB3AdhdOv4h6+S0D6fr/HWLVaLd1jBgsAFNChV5RLAVpKelDfbrd16lk1QCLczLZjoPEW6SiMG4FAdUphXO/tCSkD8GEwi9uVDvZLny6gZg6HQ8YvDYBr23g8DnR9319S6XjH5XIJGw4Gg7fxZwwiFDfsyGfQSQeRvnci1ghE+XXz1d3W7bb5WVF3ABpzvpZVdwDM+8CYqyOAmzPRx6l6ARyPx3w/CNd9jQDgNEHq+RO6LgCgvt/vA8Dlcol2tjLnQPRmhW00n+W4rNPpPB6P6J5cAHofqnhlSffIAIDT8/k8n+jNZhO8l6qPHOYKPxlJ3+Wj1swpqJRzKPc06n6JIOf4GBmz1U778kpWxJmvQ9EjEOQergEX1KegcEnIAHAvPgaIRgDpMwO/jjUA/N1hJT3Hia7iL64c4KtRfP/4mkQrq9r3rVUngEBMUgQYCZtqIGRSSqGUQgwHUgoZTSrrQ3KhjN8oYiN/+PJPqpb83Htu7qcAAAAASUVORK5CYII="
  />

  <p>You are pointing at some <strong><span id="selector">color</span></strong>
  </p>

  <hr/>
  <h4>Layer image (reference only, not displayed in page)</h4>

  <img id="layer" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEAAAABACAYAAACqaXHeAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAOwgAADsIBFShKgAAAABp0RVh0U29mdHdhcmUAUGFpbnQuTkVUIHYzLjUuMTFH80I3AAAA5klEQVR4Xu3WMQ7CMAwF0IxcgCtw/xsGdWApItWHKorLQ8qESe2HbWjNiwABAgQIECBAgAABAgQIECBAgMAUgd4e/ehMSeTMh4wK2j/nqPjt/TNzm3IXgEFb64CdgBG44hKcsmg8pKDAvbf+7SlY7nvK3xb/+lx5BAA/jMCGpwOqC/z9CFT/ApfP/9Z6T87yBaUJJsVvsen9y8cDMAJ2gCWY7IHll1qaYFL84a9AelnF+CFwxYLSnAGMBFLNivE6QAcMBCq2dJqzETACRuCzQDpPFePTv9riCRAgQIAAAQIECBC4nsATagY67TVyuhAAAAAASUVORK5CYII="
  />
</body>