CSS transform breaking mouse position events

I'm scaling a div (zoom functionality) on page with non-scaled divs. The scaled div has mouseover events that cause text to follow the mouse. Scaling breaks the position of the element that should follow the mouse.

Hover text is done like:

$("#container").on('mousemove','.mouseMe',function(e){
    $("#followA").css("top",e.clientY)
                 .css("left",e.clientX);
});
//also some additional mouseenter/leave events are used to display hover

Scaling:

#container{
    transform-origin: top left;
    transform: scale(1.1,1.1);
}

I think what I need is to get the mouse's position on a css scaled div as if it wasn't scaled. (example: if the mouse is at the center of the div say [25,25], it should always return [25,25] even if the div is scaled). I could be wrong about what I need though, so the functional requirements take priority:

Element needs to follow mouse when hovered
Element container (or several containers up) needs to be scalable via css without breaking hovers (other transforms not relavant and no nested scaling)
JS, JQuery, CSS are all in use.
Chrome support is primary. Should also work in FF but not crucial. IE isn't supported.

This fiddle may explain this better and shows what doesn't work: http://jsfiddle.net/yvanaxe1/4/ (make the result pane big enough)

Solution

Is having those “follower” elements be descendants of the scaled element(s) an absolute requirement? Because, if you could take them out of there, and then simply position them over the top left edge of the mouseover-triggering element (by using the clientX/-Y values everywhere, plus some offsets to re-position them from there to appropriate distances), I think you might get there easier … http://jsfiddle.net/yvanaxe1/6/

I increased the scale value here, so that the effect on the follower elements (that the scaling has been applied to as well) is more obvious.