How do you zoom into a specific point (no canvas)?

The goal is simple, using a mousewheel, zoom into a specific point (where the mouse is). This means after zooming the mouse will be in the same roughly the same spot of the picture.

(Purely illustrative, I don't care if you use dolphins, ducks or madonna for the image)

I do not wish to use canvas, and so far I've tried something like this:

HTML

<img src="whatever">

JS

function zoom(e){
    var deltaScale = deltaScale || -e.deltaY / 1000;
    var newScale = scale + deltaScale;
    var newWidth = img.naturalWidth * newScale;
    var newHeight = img.naturalHeight * newScale;
    var x = e.pageX;
    var y = e.pageY;
    var newX = x * newWidth / img.width;
    var newY = y * newHeight / img.height;
    var deltaX = newX - x;
    var deltaY = newY - y;
    setScale(newScale);
    setPosDelta(-deltaX,-deltaY);
}

function setPosDelta(dX, dY) {
    var imgPos = getPosition();
    setPosition(imgPos.x + dX, imgPos.y + dY);
}

function getPosition() {
    var x = parseFloat(img.style.left);
    var y = parseFloat(img.style.top);
    return {
        x: x,
        y: y
    }
}

function setScale(n) {
    scale = n;
    img.width = img.naturalWidth * n;
    img.height = img.naturalHeight * n;
}

What this attempts to do is calculate the x,y coordinates of the dolphin's eye before and after the zoom, and after calculating the distance between those two points, substracts it from the left,top position in order to correct the zoom displacement, with no particular success.

The zoom occurs naturally extending the image to the right and to the bottom, so the correction tries to pull back to the left and to the top in order to keep the mouse on that damn dolphin eye! But it definitely doesn't.

Tell me, what's wrong with the code/math? I feel this question is not too broad, considering I couldn't find any solutions besides the canvas one.

Thanks!

[EDIT] IMPORTANT

CSS transform order matters, if you follow the selected answer, make sure you order the transition first, and then the scale. CSS transforms are executed backwards (right to left) so the scaling would be processed first, and then the translation.

Solution

Here is an implementation of zooming to a point. The code uses the CSS 2D transform and includes panning the image on a click and drag. This is easy because of no change in scale.

The trick when zooming is to normalize the offset amount using the current scale (in other words: divide it by the current scale) first, then apply the new scale to that normalized offset. This keeps the cursor exactly where it is independent of scale.

var scale = 1,
    panning = false,
    xoff = 0,
    yoff = 0,
    start = {x: 0, y: 0},
    doc = document.getElementById("document");

function setTransform() {
  doc.style.transform = "translate(" + xoff + "px, " + yoff + "px) scale(" + scale + ")";
}

doc.onmousedown = function(e) {
  e.preventDefault();
  start = {x: e.clientX - xoff, y: e.clientY - yoff};    
  panning = true;
}

doc.onmouseup = function(e) {
  panning = false;
}

doc.onmousemove = function(e) {
  e.preventDefault();         
  if (!panning) {
    return;
  }
  xoff = (e.clientX - start.x);
  yoff = (e.clientY - start.y);
  setTransform();
}

doc.onwheel = function(e) {
    e.preventDefault();
    // take the scale into account with the offset
    var xs = (e.clientX - xoff) / scale,
        ys = (e.clientY - yoff) / scale,
        delta = (e.wheelDelta ? e.wheelDelta : -e.deltaY);

    // get scroll direction & set zoom level
    (delta > 0) ? (scale *= 1.2) : (scale /= 1.2);

    // reverse the offset amount with the new scale
    xoff = e.clientX - xs * scale;
    yoff = e.clientY - ys * scale;

    setTransform();          
}

html, body {
  margin: 0;
  padding: 0;
  width: 100%;
  height: 100%;
  overflow: hidden;
}

#document {
  width: 100%;
  height: 100%;
  transform-origin: 0px 0px;
  transform: scale(1) translate(0px, 0px);
}

<div id="document">
  <img style="width: 100%"
       src="https://i.imgur.com/fHyEMsl.jpg"
       crossOrigin="" />
</div>