Converting between mouse and 2D "world," positions in webGL program

Question

I'm making a simple painting program with panning and zooming.

You can find a minimum reproduction of the issue here: https://codesandbox.io/s/mouse-coords-min-repro-4rfg1v?file=/src/index.ts

The issue is what's happening when I try to convert between mouse coordinates and "world," coordinates in calcMouseWorldPos(). Since the canvas is 300 units wide, each corner should be at exactly +/-160 (half of 320) in "world space," meaning if I were to successfully convert mouse coordinates to world space, hovering over each corner should print {x: +/-160, y: +/-160} to the console.

It currently works that way when I zoom in/out, but the second I start panning it breaks horribly. I'm coming from ThreeJS/Unreal background and trying to learn more ground level graphics programming and this one is making my head spin a bit. Shouldn't this just be a simple inverse-matrix problem? What am I doing wrong here?

EDIT: Here's the important bits in question

const vertexSource = `
uniform mat3 u_viewMatrix;

attribute vec2 a_position;
attribute vec2 a_uv;

varying vec2 vUv;

void main(){
  vec3 aspPos = vec3(a_position, 1.0) * u_viewMatrix;

  vUv = a_uv;

  gl_Position = vec4(aspPos, 1.0);
}

function updateViewMatrix(): Mat3 {
  const { x, y } = offset;
  const dimensions = new Vector(canvas.clientWidth / 2, canvas.clientHeight / 2);
  const zoomMat = new Mat3([zoom, 0, 0, 0, zoom, 0, 0, 0, 1]);
  const transMat = new Mat3([1, 0, x, 0, 1, y, 0, 0, 1]);
  const aspectMat = new Mat3([
      1.0 / dimensions.x, 0, 0,
      0, 1.0 / dimensions.y, 0,
      0, 0, 1.0
  ]);
  viewMatrix.copy(zoomMat.multiply(aspectMat).multiply(transMat));
  viewMatrixUniform.set(false, viewMatrix.data);
  return viewMatrix;
}
`;

function calcMouseWorldPos(event: MouseEvent): void {
  const mousePos = new Vector(event.offsetX, event.offsetY).multiplyScalar(devicePixelRatio);
  const windowDimensions = new Vector(canvas.width, canvas.height);

  //convert from pixels to clip space
  mousePos.divide(windowDimensions).subtractScalar(0.5).multiplyScalar(2);
  mousePos.y *= -1;

  //convert from clip space to world space
  mousePos.multiplyMat3(updateViewMatrix().clone().inverse());

  console.log(mousePos.toObject());
}

Answer 1

Original math for calculating mouse -> world-pos

clipSpace = ((mousePos / windowDimensions) - 0.5) * 2;
clipSpace.y *= -1;

worldSpace = clipSpace * viewMatrix.inverse()

by swapping out the last line with

worldSpace = clipSpace * viewMatrix.inverse().transpose()

everything works. I didn't even have a native transpose method in my matrix library yet, but decided to try it out as a Hail Mary last ditch effort, and it turns out that's what was needed. No idea why it was needed, as I've never seen it mentioned before and the output of my matrix library's inverse() method matches up with the output of all the online calculators, but I guess it was needed here for some reason.

Now I know, when in doubt, transpose!

Also, anyone reading this who understands matrices better than me, I'd still love to know why it was needed so I could better intuit situations like this in the future, a quick explanation would be fantastic.