Convert handwritten input to a collection of geometry on a canvas

Question

I am developing a Windows app that runs on computers with a pen input. I have a canvas object that receives an event every time the pen has moved to know where exactly it is on the canvas, the pressure on the pen, etc etc, and the canvas itself can display vector graphics (eg lines, curves, and other shapes).

How do I convert this pen movement into a vector on screen in real time, also accounting for pressure, tilt, etc? My goal is for it to feel like writing in other pen based note taking apps, like OneNote or Goodnotes.

How would I deal with the user drawing many strokes, such as writing long hand written text, where many lines (presumably separate objects) will have to be drawn, without damaging performance?

Answer 1

I would start by making it simple. Have a loop (requestAnimationFrame + a delta time variable) and on each tick get the current mouse position and draw a line from there to the last position, and repeat. Take into account when the mouse is not clicked (pen not on the surface).

Play with the delta time in the loop to have more or less resolution on your handwritten to vector transformation.

After experimenting with this you can start adding more features, like instead of just drawing the line, storing it as a vector in some structure. That will allow you to redraw the canvas, after zooming in our out, exporting a file, selecting objects, changing colors, etc…

This is a minimal example (try it in codepen):

const canvas = document.querySelector("canvas");
const ctx = canvas.getContext("2d");

let penIsDown = false;
let drawing = false;
let x,
  y = 0;
let lastTime = 0;

ctx.lineWidth = 10;
ctx.fillStyle = "white";
ctx.fillRect(0, 0, 600, 600);

canvas.addEventListener("mousedown", () => {
  penIsDown = true;
});

canvas.addEventListener("mouseup", () => {
  penIsDown = false;
});

canvas.addEventListener("mousemove", (e) => {
  x = e.clientX;
  y = e.clientY;
});

const loop = () => {
  const now = Date.now();
  const deltaTime = now - lastTime;
  if (deltaTime >= 10) {
    if (!penIsDown) {
      if (drawing) {
        ctx.stroke();
        drawing = false;
      } else {
        // nothing
      }
    } else {
      if (drawing) {
        ctx.lineTo(x, y);
        ctx.stroke();
      } else {
        ctx.moveTo(x, y);
        ctx.beginPath();
        drawing = true;
      }
    }
    lastTime = now;
  }
  requestAnimationFrame(loop);
};

loop();