Smooth drawing with Apple Pencil

Question

I am developing a diagraming app for Apple Pencil using Qt-5.8/PyQt5 and am trying to get the pencil strokes as smooth as some of the other apps that I am seeing, namely Notability and PDF Expert. I patched Qt-5.8 to provide fast access to the floating-point coalesced and predicted UITouch data provided by Apple and my app code is fast and responsive, but the lines are still jittery (see screenshots):

Notability and PDF Expert both produce lines that maintain their sharpness at various levels of zoon, which suggests to me that they may be vectorized.

Anyone have a suggestion for smoothing out my painting? I am already painting at retina resolution and using the same 250Hz Apple Pencil data as they are. Is there a mathematical technique for smoothing a series of points, or some other trick out there?

Answer 1

Note taking apps tend to actually store and paint the drawings as vectors, which is why they are smooth. It also enables several cool features, like being able to select and move text around, change its color and style, it is also very efficient for storage and can be zoomed in or out without loss of resolution, compared to raster painting.

In some applications there is even a two step process, there is an initial smoothing taking place while drawing a specific glyph and another pass which takes place after you lift the pen and the glyph is considered finished.

Your code on the other hand looks very raster-y. There is a number of ways to simplify the input points, ranging from very simple to incredibly complex.

In your case what you could try is rather simple, and should work fine for the kind of usage you are aiming at.

You need to keep processing each stroke / glyph as the pen moves, and instead of adding every intermediate position to the stroke control points, you only add points that deviate from the current angle / direction above a certain threshold. It is conceptually a lot like the Ramer–Douglas–Peucker algorithm, but you don't apply it on pre-existing data points, but rather while the points are created, which is more efficient and better for user experience.

Your first data point is created when you put down the pen on the screen. Then you start moving the pen. You now have a second point, so you add that, but also calculate the angle of the line which the two points form, or the direction the pen is going. Then, as you move the pen further, you have a third point, which you check against the second point, and if the angle difference is not above the threshold, instead adding the third point you modify to extend the second point to that position effectively eliminating a redundant point. So you only end up creating points with deviate enough to form the rough shape of the line, and skip all the tiny little variances which create your jittery lines.

This is only the first step, this will leave you with a simplified, but faceted line. If you draw it directly, it will not look like a smooth curve, but like a series of line segments. The second step is point interpolation, probably regular old cubic interpolation will do just fine. You then get each actual position by interpolating between each set of 3 points, and draw the brush stroke at every brush spacing threshold. When interpolating the position, you also interpolate the brush pressure between the two points defining the currently drawn segment, which you must store along with each curve defining point. The pressure interpolation itself can be as simple as linear.