Bug 446787 - Implementing a new brush smoothing algorithm simply by up sampling and window averaging
Summary: Implementing a new brush smoothing algorithm simply by up sampling and window...
Status: REPORTED
Alias: None
Product: krita
Classification: Applications
Component: Tools/Freehand (show other bugs)
Version: unspecified
Platform: unspecified Unspecified
: NOR wishlist
Target Milestone: ---
Assignee: Krita Bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-12-10 14:37 UTC by K024
Modified: 2021-12-10 14:38 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
simple and weighted smoothing unexpected behavior (33.00 KB, image/png)
2021-12-10 14:37 UTC, K024
Details

Note You need to log in before you can comment on or make changes to this bug.
Description K024 2021-12-10 14:37:24 UTC
Created attachment 144433 [details]
simple and weighted smoothing unexpected behavior

SUMMARY
The brush smoothing algorithms have been untouched for a LONG time. But as I see, they are all implemented in a very different way compare to some famous commercial paint tools and not that comfortable. So I spent some time watching the stabilizers in SAI2 and Affinity Photo and I prototyped a very simple algorithm by up sampling and window averaging. I find the algorithm works very similar to those mentioned above and has a lot of advantages. 
The prototype code can be found under https://github.com/K024/stabilizer and an online preview at https://stabilizer.vercel.app/

PROBLEM
The simple and weighted smoothing are generating strange strokes at the begging and end. To be move precise, it sometimes has unexpected hard (in pressure) beginning of the stroke, and also some influent turns at the beginning and also the end. You can check this behavior in the attachment, and also the video mentioned in https://bugs.kde.org/show_bug.cgi?id=414543#c8 (https://youtu.be/ThLW7-LdQQI?t=30 very clear at time 00:30~00:35).
The stabilizer doesn't have the problems mentioned above, but it really works in someway strange but hard to point out and also somehow very "laggy".

PROPOSAL
I'm not an expert in cpp and I prototyped this algorithm with html5 canvas and typescript. 
The key components are: up sampling, window averaging and interval resampling.
Up sampling:
This is just a workaround that I find the algorithm works better when sample rate reaches above 200, but the input rate is limited by the browser to about 60. I up sampled the input just by linear interpolation with fixed sample count between two samples. Maybe bezier interpolation works better, but I find it hard to define the control points.
Window averaging:
As the name implies, just average everything: coordinates, pressure, rotation and so on. The window size can be a parameter for user, similar to stabilizer. At start of a stroke, fill the window with the first sample. And when user releases, fill the window with the last sample once a time and emit an averaged sample each step until the window is totally filled with the last sample. This can be done in a single event handling process.
Interval resampling:
This is a trick that resamples the last sample at a stable interval (30ms). This resampling allows the brush catch up with the cursor, i.e. if you have a window size of 20 and resample interval of 30ms, and when you stop your cursor without releasing it, it will take 600ms for the brush to catch up. I find 30ms very suitable with a sample rate above 200. It means the resampling won't predominant normal samples (<5ms). 

ADVANTAGES
This method has only a prerequisite of a high sample rate. Anything can be smoothed in the same simple method by averaging. The most important thing is that this method really makes users feel cursor catching with perceptible smoothing. An average window is filled very quickly and resampling also helps catching up.

DISCUSSION
Maybe another method to implement this algorithm is by a fixed window B spine. But that's too much for prototyping.
The most important thing I think is to follow the sample rate from the input device and really draw something when an input event triggered.