Bug 421295 - Krita zooms to the max and rotates wildly when using touch gestures with a Wacom Intuos Pro M on Linux
Summary: Krita zooms to the max and rotates wildly when using touch gestures with a Wa...
Status: RESOLVED FIXED
Alias: None
Product: krita
Classification: Applications
Component: Tablets (tablet issues are only very rarely bugs in Krita!) (show other bugs)
Version: git master (please specify the git hash!)
Platform: Compiled Sources Linux
: NOR normal
Target Milestone: ---
Assignee: Krita Bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-05-10 16:45 UTC by P. Varet
Modified: 2020-05-18 20:17 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
PyQt5 program to demonstrate the TouchPoint coordinate bug. (2.47 KB, text/x-python)
2020-05-10 16:45 UTC, P. Varet
Details
Patch fixing the issue. (8.86 KB, patch)
2020-05-14 00:49 UTC, P. Varet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description P. Varet 2020-05-10 16:45:22 UTC
Created attachment 128340 [details]
PyQt5 program to demonstrate the TouchPoint coordinate bug.

SUMMARY

Context: this issue happens with a Wacom Intuos Pro M tablet on (at least) Ubuntu Linux 19.10 and 20.04, using (at least) Krita 4.2.9 and Krita from git at head.

The zoom/rotate touch features randomly misbehaves, and zooms to the max (8000% or so) while rotating to a random angle. This happens mostly at the beginning of gestures, when both fingers come into contact with the tablet, and to a lesser extent at the end of the gesture, when at least one of the fingers leaves the tablet.

This happens both in the flatpak package (4.2.9.0 from flathub) and git at head. As to drivers, package xserver-xorg-input-wacom is at version 1:0.39.0-0ubuntu1 and package libwacom2 is at version 1.3-2ubuntu1.

Qt is at version 5.12.8+dfsg-0ubuntu1 for the version of Krita compiled from git master. KDE is at version runtime/org.kde.Platform/x86_64/5.14 (I don't know which Qt version is in there but I'm assuming 5.14 too) for the flatpak install of Krita.



STEPS TO REPRODUCE

1. Plug in a Wacom Intuos Pro M (or presumably any touch-enabled Wacom tablet of that range?)

2. Configure the tablet to report touches instead of converting them to gestures itself (for instance with "xsetwacom --set "Wacom Intuos Pro M (WL) Finger touch" Gesture off" in a terminal). Tablet is also in absolute mode.

3. Start Krita from git master on Linux.

4. Start a new image. Perhaps draw a few scribbles for a better visualization of the problem.

5. Try to zoom/rotate by a few percents with a two finger gesture.



OBSERVED RESULT

The picture zooms to the maximum and rotates to a random angle.


EXPECTED RESULT

The picture zooms/rotates by a few percents.


SOFTWARE/OS VERSIONS

Linux/KDE Plasma: Ubuntu 19.10 and 20.04 using Cinnamon or i3 as the DE.
KDE Plasma Version: N/A.
KDE Frameworks Version: 5.12 and 5.14.
Qt Version: 5.12 (and probably 5.14).



ADDITIONAL INFORMATION

The bug here seems to be that Qt feeds spurious TouchEvent positions to Krita at the beginning and end of touches and Krita fails to filter them out.

Weirdly this happens with TouchPoint.pos() and .screenPos(), but .normalizedPos() works as expected, except at the very end of the gesture when one of the TouchPoints is in state RELEASED, where that TouchPoint's coordinates are off by some distance no matter what type of coordinates is used.

In order to demonstrate this I wrote a pure Qt5 demo (in Python, sorry). You can find it attached to this bug, or here: https://pastebin.com/Zgi2UxEi

This demo shows the following noteworthy behaviors:

- When a two-finger touch event begins, one TouchPoint is in state PRESSED, and both TouchPoints report the same coordinates even if the fingers are quite apart on the tablet.

- Those initial coordinates are always integers, whereas they become floats once the TouchPoints start moving around. This suggests that the coordinates for TouchPoints in state PRESSED are the result of some internal Qt fudging, rather than direct reports from the hardware or its driver.

- The fact that both TouchPoints initially have the same coordinates probably explains the instantaneous maximum zoom.

- In following events, the TouchPoint that started off PRESSED remains at those incorrect coordinates so long as it's in state STATIONARY. Once it starts being reported as MOVED, it "jumps" to the correct coordinates.

- The TouchPoint that did not start off PRESSED does not demonstrate this jump; its reported position seems to remain reliable.

- Therefore it looks like the issue is that the state of the non-PRESSED TouchPoint is incorrectly copied to the PRESSED one somewhere during Qt's processing of the event.

- TouchPoints in state RELEASED also frequently report coordinates some distance off from their previous position. This feels like it's probably a distinct bug, but can be addressed at the same time.

- Replacing the .pos() method with .normalizedPos() fixes those issues except for the one pertaining to TouchPoints in state RELEASED.


Therefore:

The bug is not Krita's fault, but Krita probably shouldn't fully trust TouchPoint coordinates.



CONCLUSIONS

- TouchPoint.pos() cannot be trusted for a TouchPoint that has been in state PRESSED until that TouchPoint has reached state MOVED.
- No TouchPoint coordinates can be trusted for a TouchPoint that is in state RELEASED.


SUGGESTED WORKAROUND

- For the RELEASED TouchPoint issue, stop handling the event as soon as one of the TouchPoints reaches this state.
- For the PRESSED TouchPoint issue, there are two options.
  - Option 1: switch to .normalizedPos() instead of .pos(). This will require some fudging with the reported values, as the zoom gesture also handles panning and seems to expect coordinates that make sense for that purpose. This also doesn't preserve the aspect ratio of the gesture on the device, as normalized positions are floats in range [0, 1].
  - Option 2: ignore TouchPoints that have been in state PRESSED until they reach state MOVED. This may take multiple events.

I will try to write a patch in that light.
Comment 1 P. Varet 2020-05-14 00:49:41 UTC
Created attachment 128440 [details]
Patch fixing the issue.
Comment 2 P. Varet 2020-05-14 01:32:15 UTC
Pull request sent.
Comment 3 Halla Rempt 2020-05-14 08:08:46 UTC
Thanks! We're now checking whether it breaks stuff on Android... And I'll build the patch to check with my Intuos.
Comment 4 Halla Rempt 2020-05-16 09:08:27 UTC
Git commit fb114d7686ee81f7e3a6999287534e1e769102ae by Boudewijn Rempt, on behalf of P. Varet.
Committed on 16/05/2020 at 09:08.
Pushed by rempt into branch 'master'.

Fix touch-based rotation and zoom on Wacom tablets.

Turns out the TouchPoints in a QTouchEvent are not always trustworthy,
and we need to filter out situations where we can tell they are not
reporting the correct coordinates.

M  +2    -1    libs/ui/canvas/kis_tool_proxy.cpp
M  +36   -7    libs/ui/input/kis_rotate_canvas_action.cpp
M  +62   -22   libs/ui/input/kis_zoom_action.cpp

https://invent.kde.org/kde/krita/commit/fb114d7686ee81f7e3a6999287534e1e769102ae
Comment 5 Halla Rempt 2020-05-16 09:08:56 UTC
Git commit a855e03a2de2190cce98b171730f64245dee2a8c by Boudewijn Rempt, on behalf of P. Varet.
Committed on 16/05/2020 at 09:08.
Pushed by rempt into branch 'krita/4.3'.

Fix touch-based rotation and zoom on Wacom tablets.

Turns out the TouchPoints in a QTouchEvent are not always trustworthy,
and we need to filter out situations where we can tell they are not
reporting the correct coordinates.
(cherry picked from commit 2302229958c0b629def1bb37b3b15a09c4568962)

M  +2    -1    libs/ui/canvas/kis_tool_proxy.cpp
M  +36   -7    libs/ui/input/kis_rotate_canvas_action.cpp
M  +62   -22   libs/ui/input/kis_zoom_action.cpp

https://invent.kde.org/kde/krita/commit/a855e03a2de2190cce98b171730f64245dee2a8c
Comment 6 Dmitry Kazakov 2020-05-18 20:12:57 UTC
Git commit 42219dfd7dcf486a05910b58edcc479940e8e872 by Dmitry Kazakov, on behalf of P. Varet.
Committed on 18/05/2020 at 20:06.
Pushed by dkazakov into branch 'krita/4.3'.

Fixed canvas "jump" on finger touches.

Further touch handling fixes!

In https://invent.kde.org/graphics/krita/-/merge_requests/333#note_47474
@dkazakov observed that the canvas "jumps" randomly when the user's
fingers leave or touch the canvas.

This is due to the QTouchEvent with unreliable touch positions also
firing in the form of a QGestureEvent with equally dubious positions,
and that event bubbling up all the way to the parent
QAbstractScrollArea.

So we just prevent QGestureEvent events from bubbling up to the parent
QAbstractScrollArea, and call it a day.

M  +11   -0    libs/flake/KoCanvasControllerWidget.cpp
M  +2    -0    libs/flake/KoCanvasControllerWidget.h

https://invent.kde.org/graphics/krita/commit/42219dfd7dcf486a05910b58edcc479940e8e872
Comment 7 Dmitry Kazakov 2020-05-18 20:17:07 UTC
Git commit e4ac1fd2cbc38d28fc3d393ea4efcbaa381920b4 by Dmitry Kazakov, on behalf of P. Varet.
Committed on 18/05/2020 at 20:16.
Pushed by dkazakov into branch 'master'.

Fixed canvas "jump" on finger touches.

Further touch handling fixes!

In https://invent.kde.org/graphics/krita/-/merge_requests/333#note_47474
@dkazakov observed that the canvas "jumps" randomly when the user's
fingers leave or touch the canvas.

This is due to the QTouchEvent with unreliable touch positions also
firing in the form of a QGestureEvent with equally dubious positions,
and that event bubbling up all the way to the parent
QAbstractScrollArea.

So we just prevent QGestureEvent events from bubbling up to the parent
QAbstractScrollArea, and call it a day.

M  +11   -0    libs/flake/KoCanvasControllerWidget.cpp
M  +2    -0    libs/flake/KoCanvasControllerWidget.h

https://invent.kde.org/graphics/krita/commit/e4ac1fd2cbc38d28fc3d393ea4efcbaa381920b4