Bug 386954 - Pointer locking/confinement issues
Summary: Pointer locking/confinement issues
Status: RESOLVED FIXED
Alias: None
Product: kwin
Classification: Plasma
Component: input (show other bugs)
Version: 5.11.3
Platform: Arch Linux Linux
: NOR normal
Target Milestone: ---
Assignee: KWin default assignee
URL: https://phabricator.kde.org/D9647
Keywords:
Depends on:
Blocks:
 
Reported: 2017-11-15 17:49 UTC by Yannick
Modified: 2018-02-08 15:15 UTC (History)
0 users

See Also:
Latest Commit: https://commits.kde.org/kwin/6ab625ed9770feae1924f43969ea63c5ef5ef0f5
Version Fixed In:
mgraesslin: Wayland+
mgraesslin: X11-
mgraesslin: ReviewRequest+

Attachments
Pressing "escape" during a Neverball game (538.04 KB, image/png)
2017-11-15 17:49 UTC, Yannick 		Details
NetRadiant layout, with 3D and 2D views (517.44 KB, image/png)
2017-11-15 17:49 UTC, Yannick 		Details
View All Add an attachment

Note You need to log in before you can comment on or make changes to this bug.
Description Yannick 2017-11-15 17:49:04 UTC 
Created attachment 108880 [details]
Pressing "escape" during a Neverball game

Hello,

I gave the Plasma Wayland session a try and ran into some issues with pointer locking/confinement.
All games and apps in my tests ran through XWayland.

If this bug report should be split into several parts, please tell me :)

 - Neverball, a 3D game, works mostly fine in windowed mode, but when you press "escape" during a game, the mouse pointer is not usable right away. You have to press "Escape" during 3 seconds to unlock it and navigate in the game menu.
Screenshot in attachment.

 - I also tried Xonotic (an open-source FPS game) in fullscreen mode. When the game starts, it displays a menu. A Kwin OSD appears, saying that the pointer is confined to the current window. However, the in-game mouse pointer disappears on some areas of the screen and is almost impossible to use.
By opening the in-game console, I can however start a game session. Another OSD appears and states that the mouse pointer is locked to its current position. Most of the time it works just fine, it is perfectly possible to play and returning to the menu unlocks the in-game cursor, yay :)
However, sometimes the pointer isn't locked to its position and it is impossible to reliably move around in the game.

 - I also experienced an issue with NetRadiant, an open-source map editor for games based on Quake engines.
You can move around a 3D or 2D view of a map by right-clicking in the chosen view, then moving the mouse (see the attached screenshot).
A Kwin OSD states that the pointer is locked when you right-click on it, everything works fine in that regard.
However, when the right mouse button is released, the pointer is not unlocked, and you have to press "Escape" for 3 seconds. 
It is a big usability issue, because you often switch from one view to another.


On the Plasma X11 session, everything works fine, as there are no OSDs and no need to press "Escape" during 3 seconds to unlock the cursor.
Comment 1 Yannick 2017-11-15 17:49:49 UTC 
Created attachment 108881 [details]
NetRadiant layout, with 3D and 2D views
Comment 2 Yannick 2017-11-15 17:58:44 UTC 
I am aware that the issue with NetRadiant is not easily reproducible, because this application is quite specific and requires a special setup, sorry for that :(

But maybe other applications have the same usability issue (it's probably the case for other *radiant forks like GTKRadiant).

I tried Blender and it worked fine, though.
Comment 3 Martin Flöser 2017-11-15 20:08:06 UTC 
I have a few questions:
1. what's your exact xwayland version?
2. what is the screenshot of neverball supposed to show me?
Comment 4 Yannick 2017-11-15 20:13:25 UTC 
Hi,

1. I use xorg-server-xwayland 1.19.5
2. There should be a mouse cursor on the neverball screenshot
Comment 5 Yannick 2017-11-15 20:39:26 UTC 
Sorry, my bad:

I made another test with Neverball. When I press "Escape", the mouse pointer remains visible. It just won't move.

I noticed a few more things:
 - I can also unlock the pointer on Neverball by pressing Alt+Tab a few times.
 - As long as the OSD is visible, no game input is taken into consideration (for example, I can't use Escape to exit Neverball or Space/right click to jump into Xonotic action).
 - When I quit Xonotic, the mouse cursor remains locked on the desktop. Pressing Escape during 3 seconds unlocks it.
Comment 6 Martin Flöser 2017-11-15 20:52:26 UTC 
Ok, I just installed neverball to see what's going on. The main problem is that neverball acts on repeated key press events for escape key and do to that the breaking of the confinement doesn't work correctly.
Comment 7 Martin Flöser 2017-11-15 21:06:58 UTC 
the other problem is the locking. Well, that looks like an xwayland bug to me. Obviously the mouse doesn't move when locked. That's the idea behind it.
Comment 8 Yannick 2017-11-15 22:10:12 UTC 
I tried Neverball and Xonotic on a Weston session, and didn't have any of the problems I encountered with Plasma.

The pointer is correctly locked/unlocked on Weston.

I made sure to run both games with the X11 SDL backend (SDL_VIDEODRIVER=x11).
Comment 9 Martin Flöser 2017-11-16 16:20:15 UTC 
It might be that XWayland assumes a bug in the Wayland compositor. I read the documentation of pointer locking again and looked at the source code of XWayland and that doesn't fit. We have in the protocol:

"If the client is drawing its own cursor, it should update the position	hint to the position of its own cursor. A compositor *may* use this information to warp the pointer *upon unlock* in order to avoid pointer jumps."

So KWin can ignore this hint (and does so) and it should only use it on unlock. But XWayland has a xwl_pointer_warp_emulator_set_fake_pos which uses the hint described above. I'll further investigate, but I think XWayland is built against the behavior of Weston and not against the spec.
Comment 10 Martin Flöser 2018-01-02 21:01:41 UTC 
I found a potential issue in KWin. It might be that the OSD is stealing pointer events.
Comment 11 Martin Flöser 2018-01-03 20:16:52 UTC 
Possible patch at: https://phabricator.kde.org/D9647

I'm quite confident it fixes the issue, but I haven't tested yet. If you have a chance to test I would appreciate it.
Comment 12 Martin Flöser 2018-01-03 20:33:08 UTC 
Git commit 6ab625ed9770feae1924f43969ea63c5ef5ef0f5 by Martin Flöser.
Committed on 03/01/2018 at 20:15.
Pushed by graesslin into branch 'master'.

Ignore outputOnly QWindows for pointer and touch events

Summary:
The idea is to be output only, so also ignore for pointer and touch.
This is similar to the previous commit for keyboard events.

Reviewers: #kwin, #plasma

Subscribers: plasma-devel, kwin

Tags: #plasma

Differential Revision: https://phabricator.kde.org/D9647

M  +3    -0    input.cpp

https://commits.kde.org/kwin/6ab625ed9770feae1924f43969ea63c5ef5ef0f5
Comment 13 Yannick 2018-01-04 15:06:10 UTC 
I applied the patch on top of Kwin 5.11.5.
I'm not sure if it's the right thing to do, but building kwin-git and all its *-git dependencies would take ages on my computer.

I still had issues with Neverball and Xonotic. The pointer is not unlocked when I press Escape on Neverball, and is often not properly confined to the Xonotic in-game menu.
Comment 14 Yannick 2018-02-08 15:15:30 UTC 
The issue is still here with Plasma 5.12.0.

When I start Neverball with both the X11 and Wayland SDL backends, pressing "Escape" in the game will not unlock the cursor.
I have to Alt+tab out or press "Escape" during 3 seconds to unlock it.