Bug 467182

Summary: Cannot pan zoom area to certain locations when using KWin Zoom plugin with certain multi-monitor configurations
Product: [Plasma] kwin Reporter: Ritchie Frodomar <alkalinethunder>
Component: effects-variousAssignee: KWin default assignee <kwin-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: duha.bugs, jacobryanham, kde, nate
Priority: NOR Keywords: accessibility
Version: unspecified   
Target Milestone: ---   
Platform: Arch Linux   
OS: Linux   
Latest Commit: Version Fixed In: 6.0
Sentry Crash Report:

Description Ritchie Frodomar 2023-03-10 23:51:26 UTC
When using certain multi-monitor setups where the monitors aren't physically aligned with one another (e.x: different sizes, one monitor is physically higher than the other, etc.), it can be impossible to pan the KWin zoom area to small areas on the edges of certain monitors. For example, having two monitors where one on the left sits lower than one on the right, you won't be able to move the zoom area to see a small region at the top of the lower monitor.


STEPS TO REPRODUCE
1. Enable the Zoom plugin and set the movement mode to Push.
2. Plug two monitors into the device, arrange them such that the left monitor's top edge is lower than the right's.
3. Attempt to move the mouse to the top of the left monitor after moving it to the bottom of the left monitor.

OBSERVED RESULT

Although your cursor will move as expected, the zoom area will stop at a certain threshold based on how mis-aligned the monitors are. This means you won't be able to see that region on-screen since it'll be cut off by the edge of the zoom area.

EXPECTED RESULT

The zoom area should pan such that any region on the workspace may be visible, just like how it would if all displays were the exact same size and orientation and were perfectly aligned with one another.

SOFTWARE/OS VERSIONS
Operating System: Arch Linux 
KDE Plasma Version: 5.27.2
KDE Frameworks Version: 5.103.0
Qt Version: 5.15.8
Kernel Version: 6.2.2-zen1-1-zen (64-bit)
Graphics Platform: X11
Processors: 16 × AMD Ryzen 7 3700X 8-Core Processor
Memory: 62.7 GiB of RAM
Graphics Processor: NVIDIA GeForce RTX 3060 Ti/PCIe/SSE2

ADDITIONAL INFORMATION
I apologize in advance for both formatting issues and if this is a duplicate bug. I'm severely visually impaired which is why I rely on Zoom plugin in the first place. I did search for bugs related to multi-monitor and zoom panning but couldn't find any in the few dozen results I did read.
Comment 1 jacob ham 2023-05-21 20:39:17 UTC
This phenomenon happens because of an unfortunate reaction between the zoom desktop effect and the way that multimonitor is implemented. Plasma is not the only desktop to have it; I noticed the same thing on Windows (7 and 10, though I imagine it still exists today) as well.  When displays have a vertical physical offset Kwin renders black boxes corresponding to those offsets to make the whole frame a rectangle. These boxes also define the bounds for where the mouse can move the zoomed frame, because they lay beyond what would normally be usable screen space it would not normally make sense to allow the mouse or windows to move into these unseen spaces. Because the mouse can't move into the boxes and the Zoom effect scales the whole frame, trying to the displayed zoom region up to the lower monitor's top does suggest seemingly inaccessible areas. 

It is actually possible to access that screen portion, however:
1. Move the cursor to the uppermost offset screen region and push the zoom area to the top of the screen. 
2. Move the cursor back down (without moving the zoomed region) and push the region back towards the lower display.
3. Eventually, when you cross the virtual screen boundary that corresponds to the where the displays meet horizontally in the frame you'll see a black box occupy the topmost portion of your lower display. Below the box is the top of your lower screen, and you can push the zoom frame over the area you couldn't see before.

Alternatively, switching to proportional or centered mouse tracking lets you see all areas of the frame like you describe iun the expected behaviour section without doing the annoying maneuver outlined above.

Writing the above made me wonder why us blind users don't have a window manager that just creates an infinite plane that we can push around the viewport of, since zoom and magnification challenges the notion of physical display boundaries.
Comment 2 Tammes Burghard 2024-02-06 10:16:31 UTC
Acidic light made a video about this explaining it in great detail: https://yewtu.be/watch?v=3wH0wDqfF5g

And also I can reproduce this on the following system (so it is neither a problem with X11 nor Nvidia):
Operating System: Crystal Linux 
KDE Plasma Version: 5.93.0
KDE Frameworks Version: 5.249.0
Qt Version: 6.7.0
Kernel Version: 6.7.3-arch1-2 (64-bit)
Graphics Platform: Wayland
Processors: 8 × AMD Ryzen 7 4700U with Radeon Graphics
Memory: 30.7 GiB of RAM
Graphics Processor: AMD Radeon Graphics
Manufacturer: TUXEDO
Product Name: TUXEDO Aura 15 Gen1
Comment 3 Ritchie Frodomar 2024-02-06 17:50:52 UTC
I would like to point out that neither Proportional nor Centered mode are a usable workaround for the bug. They do technically prevent the bug from happening, but they also prevent me from being able to move the mouse out of the way of things I'm trying to read/interact with. 

I would also like to point out that this is, unlike what others have mentioned, not a bug on other desktops.

Windows treats the edges of the monitor you're currently on as the bounding box for the zoom area, even if Windows also renders multiple monitors as virtual screens on a giant virtual workspace. The Windows Magnifier has handled this properly ever since full-screen zoom was added in Windows 7.

macOS also handles things the same way Windows does. I think GNOME does as well, however there's a more severe rendering bug that makes my computer literally unusable. I've reported that months ago on the GNOME GitLab as well.

I've looked at the code for Push mode in Kwin desktop zoom. I would love to attempt a fix, however I have no idea what I'm doing in terms of building and testing a from-source Kwin build. 

<code>
case MouseTrackingPush: {
    // touching an edge of the screen moves the zoom-area in that direction.
    int x = cursorPoint.x() * zoom - prevPoint.x() * (zoom - 1.0);
    int y = cursorPoint.y() * zoom - prevPoint.y() * (zoom - 1.0);
    int threshold = 4;
    xMove = yMove = 0;
    if (x < threshold) {
        xMove = (x - threshold) / zoom;
    } else if (x + threshold > screenSize.width()) {
        xMove = (x + threshold - screenSize.width()) / zoom;
    }
    if (y < threshold) {
        yMove = (y - threshold) / zoom;
    } else if (y + threshold > screenSize.height()) {
        yMove = (y + threshold - screenSize.height()) / zoom;
    }
    if (xMove) {
        prevPoint.setX(std::max(0, std::min(screenSize.width(), prevPoint.x() + xMove)));
    }
    if (yMove) {
        prevPoint.setY(std::max(0, std::min(screenSize.height(), prevPoint.y() + yMove)));
    }
    xTranslation = -int(prevPoint.x() * (zoom - 1.0));
    yTranslation = -int(prevPoint.y() * (zoom - 1.0));
    break;
}
</code>

(Apologies, I don't know how to pre-format code in Bugzilla)

The bug is in the part of the code where it detects whether the mouse has hit an edge of the virtual screen. It allows the mouse cursor to go outside of your physical screen and into the norrmally-hidden black bar, but doesn't detect this happening. I'm aware you can work around it by moving the mouse to another monitor and continuing to pan, and this is what I currently do as described in the video.

A possible solution could be to track which physical screen the mouse is from the perspective of the zoom area, and find the edges of the screen that aren't adjacent to another screen. These would be the outer edges of the user's actual workspace. Edges that are adjacent to another screen are called inner edges.

Then, if the mouse reaches an inner edge, it's allowed to pass through to that screen without the zoom area panning. This lets you move your mouse to other physical screens.

If the mouise reaches an outer edge, however, then this is when you pan. This makes it so the mouse is always on one screen, and the user pans the zoom area by pushing against the edge of one of their outer monitors instead of the edge of the workspace itself. The mouse will never be able to reach those invisible black bars.
Comment 4 Bug Janitor Service 2024-02-12 04:24:39 UTC
A possibly relevant merge request was started @ https://invent.kde.org/plasma/kwin/-/merge_requests/5169
Comment 5 fanzhuyifan 2024-02-20 20:23:39 UTC
Git commit 86f0d9914e31a410ca9468ef0eca707d0a085d48 by Yifan Zhu, on behalf of Michael VanOverbeek.
Committed on 20/02/2024 at 20:23.
Pushed by fanzhuyifan into branch 'master'.

Fix zoom push mouse tracking on multi-monitor workspaces

Zoom push tracking now considers the layout of the user's monitors, accounting for situations where the monitor layout doesn't form a perfect rectangle. These changes help prevent the zoom area from being unable to reach certain areas of the workspace depending on which edge of which screen the user pushes against.

One known issue is that, if the mouse moves too quickly, the zoom area can sometimes imperfectly track the movement. It will look the same as the original bug (areas of the screen will appear to be cut off/unreachable), but moving the mouse in the opposite direction a tiny bit snaps the zoom area back to where it should be. 

@teams/qa Heads-up that I'm very blind, and this is the first time I've ever contributed to a KDE project. I've tested the changes on my system and they fix the bug, but I want to make sure I didn't break anything in the process.

M  +36   -13   src/plugins/zoom/zoom.cpp
M  +3    -0    src/plugins/zoom/zoom.h

https://invent.kde.org/plasma/kwin/-/commit/86f0d9914e31a410ca9468ef0eca707d0a085d48
Comment 6 fanzhuyifan 2024-02-20 21:07:51 UTC
Git commit dde3219c691b9d80a381a4df225ed50547d13bad by Yifan Zhu, on behalf of Michael VanOverbeek.
Committed on 20/02/2024 at 20:57.
Pushed by fanzhuyifan into branch 'Plasma/6.0'.

Fix zoom push mouse tracking on multi-monitor workspaces

Zoom push tracking now considers the layout of the user's monitors, accounting for situations where the monitor layout doesn't form a perfect rectangle. These changes help prevent the zoom area from being unable to reach certain areas of the workspace depending on which edge of which screen the user pushes against.

One known issue is that, if the mouse moves too quickly, the zoom area can sometimes imperfectly track the movement. It will look the same as the original bug (areas of the screen will appear to be cut off/unreachable), but moving the mouse in the opposite direction a tiny bit snaps the zoom area back to where it should be. 

@teams/qa Heads-up that I'm very blind, and this is the first time I've ever contributed to a KDE project. I've tested the changes on my system and they fix the bug, but I want to make sure I didn't break anything in the process.


(cherry picked from commit 86f0d9914e31a410ca9468ef0eca707d0a085d48)

M  +36   -13   src/plugins/zoom/zoom.cpp
M  +3    -0    src/plugins/zoom/zoom.h

https://invent.kde.org/plasma/kwin/-/commit/dde3219c691b9d80a381a4df225ed50547d13bad