Created attachment 170268 [details] strace - high cpu SUMMARY Recently I noticed that sometimes kwin_wayland eats a significant portion of my CPU. Normally it sits at around 2.5% (in top) but in those situation it might go up to 45%. Finally I was able to narrow it down to the use of some bash scripts that monitor and process the clipboard. Once terminated, the kwin_wayland CPU use goes down to normal. I use wl-paste in those scripts. The simplest one to reproduce would be something like (tested): #!/bin/bash while true; do sleep 0.1s clip=$(wl-paste -n -t text 2> /dev/null) done; Once run the kwin_waylands CPU use increases to about 25% (NOTE: this system is a measly J5005, might be much less on a modern system). STEPS TO REPRODUCE 1. run top to monitor the kwin_wayland 1. run the bash script above (or similar) 2. in top - observe increased cpu use 3. run strace on the kwin_wayland process ( strace -p <pid> -i ) and observe the weirdness 4. use strace to get stack trace on those readlink calls: strace -p <pid> -k --stack-trace-frame-limit=16 -e readlink --syscall-limit=100 5. and (probably) related getcwd calls: strace -p <pid> -k --stack-trace-frame-limit=16 -e getcwd --syscall-limit=100 OBSERVED RESULT 1. kwin_wayland cpu use goes way up 2. We start getting a very weird strace listing with entries like: [00007fe30811dd5b] readlink("/usr", 0x7ffc3483a350, 1023) = -1 EINVAL (Nepřípustný argument) [00007fe30811dd5b] readlink("/usr/bin", 0x7ffc3483a350, 1023) = -1 EINVAL (Nepřípustný argument) [00007fe30811dd5b] readlink("/usr/bin/syncthing", 0x7ffc3483a350, 1023) = -1 EINVAL (Nepřípustný argument) [00007fe30811c6cb] getcwd("/home/usrnm", 1024) = 12 [00007fe30811dd5b] readlink("/home/usrnm/lsp-plugins-para-equalizer-x16-stereo", 0x7ffc3483a350, 1023) = -1 ENOENT (Adresář nebo soubor neexistuje) [00007fe30811c6cb] getcwd("/home/usrnm", 1024) = 12 [00007fe30811dd5b] readlink("/home/usrnm/gnome-control-center", 0x7ffc3483a350, 1023) = -1 ENOENT (Adresář nebo soubor neexistuje) [00007fe30811c6cb] getcwd("/home/usrnm", 1024) = 12 [00007fe30811dd5b] readlink("/home/usrnm/qdirstat", 0x7ffc3483a350, 1023) = -1 ENOENT (Adresář nebo soubor neexistuje) [00007fe30811c6cb] getcwd("/home/usrnm", 1024) = 12 [00007fe30811dd5b] readlink("/home/usrnm/qvidcap", 0x7ffc3483a350, 1023) = -1 ENOENT (Adresář nebo soubor neexistuje) ... and many many more [ more in strace-high_cpu.txt ] 3. This raises many many questions,like : - Why does the kwin_wayland seem to be enumerating all the executables on my system? - Why does it trying to locate them (symlinked) in the base of my home directory (which is not even in the $PATH)? 4. Stack trace of the readlink calls might point us in the general direction: readlink("/usr/bin", 0x7ffc3483a350, 1023) = -1 EINVAL (Nepřípustný argument) > /usr/lib64/libc.so.6(readlink+0xb) [0x10ad5b] > /usr/lib64/libc.so.6(realpath@@GLIBC_2.3+0x3b7) [0x41df7] > /usr/lib64/libQt6Core.so.6.7.1(QFileSystemEngine::canonicalName(QFileSystemEntry const&, QFileSystemMetaData&)+0x94) [0x345314] > /usr/lib64/libQt6Core.so.6.7.1(QFileInfoPrivate::getFileName(QAbstractFileEngine::FileName) const+0x1fa) [0x14a43a] > /usr/lib64/libQt6Core.so.6.7.1(QFileInfo::canonicalFilePath() const+0xac) [0x14b83c] > /usr/lib64/libkwin.so.6.0.5(std::_Function_handler<bool (QExplicitlySharedDataPointer<KService> const&), KWin::fetchProcessServiceField(QString const&, QString const&)::{lambda(QExplicitlySharedDataPointer<KService> const&)#1}>::_M_invoke(std::_Any_data const&, QExplicitlySharedDataPointer<KService> const&)+0x114) [0x387eb4] > /usr/lib64/libKF6Service.so.6.2.0(applyFilter(QList<QExplicitlySharedDataPointer<KService> >&, std::function<bool (QExplicitlySharedDataPointer<KService> const&)>, bool)+0x113) [0x18c73] > /usr/lib64/libKF6Service.so.6.2.0(KApplicationTrader::query(std::function<bool (QExplicitlySharedDataPointer<KService> const&)>)+0x97) [0x1c847] > /usr/lib64/libkwin.so.6.0.5(KWin::KWinDisplay::allowInterface(KWin::ClientConnection*, QByteArray const&)+0x654) [0x38ab34] > /usr/lib64/libkwin.so.6.0.5(KWin::FilteredDisplayPrivate::globalFilterCallback(wl_client const*, wl_global const*, void*)+0x64) [0x4aa7f4] > /usr/lib64/libwayland-server.so.0.22.0(display_get_registry+0x9c) [0xbcbc] > /usr/lib64/libffi.so.8.1.2(ffi_call_unix64+0x55) [0x9055] > /usr/lib64/libffi.so.8.1.2(ffi_call_int.lto_priv.0+0x1ef) [0x569f] > /usr/lib64/libffi.so.8.1.2(ffi_call+0x12d) [0x84ed] > /usr/lib64/libwayland-server.so.0.22.0(wl_closure_invoke.constprop.0+0x172) [0x67e2] > /usr/lib64/libwayland-server.so.0.22.0(wl_client_connection_data+0xd1f) [0xb07f] > too many stack frames readlink("/usr/bin/syncthing", 0x7ffc3483a350, 1023) = -1 EINVAL (Nepřípustný argument) > /usr/lib64/libc.so.6(readlink+0xb) [0x10ad5b] > /usr/lib64/libc.so.6(realpath@@GLIBC_2.3+0x3b7) [0x41df7] > /usr/lib64/libQt6Core.so.6.7.1(QFileSystemEngine::canonicalName(QFileSystemEntry const&, QFileSystemMetaData&)+0x94) [0x345314] > /usr/lib64/libQt6Core.so.6.7.1(QFileInfoPrivate::getFileName(QAbstractFileEngine::FileName) const+0x1fa) [0x14a43a] > /usr/lib64/libQt6Core.so.6.7.1(QFileInfo::canonicalFilePath() const+0xac) [0x14b83c] > /usr/lib64/libkwin.so.6.0.5(std::_Function_handler<bool (QExplicitlySharedDataPointer<KService> const&), KWin::fetchProcessServiceField(QString const&, QString const&)::{lambda(QExplicitlySharedDataPointer<KService> const&)#1}>::_M_invoke(std::_Any_data const&, QExplicitlySharedDataPointer<KService> const&)+0x114) [0x387eb4] > /usr/lib64/libKF6Service.so.6.2.0(applyFilter(QList<QExplicitlySharedDataPointer<KService> >&, std::function<bool (QExplicitlySharedDataPointer<KService> const&)>, bool)+0x113) [0x18c73] > /usr/lib64/libKF6Service.so.6.2.0(KApplicationTrader::query(std::function<bool (QExplicitlySharedDataPointer<KService> const&)>)+0x97) [0x1c847] > /usr/lib64/libkwin.so.6.0.5(KWin::KWinDisplay::allowInterface(KWin::ClientConnection*, QByteArray const&)+0x654) [0x38ab34] > /usr/lib64/libkwin.so.6.0.5(KWin::FilteredDisplayPrivate::globalFilterCallback(wl_client const*, wl_global const*, void*)+0x64) [0x4aa7f4] > /usr/lib64/libwayland-server.so.0.22.0(display_get_registry+0x9c) [0xbcbc] > /usr/lib64/libffi.so.8.1.2(ffi_call_unix64+0x55) [0x9055] > /usr/lib64/libffi.so.8.1.2(ffi_call_int.lto_priv.0+0x1ef) [0x569f] > /usr/lib64/libffi.so.8.1.2(ffi_call+0x12d) [0x84ed] > /usr/lib64/libwayland-server.so.0.22.0(wl_closure_invoke.constprop.0+0x172) [0x67e2] > /usr/lib64/libwayland-server.so.0.22.0(wl_client_connection_data+0xd1f) [0xb07f] > too many stack frames readlink("/home/usrnm/lsp-plugins-para-equalizer-x16-stereo", 0x7ffc3483a350, 1023) = -1 ENOENT (Adresář nebo soubor neexistuje) > /usr/lib64/libc.so.6(readlink+0xb) [0x10ad5b] > /usr/lib64/libc.so.6(realpath@@GLIBC_2.3+0x3b7) [0x41df7] > /usr/lib64/libQt6Core.so.6.7.1(QFileSystemEngine::canonicalName(QFileSystemEntry const&, QFileSystemMetaData&)+0x94) [0x345314] > /usr/lib64/libQt6Core.so.6.7.1(QFileInfoPrivate::getFileName(QAbstractFileEngine::FileName) const+0x1fa) [0x14a43a] > /usr/lib64/libQt6Core.so.6.7.1(QFileInfo::canonicalFilePath() const+0xac) [0x14b83c] > /usr/lib64/libkwin.so.6.0.5(std::_Function_handler<bool (QExplicitlySharedDataPointer<KService> const&), KWin::fetchProcessServiceField(QString const&, QString const&)::{lambda(QExplicitlySharedDataPointer<KService> const&)#1}>::_M_invoke(std::_Any_data const&, QExplicitlySharedDataPointer<KService> const&)+0x114) [0x387eb4] > /usr/lib64/libKF6Service.so.6.2.0(applyFilter(QList<QExplicitlySharedDataPointer<KService> >&, std::function<bool (QExplicitlySharedDataPointer<KService> const&)>, bool)+0x17b) [0x18cdb] > /usr/lib64/libKF6Service.so.6.2.0(KApplicationTrader::query(std::function<bool (QExplicitlySharedDataPointer<KService> const&)>)+0x97) [0x1c847] > /usr/lib64/libkwin.so.6.0.5(KWin::KWinDisplay::allowInterface(KWin::ClientConnection*, QByteArray const&)+0x654) [0x38ab34] > /usr/lib64/libkwin.so.6.0.5(KWin::FilteredDisplayPrivate::globalFilterCallback(wl_client const*, wl_global const*, void*)+0x64) [0x4aa7f4] > /usr/lib64/libwayland-server.so.0.22.0(display_get_registry+0x9c) [0xbcbc] > /usr/lib64/libffi.so.8.1.2(ffi_call_unix64+0x55) [0x9055] > /usr/lib64/libffi.so.8.1.2(ffi_call_int.lto_priv.0+0x1ef) [0x569f] > /usr/lib64/libffi.so.8.1.2(ffi_call+0x12d) [0x84ed] > /usr/lib64/libwayland-server.so.0.22.0(wl_closure_invoke.constprop.0+0x172) [0x67e2] > /usr/lib64/libwayland-server.so.0.22.0(wl_client_connection_data+0xd1f) [0xb07f] > too many stack frames EXPECTED RESULT 1. Accessing (wayland) clipboard should be a relatively cheap operation (in terms of CPU use). 2. Enumerating the binaries - is that really necessary? Even if so, something seems to be broken there (the readlink calls point to a weird location in the home dir). NOTE: wl-paste has no GUI, maybe that might be triggering the issue? SOFTWARE/OS VERSIONS Linux/KDE Plasma: Fedora 40 KDE Plasma Version: 6.0.5 KDE Frameworks Version: 6.2.0 Qt Version: 6.7.1 ADDITIONAL INFORMATION
Created attachment 170269 [details] Stack trace for the readlink calls
One more thing: Noticed there seem to be quite a lot of getpid() calls sprinkeled in as well. Caching that (storing the result) could also help performance (avoid context switch).
getpid is cached by glibc, a cache on top wouldn't gain anything.
To explain the trace; whenever we get a new client we match it up to a relevant .desktop file on connect. This is for a faux-security task of limiting various wayland specific interfaces to certain clients and not others. When we don't have solid info it's based on matching Exec lines with desktop files. That's all cached, so relatively fast, but it's doing something with symlink evaluation in there. It's not slow because of clipboard access, it's slow determining whether wl-paste is told there's an interface to move it's window / etc. This will happen only once on app startup, but wl-paste is a new client every time.
Whilst I don't like the above, it's not a huge burden for typical usage. Can you run `time wl-paste` (the total time will include the time blocked on kwin doing all this) >Finally I was able to narrow it down to the use of some bash scripts that monitor and process the clipboard. Are they monitoring by polling? If so, that's on you to fix. You might like the env var: KWIN_WAYLAND_NO_PERMISSION_CHECKS
(In reply to David Edmundson from comment #3) > getpid is cached by glibc, a cache on top wouldn't gain anything. Well it shows up in a large consecutive sequences in my strace => The syscalls are made repeatedly. So either there is a bug in the glibc caching or these don't originate from glibc.
> Can you run `time wl-paste` (the total time will include the time blocked on > kwin doing all this) # time wl-paste -n -t text real 0m0,047s user 0m0,000s sys 0m0,006s
> When we don't have solid info it's based on matching Exec lines with desktop > files. That's all cached, so relatively fast, but it's doing something with > symlink evaluation in there. Yes, something seems to be messed-up in this evaluation. The issue might no even be part of kwin itself but some foundational KDE library (i.e. /usr/lib64/libKF6Service.so.6.2.0). I used the stack trace to look-up the problematic code but didn't get very far (the functions/methods mostly lack documentations so I was only guessing their purpose and I'm not familiar with the codebase). Looking up the binary in a home dir is (almost) guaranteed to fail & also it's probably performed __ instead of __ the correct binary path. Also wl-clipboard rpm package (containing wl-paste) doesn't include any .desktop files, so maybe some optimizations could be made / or it might be a part of the issue? > > It's not slow because of clipboard access, it's slow determining whether > wl-paste is told there's an interface to move it's window / etc. I wasn't trying to suggest the core issue is the clipboard handling. Just discovered a weird behavior and this is (so far) the only reliable way to replicate it I found. I know invoking wl-paste repeatedly isn't very efficient (the scripts are more general and can also use xclip which doesn't have the --watch mode) , but that is not the point here. The CPU spike is in kwin, the related traces are weird, quite obviously buggy, and in my opinion worth investigating (regardless if it will lead to performance improvement in the end).
Hope I didn't miss anything & supplied all the requested info. Let me know if I can help in any other way. BTW. Can you replicate this locally?
I'm wrong about getpid, there's a whole section in the man page about how it did now doesn't. I'm too old! >BTW. Can you replicate this locally? Yes. There's definitely lots of scope to optimise. Either a cache, or turning this logic on it's head. We could also explore throwing it in the bin now we have better paths for the goal this set out to achieve.