Bug 358231

Summary: desktop locks up when moving lots of files
Product: [Frameworks and Libraries] frameworks-kio Reporter: richlv
Component: generalAssignee: KIO Bugs <kio-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: andreowitsch.opensource, bizyaev, bodom, bugseforuns, davidmarch007, demetrykravchenko, eridanired123, fanisatt, filipfila.kde, fincer89, finex, friedmann.christopher, hhielscher, ilikefoss, joao.vidal.silva, jodr666, katze_942, kde, kdelibs-bugs, kurko2000, kyle.white, kyrawertho, mahen, manliodp, meven29, milakam, nate, nick.craig.law, nnikita.beloglazov, null, plasma-bugs, postix, s404.gordon, sitter, stratiladimitrie, tagwerk19, tim, voidarray, windows2linux
Priority: VHI    
Version: git master   
Target Milestone: ---   
Platform: openSUSE   
OS: Linux   
See Also: https://bugs.kde.org/show_bug.cgi?id=390748
Latest Commit: Version Fixed In: 5.106
Sentry Crash Report:
Attachments: "zombie" notifications
"job failed" notification
Hang with graphical artifacts
Video demonstration of the problem.

Description richlv 2016-01-19 23:11:30 UTC
this might not be a dolphin problem, as it also locks up the desktop, but i'm not sure what would be a better component.
i've searched around for an existing bugreport, and bug #342056 seems similar - but the symptoms and the cause seem to be somewhat different here.

when moving files and the source directory contains lots of files, the source dolphin window becomes unresponsive for some time. whole desktop (taskbar, systray, notifications etc) completely lock up for an even longer time. experiencing this with jpg files specifically. happens both with previews enabled and disabled.

this does not happen when moving one or two files. moving 7 files locks the system up for some 4 seconds. moving 20 files locks the system up for some 13-20 seconds. larger amounts of files moved lock the system up for a much longer time.

note that actual applications work perfectly fine, and it is possible to alt-tab to them. the kde panel (including systray, taskbar, menu, notification and other popups) is the only thing that locks up. the source dolphin window also locks up, but for a shorter period of time.

this is tied to the amount of files in the source window and the amount of files moved. moving images into the directory with many files works much faster.

Reproducible: Always

Steps to Reproduce:
1. have a directory with a large amount of files - for example, 1500
2. select some of those files - for example, 100
3. move them to another directory (ctrl-x, ctrl-v in another dolphin window)

Actual Results:  
the source dolphin window locks up for some time. taskbar, systray, all popups lock up for an even longer time.

Expected Results:  
files are moved instantly, like they did in kde4
Comment 1 richlv 2016-01-19 23:16:26 UTC
the cpu usage does not seem to increase much during the lockup time - plasmashell eats ~ 1.3%

the part about no lockup when moving files into the directory with lots of files was wrong - actually, it does seem to lock up in a similar way (it's an operation i don't do often and apparently i messed up my tests a bit)

disabling "track file transfers and other jobs" for notifications does not seem to help
Comment 2 Thomas Wihl 2016-09-06 14:26:33 UTC
I have something similar, but only Dolphin seems to be affected.
Whenever I write a file to whatever filesystem (I tested my SSD, HDD and Network Share) Dolphin becomes unresponsive/freezes for several seconds.
* Open a file with Kate
* Change some characters
* Save
* Go back to dolphin
* No UI update for several seconds
Comment 3 FiNeX 2016-10-02 09:40:44 UTC
I'm experiencing the same bug: copying files from dolphin to dolphin (drag 'n drop or CTRL+X/V) the plasma shell becames unresponsive for some seconds. I'm using a quite recent PC (CPU Intel i7 4770K, 16gb RAM). Copying about 20 files will stops the shell for at least 10 seconds.

Dolphin 16.08.1, KDE 5.26.0, Qt 5.7.0
Comment 4 Tony 2016-10-02 15:47:02 UTC
Similar/duplicate 369568
Comment 5 FiNeX 2016-10-02 16:54:25 UTC
*** Bug 369568 has been marked as a duplicate of this bug. ***
Comment 6 Thomas Wihl 2016-10-03 06:17:34 UTC
Well I "fixed" it by creating a new user/home directory. I tried deleting some of the config files in my old home but couldn't figure out which one caused the trouble.
Comment 7 andreowitsch.opensource 2017-11-16 18:00:39 UTC
I can confirm this bug both on kde neon and debian buster (plasma 5.10) as new user with the default configuration.

Moving many files (in my case 177 files with a total 340 MiB) causes the plasma shell to hang for quite sime time. This is especially grave when overwriting some or all of the destination files. The cpu load is low and all applications except the panel etc. remain responsive.

This "hanging" continues for an unknown duration (at least several minutes), even after the moving/overwring of files is finished. 

However closing all dolphin instances (after the process is finished) immideatedly makes the whole system responsive again.
Comment 8 Christoph Feck 2017-11-29 00:11:17 UTC
I cannot remember that I had this issue in the past (i.e. at the time this ticket was reported), but since recently, I also experience this. Moving a single folder with many items works fine, but selecting all items in the folder, and moving them in one go causes the described lockup for minutes after moving.
Comment 9 Patrick Silva 2018-01-25 03:38:32 UTC
duplicate?
https://bugs.kde.org/show_bug.cgi?id=378935
Comment 10 Nate Graham 2018-02-06 16:10:38 UTC
*** Bug 378935 has been marked as a duplicate of this bug. ***
Comment 11 Jaime Torres 2018-02-19 20:10:02 UTC
Git commit e1fe26f1fc78082fdb215cb818177541d4607d9c by Jaime Torres.
Committed on 19/02/2018 at 20:09.
Pushed by jtamate into branch 'master'.

Reduce plasmashell frozen time to almost nothing

Summary:
Related: bug 342056
Even the icon with the number of tasks pending moves from time to time.

To reduce the frozen time, a similar patch must be applied also to
frameworks/kwindowsystem src/platforms/xcb/kxmessages.cpp
frameworks/plasma-framework src/plasma/private/effectwatcher.cpp

According to the documentation (and a look to the source code)
http://doc.qt.io/qt-5/qabstractnativeeventfilter.html

The type of event eventType is specific to the platform plugin chosen
at run-time, and can be used to cast message to the right type.

On X11, eventType is set to "xcb_generic_event_t", and the message can
be casted to a xcb_generic_event_t pointer.

The other eventType are "windows_generic_MSG" and "mac_generic_NSEvent".
No other eventType starts with an 'x'.

Test Plan:
Cut & paste 2000 small files.
Before, a freeze (plasmashell not responding) of minutes
After, a freeze of seconds

Reviewers: #frameworks, #plasma, davidedmundson

Reviewed By: #plasma, davidedmundson

Subscribers: broulik, davidedmundson, plasma-devel

Tags: #plasma

Differential Revision: https://phabricator.kde.org/D10627

M  +1    -1    shell/screenpool.cpp

https://commits.kde.org/plasma-workspace/e1fe26f1fc78082fdb215cb818177541d4607d9c
Comment 12 Patrick Silva 2018-05-05 15:30:12 UTC
*** Bug 393748 has been marked as a duplicate of this bug. ***
Comment 13 Nick 2018-05-08 15:09:13 UTC
I'm not convinced this problem has been fully fixed .. yes, git commit e1fe26f1fc78082fdb215cb818177541d4607d9c by Jaime Torresit's has reduced the plasma hang time but 9 seconds to move 1000 1Mb files on the same partition on a HDD that takes the mv command less than a 10th of a second is still not right.

Here's my comments from https://bugs.kde.org/show_bug.cgi?id=393748

 "I've tested the developer edition 20180508-1053 today and although you have improved things ie plasma panel only hangs for 4 seconds for a 120 file move (9 seconds for a 1000 file move.

You've not fixed the plasma panel/desktop freeze

Looking at Dolphin stderr it outputs this message "QXcbClipboard: SelectRequest too old" 11,900 times during the period plasma panel has hung.

So sorry guys it's not been fixed. As before this problem is 100% reproducible using the procedure I used above.

During the panel freeze the CPU is idle & all disc I/O completed within a second.

I know I'm not familiar with the code but I would have thought the error "QXcbClipboard: SelectRequest too old" was a good place to start. Find the cause of that and you might just find why plasma freezes ?.

I've reopened this bug and I'm assuming I'm looking at the latest code although I did download dev stable rather than dev unstable."
Comment 14 Nick 2018-05-08 15:13:10 UTC
I should just clarify plasma freezes for 9 seconds, the file move operation completes in less than a second. I may have given the impression the file move took 9 seconds for 1000 files.
Comment 15 David Edmundson 2018-07-12 17:41:21 UTC
*** Bug 375781 has been marked as a duplicate of this bug. ***
Comment 16 David Edmundson 2018-07-12 17:41:24 UTC
*** Bug 386781 has been marked as a duplicate of this bug. ***
Comment 17 David Edmundson 2018-07-12 17:41:31 UTC
*** Bug 383660 has been marked as a duplicate of this bug. ***
Comment 18 Méven Car 2018-09-30 09:58:36 UTC
Probably relates to https://bugs.kde.org/show_bug.cgi?id=390748 except it concerns deletion operation.
Comment 19 johan 2018-10-15 17:58:17 UTC
I can confirm this on Arch Linux. Moving as few as 10 small files using Dolphin can cause the desktop to freeze for 10 seconds or so.

plasma-desktop package is 5.13.5-1 and dolphin is 18.08.1-2
Comment 20 Mahendra Tallur 2018-10-19 12:50:44 UTC
I've just opened this bug which might be related : https://bugs.kde.org/show_bug.cgi?id=400025
Comment 21 Nate Graham 2018-10-19 13:23:43 UTC
*** Bug 400025 has been marked as a duplicate of this bug. ***
Comment 22 Patrick Silva 2019-02-23 17:07:24 UTC
This problem persists.

Operating System: Arch Linux 
KDE Plasma Version: 5.15.1
KDE Frameworks Version: 5.55.0
Qt Version: 5.12.1
Kernel Version: 4.20.11-arch2-1-ARCH
OS Type: 64-bit
Processors: 2 × Intel® Celeron® CPU G1820 @ 2.70GHz
Memory: 7,7 GiB of RAM
Comment 23 Patrick Silva 2019-02-23 17:08:04 UTC
*** Bug 404026 has been marked as a duplicate of this bug. ***
Comment 24 Patrick Silva 2019-02-24 03:11:04 UTC
*** Bug 404751 has been marked as a duplicate of this bug. ***
Comment 25 Patrick Silva 2019-10-12 11:30:11 UTC
This problem just happened on my system while Dolphin moved 358 small jpg files from a ntfs partition to my Home .

Operating System: Arch Linux 
KDE Plasma Version: 5.16.90
KDE Frameworks Version: 5.62.0
Qt Version: 5.14.0 beta1
Comment 26 Patrick Silva 2020-04-01 15:50:58 UTC
Still happening.

Operating System: Arch Linux 
KDE Plasma Version: 5.18.4
KDE Frameworks Version: 5.68.0
Qt Version: 5.14.2
Comment 27 Méven Car 2020-04-05 11:01:36 UTC
I suspect that the issue is mostly X server specific.
Could someone confirm ?
Comment 28 Patrick Silva 2020-04-05 11:55:12 UTC
Created attachment 127290 [details]
"zombie" notifications

At least on Neon unstable I can only reproduce this problem on X11.
Sometimes I also get 'zombie" notifications after files moving to be completed.
And when I close Dolphin while these "zombie" notifications are active, Plasma shows "job failed" notifications as we can see in the attached screenshot.

Operating System: KDE neon Unstable Edition
KDE Plasma Version: 5.18.80
KDE Frameworks Version: 5.69.0
Qt Version: 5.14.1
Comment 29 Patrick Silva 2020-04-05 11:55:54 UTC
Created attachment 127291 [details]
"job failed" notification
Comment 30 Patrick Silva 2020-11-08 13:47:39 UTC
This problem persists.

Operating System: Arch Linux 
KDE Plasma Version: 5.20.2
KDE Frameworks Version: 5.75.0
Qt Version: 5.15.1
Comment 31 João Vidal da Silva 2020-11-09 04:55:29 UTC
*** Bug 368480 has been marked as a duplicate of this bug. ***
Comment 32 manliodp 2020-12-09 00:27:47 UTC
Same here, still happening on manjaro with plasma 5.20.3
I confirm it happens only on X, unfortunately the wayland "ecosystem" is not there yet so I think a fix should be investigated for this issue on a fundamental/core functionality like this (moving a bunch of files!).

Thanks for your hard and appreciated work!
Comment 33 manliodp 2021-02-03 23:06:01 UTC
Hi,
Below a good way to reproduce the issue:

1) Create folder A
2) Put dozen/hundreds of files in folder A
3) Create folder B at the same level of folder A
4) Move the files from folder A to folder B
5) Immediately delete folder A

Plasma shell will freeze for a minute or more and then recover with a ghost notification

​

Thank you
Comment 34 Filip Fila 2021-03-03 10:46:13 UTC
I still have this problem on my laptop with a SATA SSD and formatted as ext4. System updates or just about any significant file operation will lock the system up.

But I managed to completely solve the problem on my PC (where I had the same issues) when I migrated to a NVMe SSD formatted as f2fs. I don't think the NVMe part should have been that significant because SATA is still plenty fast and doesn't lock up in other DEs or OS'. Ext4 is also fine in other DEs from recollection, but perhaps something in KDE just doesn't work well with it but works OK with f2fs. IIRC changing the schedulers doesn't help.

If someone wants me to do more rigorous testing just give me steps and I'll give it a try.
Comment 35 Nikita Beloglazov 2022-08-25 21:41:20 UTC
Confirmed with one large file, a hang occurs between drag and drop and the appearance of the move or copy selection menu, with graphical artifacts (see screenshot). At the same time, you can hear how the hard disk starts working at this moment, after some time the "move or copy" selection menu appears, and after that the file is instantly copied.

Operating System: openSUSE Tumbleweed 20220824 (latest)
KDE Plasma Version: 5.25.4
KDE Frameworks Version: 5.97.0
Qt Version: 5.15.5
Kernel Version: 5.19.2-1-default (64-bit)
Graphics Platform: X11
Comment 36 Nikita Beloglazov 2022-08-25 21:43:19 UTC
Created attachment 151594 [details]
Hang with graphical artifacts
Comment 37 Жора Змейкин 2022-08-27 13:28:57 UTC
Created attachment 151631 [details]
Video demonstration of the problem.

I have exactly the same problem. When copying a file, the shell freezes (but not Dolphin), but if you copy an entire folder, plasma does not hang. I tried copying a file or folder through the console with the cp and rsync command. When copying through the console, the shell did not hang.

The problem manifests itself on both X11 and Wayland. 
System: Arch Linux
KDE Plasma Version: 5.25.4
KDE Frameworks Version: 5.97.0
Qt Version: 5.15.5
Linux Kernel version: 5.15.62-1-lts

The entire system has been updated to the latest version.
Comment 38 Ilya Bizyaev 2022-08-27 14:54:36 UTC
Marking this bug confirmed, at it affects multiple users.
Comment 39 flan_suse 2022-09-01 18:03:38 UTC
(In reply to DarkVessel from comment #37)
> I have exactly the same problem. When copying a file, the shell freezes (but
> not Dolphin), but if you copy an entire folder, plasma does not hang. 

LOL!

Same issue here too. At first I thought it had to do with the file being *large*, yet if it involves a *FOLDER* (whether containing a single or multiple large files, doesn't matter) then it smoothly glides through the entire copy operation!

I thought I was going crazy until I came across this bug report. The trick of copying a FOLDER (instead of a large FILE) magically works. Not sure why...

. . .

Operating System: Manjaro Linux
KDE Plasma Version: 5.24.6
KDE Frameworks Version: 5.96.0
Qt Version: 5.15.5
Kernel Version: 5.19.1-3-MANJARO (64-bit)
Graphics Platform: X11
Processors: 8 × 11th Gen Intel® Core™ i7-1165G7 @ 2.80GHz
Memory: 15.4 GiB of RAM
Graphics Processor: Mesa Intel® Xe Graphics
Comment 40 flan_suse 2022-09-01 18:08:57 UTC
Ohhhhhhh, just discovered something interesting while testing your method, @DarkVessel.

What happens if you NAVIGATE to the folder *while* the copy operation is in progress?

While the folder copy operation is running, and the system is fine, I can re-create the "freeze" by entering the folder before it finishes. It is not until the copy operation is finished will I regain control of the desktop.

So this might have something to do with Dolphin/KDE/KIO/Plasma/whatever updating/refreshing the view of the current folder that is the destination of a large copy operation?
Comment 41 Nick 2023-02-11 22:06:43 UTC
(In reply to flan_suse from comment #40)
> Ohhhhhhh, just discovered something interesting while testing your method,
> @DarkVessel.
> 
> What happens if you NAVIGATE to the folder *while* the copy operation is in
> progress?
> 
> While the folder copy operation is running, and the system is fine, I can
> re-create the "freeze" by entering the folder before it finishes. It is not
> until the copy operation is finished will I regain control of the desktop.
> 
> So this might have something to do with Dolphin/KDE/KIO/Plasma/whatever
> updating/refreshing the view of the current folder that is the destination
> of a large copy operation?

I'm not seeing this behaviour in a later version of KDE I'm running (KDE Neon). Dragging a file 300+MB to the desktop. How big are the files you are dragging to the desktop?

Operating System: KDE neon 5.26
KDE Plasma Version: 5.26.5
KDE Frameworks Version: 5.102.0
Qt Version: 5.15.8
Kernel Version: 5.15.0-60-generic (64-bit)
Graphics Platform: X11
Processors: 20 × 12th Gen Intel® Core™ i7-12700H
Memory: 15.3 GiB of RAM
Graphics Processor: Mesa Intel® Graphics
Comment 42 Bug Janitor Service 2023-02-26 03:45:30 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 43 Evgeny 2023-02-26 04:15:15 UTC
I can't confirm when dragging a directory with lots of files (740 pictures) from dolphin to desktop, BUT can confirm when dragging a single big file (1.7 GB) from dolphin to desktop. Plasmashell freezes for about 10 seconds, then context menu appears. File is on HDD, system & desktop on NVME SSD. Just like in the attached video by Zhora Zmeikin https://bugs.kde.org/attachment.cgi?id=151631

dolphin/lunar,now 4:22.12.2-0ubuntu1 amd64
Operating System: Kubuntu 23.04
KDE Plasma Version: 5.27.0 (there are some 5.26.5 and 5.26.90 packages)
KDE Frameworks Version: 5.103.0 (there are some 5.102 and even 5.101 packages, Kubuntu Lunar is a mess)
Qt Version: 5.15.8
Kernel Version: 5.19.0-21-generic (64-bit)
Graphics Platform: X11
Processors: 16 × AMD Ryzen 7 5800X 8-Core Processor
Memory: 31.3 ГиБ of RAM
Graphics Processor: AMD Radeon RX 590 Series
Manufacturer: ASUS
Comment 44 Eridani Rodríguez 2023-02-26 19:15:22 UTC
Copying an 8 Gigabyte file across any devices that do NOT hold the system does not trigger the freeze for me.

However, copying it to the physical device THAT HOLDS the system installation, does cause panels and dolphin to become sluggish; if I continue trying to do stuff, like opening other dolphin views or emptying the trash, it would eventually become non responsive unless the transfer is cancelled or completed.

This does not happen at all if cp in a terminal is used for the operation.

The problem may be less noticeable in faster devices, so if you are not being able to reproduce it, try with files that are way larger than the speed of the device where your system is installed.

Operating System: KDE neon 5.27
KDE Plasma Version: 5.27.1
KDE Frameworks Version: 5.103.0
Qt Version: 5.15.8
Kernel Version: 5.19.0-32-generic (64-bit)
Graphics Platform: X11
Processors: 4 × Intel® Core™ i5-4670 CPU @ 3.40GHz
Memory: 31.3 GiB of RAM
File system: BTRFS
System's storage device: SATA 3 SSD
Comment 45 Bug Janitor Service 2023-04-19 12:42:46 UTC
A possibly relevant merge request was started @ https://invent.kde.org/frameworks/kio/-/merge_requests/1255
Comment 46 Harald Sitter 2023-04-21 11:58:24 UTC
Git commit b9e5af7babc9d68af70e303528ed027e3d61d501 by Harald Sitter.
Committed on 21/04/2023 at 11:54.
Pushed by sitter into branch 'master'.

file: make sure to cancel reading if the worker was aborted

otherwise we end up stuck waiting for the thread to terminate while the
thread is busy needlessly reading the file, when all we wanted was
mimetype data.

M  +3    -0    src/kioworkers/file/file.cpp

https://invent.kde.org/frameworks/kio/commit/b9e5af7babc9d68af70e303528ed027e3d61d501
Comment 47 Harald Sitter 2023-04-21 14:46:33 UTC
Git commit 804d58c260b63b182ebcdd1c445d9e6fecea27c9 by Harald Sitter.
Committed on 21/04/2023 at 12:12.
Pushed by sitter into branch 'kf5'.

file: make sure to cancel reading if the worker was aborted

otherwise we end up stuck waiting for the thread to terminate while the
thread is busy needlessly reading the file, when all we wanted was
mimetype data.

(cherry picked from commit b9e5af7babc9d68af70e303528ed027e3d61d501)

M  +4    -0    src/ioslaves/file/file.cpp

https://invent.kde.org/frameworks/kio/commit/804d58c260b63b182ebcdd1c445d9e6fecea27c9
Comment 48 Nate Graham 2023-04-27 18:37:23 UTC
*** Bug 444631 has been marked as a duplicate of this bug. ***