Bug 480665 - KDE Neon 5.27.10: baloo significantly slows down file copying to a usb device
Summary: KDE Neon 5.27.10: baloo significantly slows down file copying to a usb device
Status: REPORTED
Alias: None
Product: frameworks-baloo
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: unspecified
Platform: Neon Linux
: NOR normal
Target Milestone: ---
Assignee: baloo-bugs-null
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-02-01 09:00 UTC by Martin Tlustos
Modified: 2024-05-14 14:21 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Tlustos 2024-02-01 09:00:07 UTC
SUMMARY

I tried to copy a movie from an external hard drive to a usb stick, and it took way too long. Dolphin even became unresponsive in the process.
I tried to track the problem down, so started windows and copied the same file (2.1GB). Was done in less than a minute.
Went back to Linux, tried to copy from command line (cp): about as fast as windows.
Then I disabled baloo, and it copied the same file again in about 20 seconds.
Re-enabled baloo, and it took over five minutes to copy the same file.
***
NOTE: If you are reporting a crash, please try to attach a backtrace with debug symbols.
See https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports
***


STEPS TO REPRODUCE
1. 
2. 
3. 

OBSERVED RESULT


EXPECTED RESULT


SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 
KDE Frameworks Version: 
Qt Version: 

ADDITIONAL INFORMATION
Comment 1 Martin Tlustos 2024-02-01 09:28:26 UTC
Operating System: KDE neon 5.27
KDE Plasma Version: 5.27.10
KDE Frameworks Version: 5.114.0
Qt Version: 5.15.12
Kernel Version: 6.5.0-15-generic (64-bit)
Graphics Platform: Wayland
Processors: 6 × AMD Ryzen 5 4500U with Radeon Graphics
Memory: 15,0 GiB of RAM
Graphics Processor: AMD Radeon Graphics
Manufacturer: Acer
Product Name: Aspire A515-44G
System Version: V1.12
Comment 2 tagwerk19 2024-02-03 07:29:30 UTC
I'm afraid I'm not getting the same. I tried a similar test, a 2.2 GB video file copied over USB-A to a stick.

I did the copy with "cp" and ran iotop to see how quickly the data was actually being written. Caching makes following the size of destination file confusing...
    
I tried with and without baloo running and with baloo with a constrained amount of RAM (when it is run with "systemctl --user start kde-baloo", which limits RAM usage to 512 MB) and without (if you stop and restart Baloo with "balooctl disable" and "enable", you don't get that limitation). I saw no variation in speed.
    
I presume you are not indexing the USB Stick? If you had that set up (somehow), that might possibly have an impact. Baloo would get notifications that the file had been updated and would repeatedly try to index it.

In passing I noticed a couple of things that did make a difference to the write speed:

If I watched the file "arrive" in Dolphin, the thumbnailing process was busy. I presume it also watches for file changes as Baloo does and repeatedly generates a thumbnail for it. If the copy was slow, then the thumbnailing load was higher.
    
I was able to copy quickly on a newly rebooted machine. However if I accessed the USB from a KVM guest it was slow. If I unmounted the USB from the guest and went back to the host and tried again from there it was still slow, with a transfer speed of between 4 to 8 MB/sec (!). Even in this setup (testing from the VM guest), I didn't see a variation correlated to whether Baloo was running or not...

This was with Fedora on the host - and an older Plasma/Frameworks 5.27.8 and 5.109.0
Comment 3 Martin Tlustos 2024-02-05 04:59:11 UTC
As I said, copying with cp was a lot faster. I guess, dolphin and baloo have some kind of connection that slows dolphin down in copying even when the folder is an indexed one.
Comment 4 Martin Tlustos 2024-02-05 08:37:22 UTC
Maybe dolphin puts some temporary files somewhere in my home folders that baloo wants to index because the specific folder is not excluded?
Comment 5 tagwerk19 2024-02-05 09:02:49 UTC
(In reply to Martin Tlustos from comment #3)
> As I said, copying with cp was a lot faster. I guess, dolphin and baloo have
> some kind of connection that slows dolphin down in copying even when the
> folder is an indexed one.

(In reply to Martin Tlustos from comment #0)
> Went back to Linux, tried to copy from command line (cp): about as fast as
> windows.
> Then I disabled baloo, and it copied the same file again in about 20 seconds.

Looks like I missed that you were copying in Dolphin (with just the one test with cp). 

No problem, I will look to see if I see anything.

(In reply to Martin Tlustos from comment #4)
> Maybe dolphin puts some temporary files somewhere in my home folders that
> baloo wants to index because the specific folder is not excluded?

As far as I know, no.

Maybe I remember that in the case of Baloo, that does not make use of workfiles even when doing content indexing.

Can you check "systemctl status --user kde-baloo" when you've got Baloo running and the copy is slow? That's to see whether Baloo is working within its 512 MB memory cap.

What do you get with a "kmimetypefinder" for the file, it is properly recognised?
Comment 6 Martin Tlustos 2024-02-05 13:13:09 UTC
systemctl status --user kde-baloo says it uses 123mb, so it should be fine.

It took over 2 minutes to copy 1GB to the usb stick. The process goes something like this:

The notification area shows quickly that 100% are being copied (I guess that's the cache filling up)
Then there is a time when quite some writing takes place (20 seconds or so). Up and down between 100kb/s and 30mb/s.
Then there was almost a minute of no disk activity (I used a customized page on system monitor to track disk activity).
Then it resumes and after a while the process is finished.

The second time it went quicker, again first the cache was filled, then a period of high writing (this time ~30mb for ~30s, and then intermittent between 30mb and 0 for another 15s or so).

I also looked at balooctl status and found that it had 5.75GB, so I purged it and reindexed it with my home directory not being indexed and only indexing my documents folder (I have a huge media folder with all images, music and videos in subfolder, a total of 296,7 GiB (318 556 779 798), 70 262 files, 703 sub-folders). I don't need to index that as it's managed by digikam and other programs, so I excluded it for now.
Comment 7 Martin Tlustos 2024-02-05 13:27:16 UTC
I did another round, this time with the same file as I used for reporting (2.1GB)
Again, filling up the cache quickly (showing 100% in the notification area), then
intermittent disk activity (jumping from zero to 20-30mb for about 5 seconds, then 2-3 seconds nothing), then
over a minute of (almost) no disk activity (a few writes with 70kb/s for a second or so), then
intermittent disk activity again for about 20 seconds, then silence

and so on until after about 7 minutes the copying is finished.
Comment 8 tagwerk19 2024-02-06 17:31:35 UTC
(In reply to Martin Tlustos from comment #7)
> Again, filling up the cache quickly (showing 100% in the notification area),
> then intermittent disk activity (jumping from zero to 20-30mb for about 5
> seconds, then 2-3 seconds nothing), then over a minute of (almost) no
> disk activity (a few writes with 70kb/s for a second or so), then
> intermittent disk activity again for about 20 seconds, then silence
It doesn't make a lot of sense does it?
    
I've tried the same, copying with Dolphin and watching the "worker thread" writing in iotop. Sometimes blazingly fast, sometimes slow (comparable to your results although I don't see the long breaks with no disk activity). No obvious difference between having Baloo running or not. 

I'm getting short of ideas. The only pattern I've seen is that with the USB stick being used by a VM and I think that is somewhat tenuous...
Comment 9 Stefan Brüns 2024-05-14 14:21:28 UTC
(In reply to Martin Tlustos from comment #3)
> As I said, copying with cp was a lot faster. I guess, dolphin and baloo have
> some kind of connection that slows dolphin down in copying even when the
> folder is an indexed one.

If copying with cp is faster, the slowdown is obviously not baloo's fault.

But you should use `cp` correctly, it will report 'finished' when everything is 'written' to the cache, not when the data has actually been flushed to the disk.

Use "cp ... && sync".