Bug 406431 - Slow overview mode in directories with many files
Summary: Slow overview mode in directories with many files
Status: CONFIRMED
Alias: None
Product: gwenview
Classification: Applications
Component: general (show other bugs)
Version: 18.12.3
Platform: Arch Linux Linux
: NOR wishlist
Target Milestone: ---
Assignee: Gwenview Bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-04-11 10:43 UTC by c001-kd3-bug-squ33z3r
Modified: 2024-08-05 20:02 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description c001-kd3-bug-squ33z3r 2019-04-11 10:43:31 UTC
SUMMARY
There is one behavior in gwenview that bothers me for a few years now: Whenever I open a directory with many images (>10.000) in the overview mode, it takes ages (~20 seconds) to load all files. For contrast, when I open the same directory with dolphin it is loaded within 2 seconds or so.

My guess is that gwenview starts to process the images even before it got the full list of all files and slows down retrieving the list with that.

STEPS TO REPRODUCE
1. Navigate with Dolphin to a photo directory with ~10.000 images
2. click on a file to open the image with gwenview
3. switch to overview mode as soon as gwenview started
4. wait...

OBSERVED RESULT
Loading the image list takes a long time (~20 seconds)

EXPECTED RESULT
The list of images appears at a similar speed as within dolphin (~2 seconds)

SOFTWARE/OS VERSIONS
Linux: 5.0.5
KDE Plasma Version: 18.12.3
KDE Frameworks Version: 5.56.0
Qt Version: 5.12.2

ADDITIONAL INFORMATION

I have this issue for a long time now and finally took the time to create and bug-tracker account and file the issue today.

I access my photo directories via NFS, but I doubt that this causes the issue as dolphin has no problem with that.
Comment 1 Nate Graham 2019-04-12 18:17:41 UTC
> I access my photo directories via NFS, but I doubt that this causes the issue as dolphin has no problem with that.
Any chance you can test that out by copying them to a local directory and seeing if the issue goes away or remains the same?

Also, are you accessing the pictures via a mounted NFS share, or using an nfs://[whatever] path?
Comment 2 c001-kd3-bug-squ33z3r 2019-04-12 23:29:43 UTC
It is normally mounted via NFS4 (fstab).

Tomorrow I should be able to copy one of those directories to a local disk.
Comment 3 c001-kd3-bug-squ33z3r 2019-04-13 10:06:51 UTC
So I copied a directory with ~11000 photos on a local SSD and indeed there is a speedup compared to the NFS:

Local: SSD + ext4: ~10-15 seconds
NFS:   HDD + ext4: ~30-35 seconds

But given the hardware differences (HDD vs. SSD + 32G RAM), I wonder why gwenview still takes that much time to get to the conclusion that it has to display ~11k files. Dolphin is still miles away when gwenview just finished loading the file list. And when I ask ls how long it takes to give me the file list of an uncached directory it is much faster too:

# umount /mnt/nfs; sleep 3; mount /mnt/nfs; time ( ls -l /mnt/nfs/pictures/ | wc -l )
11343

real    0m0,575s
user    0m0,059s
sys     0m0,094s

So, if ls takes half a second to get the complete (uncached) list via NFS, why does gwenview require 15 seconds on a local SSD powered RAM cached version?

Obviously, something fishy is going on here.
Comment 4 c001-kd3-bug-squ33z3r 2019-04-13 10:18:34 UTC
Btw. if you want to reproduce the issue, but don't waste that much storage, I found out that it works with hard-links too:

for i in {00001..15000}; do ln original.jpg pic_$i.jpg; done
Comment 5 Christoph Feck 2019-04-13 10:24:37 UTC
It also needs to determine the file type, i.e. it opens every file and reads the first few bytes.
Comment 6 c001-kd3-bug-squ33z3r 2019-04-13 11:05:59 UTC
(In reply to Christoph Feck from comment #5)
> It also needs to determine the file type, i.e. it opens every file and reads
> the first few bytes.

Why?!?

I mean, sure, you probably want to filter the list and exclude any files that gwenview can't display anyway (completely reasonable for a good user experience) and determining the type by inspecting the content is a lot more reliable than by filename extension, but if that means I have to wait for the program until it read the beginning of 10k files, I would prefer a file name based filter (which works in >99% of the cases too and would be much faster).

Another alternative would be to at least cache the result of the file inspection (so that one has to wait only the first time an image has to be inspected). But honestly, I am not sure it is a good idea to throw 'Caching' at such a simple problem, even if the chance of a broken cache is so unlikely.
Comment 7 Nate Graham 2019-04-13 13:32:08 UTC
Yeah, it's possible we can speed this up a bit.
Comment 8 Oleg 2024-08-05 20:02:53 UTC
I can confirm this. Opening any file in 2000 files folder on HDD will take Gwenview half of a minute to load.