Bug 481643

Summary:	Thumbnailing should be multi threaded on SSDs and other fast devices
Product:	[Applications] dolphin	Reporter:	simonpatp
Component:	view-engine: icons mode	Assignee:	Dolphin Bug Assignee <dolphin-bugs-null>
Status:	CONFIRMED ---
Severity:	wishlist	CC:	dashonwwIII, fanzhuyifan, kfm-devel, meven29, nate
Priority:	NOR
Version:	21.12.3
Target Milestone:	---
Platform:	Debian stable
OS:	Linux
See Also:	https://bugs.kde.org/show_bug.cgi?id=445176
Latest Commit:		Version Fixed In:

Description simonpatp 2024-02-21 23:17:34 UTC

SUMMARY
I have a modern Zen4 system with 48 threads and fast SSDs, and upon opening new folders, I watch the thumbnailer thumbnail at about 4-5 100KiB pdfs at a time, or 30-40  5MiB JPEGs at a time. Needless to say, this annoys me, since I can watch the thumbails be generated in a single-threaded fashion. It would be great to have an option to enable multi-threaded thumbnailing.


STEPS TO REPRODUCE
1. Have a folder full of many pdfs
2. Clear thumbnail cache
3. watch the thumbnail only use a single thread

OBSERVED RESULT
I watch the thumbnails pop in over 10-15 seconds depending on screen size in linear order while cpu monitors show only a single core oactive

EXPECTED RESULT
All cores 100% active with low priority thumb-nailing when the filesystem is on an SSD 

ADDITIONAL INFORMATION
Obviously, care must be taken to not destroy HDD performance, and an option should probably be added to disable it.

Originally brought up in https://discuss.kde.org/t/multithreaded-preview-render/10686

Comment 1 fanzhuyifan 2024-02-21 23:48:43 UTC

I wonder if the bottleneck here is CPU cycles or reading from disk.

Comment 2 simonpatp 2024-02-22 00:16:27 UTC

It isn't quite comparable, but for reference:

$ time (sha1sum *.pdf)
real    0m0.430s
$ time (find . -name '*.pdf'  -type f -print0 | parallel -0 --line-buffer -j48 convert -thumbnail x64 -background white -alpha remove '{}[0]' '{}.png')
real    0m2.653s
$ time (for file in *.pdf; do convert -thumbnail x64 -background white -alpha remove "${file}[0]" "${file%.pdf}.png"; done) 
real     1m5.721s
$ ls *.pdf | wc -l
470

Comment 3 simonpatp 2024-02-22 00:29:34 UTC

In comparision, here are some numbers from 484  5-8MiB JPEGs from an HDD:

cat: 5.8s
cat (cached): 1.6s
convert (parallel): 6.6s
convert (single): 38.2s

Comment 4 Méven Car 2024-02-22 11:08:36 UTC

(In reply to fanzhuyifan from comment #1)
> I wonder if the bottleneck here is CPU cycles or reading from disk.

Neither, this isn't parallelized at all currently in dolphin code.

FYI, we had a contributor that tried to tackle this, I couldn't find the branch it might be old.