Version: 0.7.2 (using KDE KDE 3.4.1) Installed from: Gentoo Packages OS: Linux Hi, digikam is very inefficient when one album contains a huge number of pics, e.g. 2000-3000. It spawns *lots* (>200) of kio_thumbnail processes each with 8 to 10 MiB residual memory usage. Load goes through the roof (>200) and the machine is thrashing like hell. Well it's not fun. Thanks to the excellent linux 2.6 scheduler the machine is still reacting pretty well, but it's not really usable... There should definitely be a limit on how many kio_thumbnails are called simultaneously.
digikam doesn't schedule the running of the kio_thumbnail processes, in fact it tries hard to have only one instance of it running. i have had 2000+ pictures being shown from a tag with at most 4 instances of kio_thumbnail processes running/idle at one time. what version of KDE and also see if konqueror or other apps show similar behavior?
> digikam doesn't schedule the running of the kio_thumbnail > processes, in fact it tries hard to have only one instance of it running. i > have had 2000+ pictures being shown from a tag with at most 4 instances of > kio_thumbnail processes running/idle at one time. > > what version of KDE and also see if konqueror or other apps show similar > behavior? KDE 3.4.1. The problem was the same with 3.4.0. Konqueror has no problems whatsoever (independent of thumbnail size). What other apps could I try? BTW with digikam this happens 100% reproducibly. I never had such extreme loads before, except while experimenting with fork bombs :-) Note that I don't have to kill digikam. When it's done creating thumbnails everything is fine again. It's just that my machine is somewhat sluggish... If there are other tings I can/should test, please tell me.
Created attachment 11386 [details] test tarball for testing digikamthumbnail here's a tarball to test whether the digikamthumbnail kioslave behaves properly or not. Untar it and do the following: 1) cd thumbnail_test 2) qmake 3) make 4) run it with: ./thumbnail_test /full/path/to/any/photo/file it will try to retrieve the thumbnail for it 2000 times. while its going on check your system load and number of kio_digikamthumbnail processes created.
On Thursday 09 June 2005 19:00, Renchi Raju wrote: > it will try to retrieve the thumbnail for it 2000 times. while its going on > check your system load and number of kio_digikamthumbnail processes > created. It runs just fine. Only two processes, thumbnail_test and kio_digikamthum. Something's bothering me. I just created 2000 symlinks to some picture and pointed digikam to that test album. Now, everything's normal, i.e. only two processes. But they are named "kio_digikamthum", whereas the hundreds of processes I've complained about are named "kio_thumbnail". I'll see if I can find a way to properly reproduce this situation. Thanks anyway for trying to fix this. Gilles
aha. now we are getting somewhere. for some reason the digikamthumbnail process could not create the thumbnails for these files (btw, what kind of images). so the work was offloaded to kde's thumbnail creation process and it looks like they run amok. it might be possible that these processes are not created one after the other, but all started at the same time (i will have to check)
On Thursday 09 June 2005 22:47, Renchi Raju wrote: > digikamthumbnail process could not create the thumbnails for these files > (btw, what kind of images). so the work was offloaded to kde's thumbnail > creation process and it looks like they run amok. it might be possible that > these processes are not created one after the other, but all started at the > same time (i will have to check) Currently system load is > 200 :-) Opening my test album starts one or two kio_digikamthumbnail threads. The kio_thumbnail processes are created when I scroll down the album view. I.e. I'll immediately ask for thumbnails for most of the pictures. Then very pixelised thumbnails start to appear everywhere (probably generated by kio_thumbnail), and after a few moments the normal high quality thumbnails appear.
On Thursday 09 June 2005 22:47, Renchi Raju wrote: > digikamthumbnail process could not create the thumbnails for these files > (btw, what kind of images). Exactly! I've just noticed that in fact there are _lots_ of invalid symlinks in that album. So, it's my fault that you couldn't reproduce it. In fact it's quite easy now to reproduce, you'll just have to create an album containing only invalid files and watch. for i in $(seq 1 1000); do head -c 10000 /dev/zero > $i.jpg; echo $i; done Open such an album and you'll have lots of kio_thumbnail processes. (How did I end up with such an album? For testing purposes I created an album containing symlinks to all of the pictures. But later on I moved or deleted some, so the links became invalid.) Considering that this only appears with invalid pictures, it's clearly a very exceptional situation. Gilles
woah. talk about an edge case. in the experimental branch, all the work is done by the digikamthumbnail and kde kio_thumbnail is never used, so this problem will not arise. i will leave this bugreport open till we have a beta release for it.
(only adding my email to the cc: list)
no longer valid after the experimental branch merge
All digiKam KIO slave was dropped since version 5.0.0 in favor of a multi-threaded interface dedicated to query the database. This problem will not reproducible.