Summary: | too many kio_thumbnail processes bring machine to its knees | ||
---|---|---|---|
Product: | [Applications] digikam | Reporter: | Gilles Schintgen <gschintgen> |
Component: | Database-Thumbs | Assignee: | Digikam Developers <digikam-bugs-null> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | benjamin.dev, caulier.gilles |
Priority: | NOR | ||
Version: | 0.7.2 | ||
Target Milestone: | --- | ||
Platform: | Gentoo Packages | ||
OS: | Linux | ||
Latest Commit: | Version Fixed In: | 5.7.0 | |
Attachments: | test tarball for testing digikamthumbnail |
Description
Gilles Schintgen
2005-06-04 21:16:50 UTC
digikam doesn't schedule the running of the kio_thumbnail processes, in fact it tries hard to have only one instance of it running. i have had 2000+ pictures being shown from a tag with at most 4 instances of kio_thumbnail processes running/idle at one time. what version of KDE and also see if konqueror or other apps show similar behavior? > digikam doesn't schedule the running of the kio_thumbnail > processes, in fact it tries hard to have only one instance of it running. i > have had 2000+ pictures being shown from a tag with at most 4 instances of > kio_thumbnail processes running/idle at one time. > > what version of KDE and also see if konqueror or other apps show similar > behavior? KDE 3.4.1. The problem was the same with 3.4.0. Konqueror has no problems whatsoever (independent of thumbnail size). What other apps could I try? BTW with digikam this happens 100% reproducibly. I never had such extreme loads before, except while experimenting with fork bombs :-) Note that I don't have to kill digikam. When it's done creating thumbnails everything is fine again. It's just that my machine is somewhat sluggish... If there are other tings I can/should test, please tell me. Created attachment 11386 [details]
test tarball for testing digikamthumbnail
here's a tarball to test whether the digikamthumbnail kioslave behaves properly
or not. Untar it and do the following:
1) cd thumbnail_test
2) qmake
3) make
4) run it with: ./thumbnail_test /full/path/to/any/photo/file
it will try to retrieve the thumbnail for it 2000 times. while its going on
check your system load and number of kio_digikamthumbnail processes created.
On Thursday 09 June 2005 19:00, Renchi Raju wrote:
> it will try to retrieve the thumbnail for it 2000 times. while its going on
> check your system load and number of kio_digikamthumbnail processes
> created.
It runs just fine. Only two processes, thumbnail_test and kio_digikamthum.
Something's bothering me. I just created 2000 symlinks to some picture and
pointed digikam to that test album. Now, everything's normal, i.e. only two
processes. But they are named "kio_digikamthum", whereas the hundreds of
processes I've complained about are named "kio_thumbnail".
I'll see if I can find a way to properly reproduce this situation.
Thanks anyway for trying to fix this.
Gilles
aha. now we are getting somewhere. for some reason the digikamthumbnail process could not create the thumbnails for these files (btw, what kind of images). so the work was offloaded to kde's thumbnail creation process and it looks like they run amok. it might be possible that these processes are not created one after the other, but all started at the same time (i will have to check) On Thursday 09 June 2005 22:47, Renchi Raju wrote:
> digikamthumbnail process could not create the thumbnails for these files
> (btw, what kind of images). so the work was offloaded to kde's thumbnail
> creation process and it looks like they run amok. it might be possible that
> these processes are not created one after the other, but all started at the
> same time (i will have to check)
Currently system load is > 200 :-)
Opening my test album starts one or two kio_digikamthumbnail threads. The
kio_thumbnail processes are created when I scroll down the album view. I.e.
I'll immediately ask for thumbnails for most of the pictures. Then very
pixelised thumbnails start to appear everywhere (probably generated by
kio_thumbnail), and after a few moments the normal high quality thumbnails
appear.
On Thursday 09 June 2005 22:47, Renchi Raju wrote:
> digikamthumbnail process could not create the thumbnails for these files
> (btw, what kind of images).
Exactly! I've just noticed that in fact there are _lots_ of invalid symlinks
in that album. So, it's my fault that you couldn't reproduce it. In fact it's
quite easy now to reproduce, you'll just have to create an album containing
only invalid files and watch.
for i in $(seq 1 1000); do head -c 10000 /dev/zero > $i.jpg; echo $i; done
Open such an album and you'll have lots of kio_thumbnail processes.
(How did I end up with such an album? For testing purposes I created an album
containing symlinks to all of the pictures. But later on I moved or deleted
some, so the links became invalid.)
Considering that this only appears with invalid pictures, it's clearly a very
exceptional situation.
Gilles
woah. talk about an edge case. in the experimental branch, all the work is done by the digikamthumbnail and kde kio_thumbnail is never used, so this problem will not arise. i will leave this bugreport open till we have a beta release for it. (only adding my email to the cc: list) no longer valid after the experimental branch merge All digiKam KIO slave was dropped since version 5.0.0 in favor of a multi-threaded interface dedicated to query the database. This problem will not reproducible. |