Bug 271841

Summary: SCAN : KDirwatch and database : digikam very slow when editing iptc data
Product: [Applications] digikam Reporter: Nicolas Pomarede <npomarede>
Component: Database-ScanAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: ahuggel, caulier.gilles, npomarede
Priority: NOR    
Version: 1.9.0   
Target Milestone: ---   
Platform: Mandriva RPMs   
OS: Linux   
Latest Commit: Version Fixed In: 4.3.0
Sentry Crash Report:

Description Nicolas Pomarede 2011-04-27 14:08:05 UTC 
Version:           1.9.0 (using KDE 4.6.2) 
OS:                Linux

Although digikam is quite responsive most of the time, I noticed that each time I edit some iptc metadata in an image, the program gets very slow for a few seconds, sometimes up to 20-30 seconds. During that time, the UI is not responding immediatly (scroll, click, ...)

(this problem is not related to 1.9 only, I noticed it since I used digikam a few years ago).

Reproducible: Always

Steps to Reproduce:
This problem seems related to the number of pictures in the current album. It seems each time I modify the iptc metadata of one image, all other images are rescanned. The more images in the album, the more time it takes to get back to normal.

For example, with a rather small 300 images album ; at start, everything is responsive, I can scroll in the album view, thumbnails appear quickly, all is good.

Now, I choose one image, edit iptc metadata and change "caption" for example (the iptc element doesn't matter). Once I press OK to close the metadata window, Digikam becomes very slow, clicking or scrolling can take many seconds before having an effect, and I can see all images' thumbnails seem to be reloaded (they quickly disappear/reappear one after the other).

There's no indication in the bottom progress bar that something is happenning, but the program is really stuck.


Another frustrating experience is when selecting many images (let's say 40) and choosing "edit iptc metadata". In that case, clicking "next" in the iptc window to edit the next image's data can also take 5 seconds or more, as if a background task was started each time one image is modified and slowed everything.
While editing those 40 images data could take only a couple of minutes, it ends up taking 10 minutes or more, mostly waiting for the UI to respond each time I click on 'next'.


Is this a digikam problem, or the underlying QT window that tries to refresh everything while only 1 image was modified ?

Are there some traces I could make to try to debug what part of Digikam is creating this slowdown ?
Comment 1 caulier.gilles 2011-04-27 14:17:18 UTC 
All metadata are written to images using exiv2 library. Which version you use with digiKam ? Go to Help/Components Info for details

Gilles Caulier
Comment 2 Nicolas Pomarede 2011-04-27 14:44:57 UTC 
digikam is linked with libexiv2.so.10, which in my case comes from libexiv2_10-0.21.1-1-mdv2011.0.i586  (I'm running Mandriva cooker)
Comment 3 caulier.gilles 2011-04-27 14:48:27 UTC 
Go to Help/Components Info menu and copy and paste the dialog contents here...

Gilles Caulier
Comment 4 Nicolas Pomarede 2011-04-27 15:07:25 UTC 
digiKam version 1.9.0
Dématriçage parallélisé: Oui
Exiv2 peut écrire dans un fichier JP2: Oui
Exiv2 peut écrire dans un fichier JPEG: Oui
Exiv2 peut écrire dans un fichier PGF: Oui
Exiv2 peut écrire dans un fichier PNG: Oui
Exiv2 peut écrire dans un fichier TIFF: Oui
Exiv2 prend en charge les métadonnées XMP: Oui
LibCImg: 130
LibClapack: bibliothèque interne
LibExiv2: 0.21.1
LibJPEG: 80
LibJasper: 1.900.1
LibKDE: 4.6.2 (4.6.2)
LibKExiv2: 1.2.0
LibKdcraw: 1.2.0
LibLCMS: 119
LibLensFun: bibliothèque partagée externe
LibLqr: bibliothèque interne
LibPGF: 6.09.44 - bibliothèque interne
LibPNG: 1.2.44
LibQt: 4.7.2
LibRaw: 0.11.3
LibTIFF: LIBTIFF, Version 3.9.4 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc.
Élément graphique Marble: 0.11.0 (Stable Release)
LibGphoto2: 2.4.11
LibKipi: 1.2.0
Moteur de base de données: QSQLITE
Comment 5 caulier.gilles 2011-04-27 15:16:22 UTC 
Which type of image you want to patch with IPTC ? RAW ? JPEG ?

Gilles Caulier
Comment 6 Nicolas Pomarede 2011-04-27 15:19:47 UTC 
I'm only using jpeg at the moment.
Comment 7 Nicolas Pomarede 2011-08-30 21:49:33 UTC 
Hello

running digikam from the console, I got a lot of traces that explain waht is happening : each time I modify some iptc tags in one image, the whole album is completly rescanned. All other images (that were not modified) are read again, which cause major slowdown.
Worse, it seems that if I modify iptc in several image in one go (let's say 7 images), then the album is rescanned 7 times in parallel, which certainly cause a lot of disk accesses.

This is still with digikam 1.9.0 and kde 4.6.4, I haven't tried digikam 2 yet, maybe this issue was fixed in 2.0 ?

Here're the trace when I modify iptc in one image :

digikam(371)/KEXIV2 KExiv2Iface::KExiv2::setIptcTagsStringList:   :  Iptc.Application2.LocationCode  =>  MEX
digikam(371)/KEXIV2 KExiv2Iface::KExiv2::setIptcTagsStringList:   :  Iptc.Application2.LocationName  =>  Mexique
digikam(371)/KEXIV2 KExiv2Iface::KExiv2::setIptcTagsStringList:   :  Iptc.Application2.Byline  =>  Nicolas Pomarede
digikam(371)/KEXIV2 KExiv2Iface::KExiv2::setIptcKeywords:   ==> Iptc Keywords:  Personnes,Mexique
digikam(371)/KEXIV2 KExiv2Iface::KExiv2::KExiv2Priv::saveToFile: File Extension:  "jpg"  is supported for writing mode
digikam(371)/KEXIV2 KExiv2Iface::KExiv2::save: Metadata for file ' DSCF4130.JPG ' written to file.
digikam(371)/digikam (core) Digikam::DImg::load: "/home/npomarede/Images/Photos/DSCF4044.JPG"  : JPEG file identified
digikam(371)/digikam (core) Digikam::isJpegImage: mimetype =  "JPEG"
digikam(371)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation => Exif.Image.Orientation =>  1
digikam(371)/digikam (core) Digikam::DImg::load: "/home/npomarede/Images/Photos/DSCF4045.JPG"  : JPEG file identified
digikam(371)/digikam (core) Digikam::DImg::load: "/home/npomarede/Images/Photos/DSCF4046.JPG"  : JPEG file identified
digikam(371)/digikam (core) Digikam::DImg::load: "/home/npomarede/Images/Photos/DSCF4048.JPG"  : JPEG file identified
digikam(371)/digikam (core) Digikam::DImg::load: "/home/npomarede/Images/Photos/DSCF4050.JPG"  : JPEG file identified
digikam(371)/digikam (core) Digikam::DImg::load: "/home/npomarede/Images/Photos/DSCF4053.JPG"  : JPEG file identified
digikam(371)/digikam (core) Digikam::DImg::load: "/home/npomarede/Images/Photos/DSCF4054.JPG"  : JPEG file identified
digikam(371)/digikam (core) Digikam::DImg::load: "/home/npomarede/Images/Photos/DSCF4055.JPG"  : JPEG file identified
digikam(371)/digikam (core) Digikam::DImg::load: "/home/npomarede/Images/Photos/DSCF4056.JPG"  : JPEG file identified
digikam(371)/digikam (core) Digikam::ScanControllerLoadingCacheFileWatch::slotImageChanged: 9158 "/home/npomarede/Images/Photos/DSCF4044.JPG"
digikam(371)/digikam (core) Digikam::ScanControllerLoadingCacheFileWatch::slotImageChanged: 9159 "/home/npomarede/Images/Photos/DSCF4045.JPG"
digikam(371)/digikam (core) Digikam::ScanControllerLoadingCacheFileWatch::slotImageChanged: 9160 "/home/npomarede/Images/Photos/DSCF4046.JPG"

This continues until all images in that album are rescanned (320 in that case). When this is done, digikam becomes responsive again.

Nicolas
Comment 8 caulier.gilles 2011-08-31 04:36:54 UTC 
This is relevant of KDirWatch mechanism used with digiKam database. 

I think it's fixed with 2.0.0

Gilles Caulier
Comment 9 caulier.gilles 2011-12-15 08:48:25 UTC 
Nicolas,

This file still valid using digiKam 2.4 ?

Gilles Caulier
Comment 10 Nicolas Pomarede 2011-12-15 09:12:40 UTC 
Hello
can't say for the moment, there's still no digikam 2+ version for mandriva cooker :( Hopefully it should be available soon, I will update this report then.

Thanks
Comment 11 Marcel Wiesweg 2011-12-25 13:55:21 UTC 
Since 2.4, digikam uses inotify directly on Linux, KDirWatch only if it cannot be avoided
Comment 12 caulier.gilles 2014-08-07 07:16:02 UTC 
Following Marcel comment #11, we need a fresh feedback using digiKam 4.2.0 ?

Gilles Caulier
Comment 13 Nicolas Pomarede 2014-08-24 19:32:44 UTC 
Hello
I confirm that with digikam 4.2, editing metadata is faster, in the UI we can see that when doing a batch, only the currently modified image's thumbnail is reloaded.
Before, all thumbnails were always reloaded, which made the editing painfully slow.

I'm closing the bug as "resolved".