Tools->Maintainance->Rebuild Thumbnails (scan changed) generates large memory leak. Fedora 19 Gome, X86_64 w/ 12GB Memory, AMD quad core, MySQL database on local machine. Yum installed from Fedora 19 repository - digikam 3.5.0 Memory goes from a few GB linearly up to 11GB during the scan for changed/new thumbnails. Reproducible: Always Steps to Reproduce: 1.Select Maintainance->Rebuild thumbnails->scan for new 2. Watch the memory usage climb until you run out. 3. Actual Results: Memory climbs linearly up to max available on the machine (12 GB) Expected Results: Memory should be stable I can't build thumbnails until this is fixed.
Not reproducible here. 16 Gb ram with more than 300 Gb of picture. I rebuild all thumbnails without any problem. Sound like a dysfunction introduced by packaging rules. But which one ? I suspect that digiKam have been compiled with an external libpgf shared lib. PGF is image format used to stored thumb in DB. Mine use internal implementation of PGF included in digiKam core. To confirm, you need to run digiKam into valgrind and post here the memory leak trace reported by this tool. Follow instruction here : http://www.digikam.org/contrib Note : you need to have digiKam debug symbols package installed. It will help too to copy and paste the liste of external components used by digiKam. Go to help/components info dialog for details. Gilles Caulier
Output of components on my system. Getting debug files now. Will report more later. Jim digiKam version 3.5.0 Exiv2 can write to Jp2: Yes Exiv2 can write to Jpeg: Yes Exiv2 can write to Pgf: Yes Exiv2 can write to Png: Yes Exiv2 can write to Tiff: Yes Exiv2 supports XMP metadata: Yes LibCImg: 130 LibEigen: 3.1.4 LibExiv2: 0.23 LibJPEG: 62 LibJasper: 1.900.1 LibKDE: 4.11.2 LibKExiv2: 2.3.1 LibKGeoMap: 2.0.0 LibKdcraw: 2.3.1 LibLCMS: 2050 LibLensFun: 0.2.7-0 LibPGF: 6.12.24 - external shared library LibPNG: 1.5.13 LibQt: 4.8.5 LibRaw: 0.15.3 LibTIFF: LIBTIFF, Version 4.0.3 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc. Marble Widget: 0.16.1 (stable version) Parallelized PGF codec: No Parallelized demosaicing: No RawSpeed codec support: No Database backend: QMYSQL Database internal server: No Kipi-Plugins: 3.5.0 LibGphoto2: 2.5.2 LibKface: 3.0.0 LibKipi: 2.1.0 LibOpenCV: 2.4.6.1
>LibPGF: 6.12.24 - external shared library ==> This is what i suspected... Gilles Caulier
Interestingly, that's the same version as the bundled/embedded one.
Rex, If i remember, there is some little difference in PGF embedded and PGF bundled. I remember some Coverity entries in PGF code about memory leak that i reported to PGF team without any feedback... PGF become a problem if code is no more maintained. ì think that an alternative must be found as WEBp which provide a fast wavelets lossless compression as PGF : http://en.wikipedia.org/wiki/WebP It's maintained by Google. Gilles Caulier
Last released libpgf 6.12.24. Trying 6.12.27 from svn.
Gilles, was your report to libpgf documented anywhere? a bug or mailing list?
Hi Nueclo, 6 or 9 month ago, i work on PGF project to port library to Cmake and to improve binary compatibility. I suppose that svn trunk (6.12.27) must include my changes at least. I'm not sure because i don't check PGF code and i don't receive feedback from PGF team since a long time... But what's i sure, it's 2 or 3 memory leak entries detected in PGF codec by Coverity tool. Rex, do you have an account as KDE developer to Coverity ? https://scan.coverity.com/ If yes, i can add you to digiKam project, you can take a look of details... Another question is about future WebP QImageIO plugin for konqueror ? I remember to have see a thread about in reviewboard this summer. I don't follow anymore this entry since my serious car crash in september... If a WebP QImage plugin exist (readwrite plus data stream), it's possible to switch thumb storage from PGF to WebP easily... Gilles Caulier
Rex, I remember to have contacted PGF maintainer directly by private mail. I pointed entries in Coverity about memory leak. No more. Problem : i cannot retrieve this email from Gmail archive (i a little bit old, at least 9 month, when i start to use Coverity to scan digiKam code) Gilles
Hello, I have made a few test in rebuilding thumbnails with digikam 4.0.0 beta1 : digikam crash here too. Please view log file : digikam_thumbnail_rebuild.log Components : digiKam version 4.0.0-beta1 Codec PGF parallélisé: Non Composant graphique Marble: 0.16.2 (stable version) Demosaic GPL2 pack support: Non Demosaic GPL3 pack support: Non Dématriçage parallélisé: Non Exiv2 peut écrire dans un fichier JP2: Oui Exiv2 peut écrire dans un fichier JPEG: Oui Exiv2 peut écrire dans un fichier PGF: Oui Exiv2 peut écrire dans un fichier PNG: Oui Exiv2 peut écrire dans un fichier TIFF: Oui Exiv2 prend en charge les métadonnées XMP: Oui LibCImg: 130 LibEigen: 3.0.5 LibExiv2: 0.22 LibJPEG: 80 LibJasper: 1.900.1 LibKDE: 4.11.2 LibKExiv2: 2.3.1 LibKGeoMap: 2.0.0 LibKdcraw: 2.4.0 LibLCMS: 2040 LibLensFun: 0.2.7-0 LibPGF: 6.12.24 - bibliothèque partagée externe LibPNG: 1.2.46 LibQt: 4.8.2 LibRaw: 0.15.4 LibTIFF: LIBTIFF, Version 3.9.5 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc. Prise en charge du codec RawSpeed: Non LibGphoto2: 2.4.14 LibKface: 3.0.0 LibKipi: 2.1.0 LibOpenCV: 2.4.5 Modules externes KIPI: 3.5.0 Moteur de base de données: QMYSQL Serveur interne de base de données: Non Date de compilation : Oct 24 2013 (cible : debugfull) Rév. : c7dcf5224f6eb9c681707e6e0ca7578d40804152
Created attachment 83110 [details] digikam_thumbnail_rebuild.log
Created attachment 83113 [details] New crash information added by DrKonqi digikam (4.0.0-beta1) on KDE Platform 4.11.2 using Qt 4.8.2 - What I was doing when the application crashed: 1 - go to maintenance tool 2 - launch thumbnails rebuild with analyse check console log : digikam(15929)/digikam (core) Digikam::MaintenanceMngr::setSettings: wholeAlbums : true wholeTags : true Albums : 387 Tags : 495 useMutiCoreCPU : false newItems : false thumbnails : true scanThumbs : true fingerPrints : false scanFingerPrints : false duplicates : false similarity : 90 faceManagement : false faceScannedHandling : 0 qualitySort : false quality : EnableSorter : false DetectBlur : true DetectNoise : true DetectCompression : true LowQRejected : true MediumQPending : true HighQAccepted : true Speed : 1 Rejected Threshold : 10 Pending Threshold : 40 Accepted Threshold : 60 Blur Weight : 100 Noise Weight : 100 Compression Weight : 100 qualityScanMode : 0 metadataSync : false syncDirection : 0 digikam(15929)/digikam (core) Digikam::MaintenanceMngr::stage1: stage1 digikam(15929)/digikam (core) Digikam::MaintenanceMngr::stage2: stage2 digikam(15929)/KDCRAW KDcrawIface::RActionThreadBase::RActionThreadBase: Starting Main Thread digikam(15929)/KDCRAW KDcrawIface::RActionThreadBase::run: In action thread Run digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadStarted: RWeaverObserver: thread 1 started digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0019.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0020.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0021.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0022.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0023.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0024.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0025.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0026.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0027.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0028.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0029.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0030.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0031.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0032.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0033.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0034.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0035.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0036.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0037.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0038.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0039.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0040.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0041.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0042.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0043.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0044.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0045.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0046.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0047.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 1/0048.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0000.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0001.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0002.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0003.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0004.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0005.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0006.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0007.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0008.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0009.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0010.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0011.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0012.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0013.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 4/0014.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0000.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0001.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0002.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0003.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0004.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0005.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0006.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0007.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0008.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0009.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0010.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0011.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0012.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0013.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0014.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0015.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0016.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0017.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0018.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0019.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0020.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0021.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0022.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0023.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0024.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0025.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0026.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0027.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0028.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0030.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0031.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0032.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0033.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0034.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0035.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0036.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0037.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0038.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0039.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0040.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0041.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0042.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0043.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0044.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0045.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0046.jpg" : JPEG file identified digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0047.jpg" : JPEG file identified digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0048.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0049.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0050.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0051.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0052.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0053.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0054.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0055.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0056.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0057.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0058.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0059.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0060.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0061.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0062.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0063.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0064.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0065.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0066.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003 - 5/0067.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0000.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0001.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0002_000.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0003_001.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0004_002.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0005_003.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0006_004.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0007_000.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0008_001.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0009_002.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0010_003.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0011_004.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0012_000.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0013_001.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0014_002.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0015_003.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-2/0016_004.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0000_000.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0001_001.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0002_002.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0003_003.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0004_004.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0005.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0006.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0007.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0008.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0009.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0010.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0011.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0012.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0013.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0014.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0015.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0016.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0017.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0018.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy digikam(15929)/digikam (core) Digikam::DImg::load: "/media/DiskphotosTests/Saint Gilles 2003/Saint Gilles 2003-3/0019.jpg" : JPEG file identified digikam(15929)/KDCRAW KDcrawIface::RWeaverObserver::slotThreadBusy: RWeaverObserver: thread 1 busy Qt Concurrent has caught an exception thrown from a worker thread. This is not supported, exceptions thrown in worker threads must be caught before control returns to Qt Concurrent. terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc KCrash: Application 'digikam' crashing... KCrash: Attempting to start /usr/lib/kde4/libexec/drkonqi from kdeinit QSocketNotifier: Invalid socket 13 and type 'Read', disabling... QSocketNotifier: Invalid socket 16 and type 'Read', disabling... QSocketNotifier: Invalid socket 22 and type 'Read', disabling... QSocketNotifier: Invalid socket 66 and type 'Read', disabling... QSocketNotifier: Invalid socket 30 and type 'Read', disabling... digikam: Fatal IO error 9 (Mauvais descripteur de fichier) on X server :0.0. QSocketNotifier: Invalid socket 17 and type 'Read', disabling... -- Backtrace (Reduced): #8 0x02edc1df in __GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #9 0x02edf825 in __GI_abort () at abort.c:91 [...] #14 0x01eba5ee in QThreadPoolThread::run (this=0x9a870f0) at concurrent/qthreadpool.cpp:114 #15 0x01ec7eb0 in QThreadPrivate::start (arg=0x9a870f0) at thread/qthread_unix.cpp:307 #16 0x007d4d4c in start_thread (arg=0xa06d1b40) at pthread_create.c:308
Instead to use an external version of libPGF, can you force digiKam to recompile internal version of libpgf, where i DISABLED parallellization (using OpenMP) which is completely bugous. If you look into digiKam component list, you will see PGF parallelization disabled, but i'm not sure if this state is right here using an external version of PGF... Gilles Caulier
Rex, Look this entry about WebP support in KDELibs : https://git.reviewboard.kde.org/r/106300/ Sound like code have been submited. right ? Gilles
Rex, Look cppcheck reports into PGF code http://build.kde.org/view/extragear/job/digikam_master/800/cppcheckResult/ Just search "pgf" strings in this page. Gilles
Entries list from Coverity Scan about libpgf : 981290 Uninitialized scalar field Medium Dismissed 1 02/13/13 Unassigned Intentional Unspecified Ignore Other Uninitialized members CDecoder::CMacroBlock::CMacroBlock(CDecoder*) /mnt/devel/GIT/3.x/core/libs/3rdparty/libpgf/Decoder.h 981172 Out-of-bounds write High Dismissed 1 02/13/13 Unassigned Intentional Unspecified Ignore Other Memory - corruptions CEncoder::WriteValue(CSubband *, int) /mnt/devel/GIT/3.x/core/libs/3rdparty/libpgf/Encoder.cpp 981291 Uninitialized pointer field Medium Dismissed 1 02/13/13 Unassigned Intentional Unspecified Ignore Other Uninitialized members CEncoder::CEncoder(CPGFStream *, PGFPreHeader, PGFHeader, const PGFPostHeader &, unsigned long long &, bool) /mnt/devel/GIT/3.x/core/libs/3rdparty/libpgf/Encoder.cpp 981292 Uninitialized scalar field Medium Dismissed 1 02/13/13 Unassigned Intentional Unspecified Ignore Other Uninitialized members CEncoder::CMacroBlock::CMacroBlock(CEncoder*) /mnt/devel/GIT/3.x/core/libs/3rdparty/libpgf/Encoder.h 981188 Resource leak High Dismissed 9 02/13/13 Unassigned Intentional Unspecified Ignore Other Resource leaks CPGFImage::GetBitmap(int, unsigned char *, unsigned char, int *, bool (*)(double, bool, void *), void *) const /mnt/devel/GIT/3.x/core/libs/3rdparty/libpgf/PGFimage.cpp 981293 Uninitialized scalar field Medium Dismissed 1 02/13/13 Unassigned Intentional Unspecified Ignore Other Uninitialized members CPGFImage::CPGFImage() /mnt/devel/GIT/3.x/core/libs/3rdparty/libpgf/PGFimage.cpp 981497 Read from pointer after free High Dismissed 1 02/13/13 Unassigned False Positive Unspecified Undecided Other Memory - illegal accesses CPGFImage::Write(int, bool (*)(double, bool, void *), void *) /mnt/devel/GIT/3.x/core/libs/3rdparty/libpgf/PGFimage.cpp 981294 Uninitialized scalar field Medium Dismissed 1 02/13/13 Unassigned Intentional Unspecified Ignore Other Uninitialized members CSubband::CSubband() /mnt/devel/GIT/3.x/core/libs/3rdparty/libpgf/Subband.cpp Gilles
I saw the comments about parallelization and noted that I had checked the "work on all processor cores" checkbox in the tools->maintainance dialog. I unchecked that and ran the thumbnailer again. This time it ran to completion and only consumed 1.3GB of memory (that was not released when digikam finished), but at least it ran all the way through my 86000 pictures. Jim Shipman
No. parallelization of libPGF is to decode/encode image through OpenMP. In Maintenance tool, parallelization is performed to process more than one items at the same time. It use ThreadWaver API from KDELibs. It's completly different... In all case PGF must run in on thread. parallelization do not work and crash digiKam. I'm sure i tested it in-deep, without to found where is the problem to PGF code. This is why libPGF is in digiKam with this option forced to disabled. But, go back to this problem. Here digiKam do not crash imediatly, until memory is full due to memory leak. I'm sure that problem detected in libpgf by Coverity Scan tool is the source. As i'm not PGF algorithm developer, i cannot fix it easily. This must be done by PGF team. If PGF team do not respond, i need to found an alternative, as WebP codec for ex. Look like PGF project sound like in stand-by since a long moment. In opensource, it's really problematic... Gilles Caulier
Important : I suspect that a lots of bug-reports are relevant of PGF codec bugs. For ex in face management, we have serious and stranges crashes when face thumbs data are set/get to database. Gilles Caulier
Re: comment #14 I can confirm https://git.reviewboard.kde.org/r/110649/ landed in kde-runtime 4.11 branch
Thanks Rex, Sadly, under my Linux, i run KDE 4.10. But under my MacBook, it's last KDE 4.11. So i can add try to make a WebP wrapper for thumb database, using KImageIO plugin. Gilles
Note : another advantage to use WebP with digiKam thumbs DB is to drop definitively LibPGF code (yes another one!), and to remove another external dependency... Gilles Caulier
There is a simple test to confirm that memory leak is relevant of PGF codec implementation. In digiKam thumbnails database interface, we have code to change image compression format to store in DB. Currently PGF is hardcoded in this line : https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/threadimageio/thumbnailcreator.cpp#L687 Just change : dbInfo.type = DatabaseThumbnail::PGF; by : dbInfo.type = DatabaseThumbnail::JPEG; ... and recompile and install digiKam. Re-run maintenance tool and look if memory leak still here. Note, with this change, JPEG will be used instead PGF to store thumbnails in DB. Gilles Caulier
Marcel, Running digiKam into valgrind give me these backtraces : There is a lots of memory allocation into ThumbnailImageCatcher. This is not a memory leak. When digiKam is stopped, all memory is free properly. Look below : ==7668== 7,128,144 bytes in 15,102 blocks are possibly lost in loss record 25,758 of 25,761 ==7668== at 0x4C285F1: operator new(unsigned long) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==7668== by 0x80B40EE: QList<Digikam::ThumbnailImageCatcher::Private::CatcherResult>::node_construct(QList<Digikam::ThumbnailImageCatcher::Private::CatcherResult>::Node*, Digikam::ThumbnailImageCatcher::Private::CatcherResult const&) (qlist.h:372) ==7668== by 0x80B2D2D: QList<Digikam::ThumbnailImageCatcher::Private::CatcherResult>::append(Digikam::ThumbnailImageCatcher::Private::CatcherResult const&) (qlist.h:512) ==7668== by 0x80B197E: QList<Digikam::ThumbnailImageCatcher::Private::CatcherResult>::operator<<(Digikam::ThumbnailImageCatcher::Private::CatcherResult const&) (qlist.h:334) ==7668== by 0x80AFC76: Digikam::ThumbnailImageCatcher::slotThumbnailLoaded(Digikam::LoadingDescription const&, QImage const&) (thumbnailloadthread.cpp:1142) ==7668== by 0x80ABB70: Digikam::ThumbnailImageCatcher::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (thumbnailloadthread.moc:172) ==7668== by 0xF9AE8CE: QMetaObject::activate(QObject*, QMetaObject const*, int, void**) (in /usr/lib64/libQtCore.so.4.8.5) ==7668== by 0x808FCEA: Digikam::LoadSaveThread::signalThumbnailLoaded(Digikam::LoadingDescription const&, QImage const&) (loadsavethread.moc:174) ==7668== by 0x8090538: Digikam::LoadSaveThread::thumbnailLoaded(Digikam::LoadingDescription const&, QImage const&) (loadsavethread.cpp:204) ==7668== by 0x80ADFEC: Digikam::ThumbnailLoadThread::thumbnailLoaded(Digikam::LoadingDescription const&, QImage const&) (thumbnailloadthread.cpp:684) ==7668== by 0x80B5565: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:229) ==7668== by 0x809029C: Digikam::LoadSaveThread::run() (loadsavethread.cpp:136) ==7668== ==7668== 335,044,008 bytes in 709,839 blocks are possibly lost in loss record 25,761 of 25,761 ==7668== at 0x4C285F1: operator new(unsigned long) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==7668== by 0x80B40EE: QList<Digikam::ThumbnailImageCatcher::Private::CatcherResult>::node_construct(QList<Digikam::ThumbnailImageCatcher::Private::CatcherResult>::Node*, Digikam::ThumbnailImageCatcher::Private::CatcherResult const&) (qlist.h:372) ==7668== by 0x80B2D59: QList<Digikam::ThumbnailImageCatcher::Private::CatcherResult>::append(Digikam::ThumbnailImageCatcher::Private::CatcherResult const&) (qlist.h:521) ==7668== by 0x80B197E: QList<Digikam::ThumbnailImageCatcher::Private::CatcherResult>::operator<<(Digikam::ThumbnailImageCatcher::Private::CatcherResult const&) (qlist.h:334) ==7668== by 0x80AFC76: Digikam::ThumbnailImageCatcher::slotThumbnailLoaded(Digikam::LoadingDescription const&, QImage const&) (thumbnailloadthread.cpp:1142) ==7668== by 0x80ABB70: Digikam::ThumbnailImageCatcher::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (thumbnailloadthread.moc:172) ==7668== by 0xF9AE8CE: QMetaObject::activate(QObject*, QMetaObject const*, int, void**) (in /usr/lib64/libQtCore.so.4.8.5) ==7668== by 0x808FCEA: Digikam::LoadSaveThread::signalThumbnailLoaded(Digikam::LoadingDescription const&, QImage const&) (loadsavethread.moc:174) ==7668== by 0x8090538: Digikam::LoadSaveThread::thumbnailLoaded(Digikam::LoadingDescription const&, QImage const&) (loadsavethread.cpp:204) ==7668== by 0x80ADFEC: Digikam::ThumbnailLoadThread::thumbnailLoaded(Digikam::LoadingDescription const&, QImage const&) (thumbnailloadthread.cpp:684) ==7668== by 0x80B5565: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:229) ==7668== by 0x809029C: Digikam::LoadSaveThread::run() (loadsavethread.cpp:136) Typically, ThumbnailImageCatcher is called in ThumbsTask from maintenance tool : https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/utilities/maintenance/thumbstask.cpp#L76 and ThumbnailImageCatcher store thumbs in a QList cache here, without limit : https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/threadimageio/thumbnailloadthread.cpp#L1142 ... until memory is full... Question : there is something missing in ThumbsTask::run() to prevent this problem ? Gilles
Raphael, In same way that thumb cache memory allocation problem from digiKam core, we have also a memory problems with PGF codec : memory leak : ==7668== 2,208 bytes in 6 blocks are possibly lost in loss record 25,044 of 25,761 ==7668== at 0x4C26DFF: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==7668== by 0x4011008: _dl_allocate_tls (in /usr/lib64/ld-2.17.so) ==7668== by 0xFD008B8: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so) ==7668== by 0x10AAACBD: ??? (in /usr/lib64/libgomp.so.1.0.0) ==7668== by 0x80BC413: CPGFImage::WriteHeader(CPGFStream*) (PGFimage.cpp:924) ==7668== by 0x80BCE86: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1145) ==7668== by 0x80C876E: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:280) ==7668== by 0x80C7BF0: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:178) ==7668== by 0x80A8E99: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==7668== by 0x80A6818: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:267) ==7668== by 0x80A63C5: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==7668== by 0x80B51F9: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) memory corruption : ==7668== Thread 16: ==7668== Conditional jump or move depends on uninitialised value(s) ==7668== at 0x80C47CF: CSubband::Quantize(int) (Subband.cpp:132) ==7668== by 0x80C5772: CWaveletTransform::ForwardTransform(int, int) (WaveletTransform.cpp:164) ==7668== by 0x80C3023: CPGFImage::WriteHeader(CPGFStream*) [clone ._omp_fn.0] (PGFimage.cpp:949) ==7668== by 0x10AAA809: ??? (in /usr/lib64/libgomp.so.1.0.0) ==7668== by 0xFCFFD17: start_thread (in /usr/lib64/libpthread-2.17.so) ==7668== by 0x10FB67CC: clone (in /usr/lib64/libc-2.17.so) ==7668== ==7668== Conditional jump or move depends on uninitialised value(s) ==7668== at 0x80C4828: CSubband::Quantize(int) (Subband.cpp:134) ==7668== by 0x80C5772: CWaveletTransform::ForwardTransform(int, int) (WaveletTransform.cpp:164) ==7668== by 0x80C3023: CPGFImage::WriteHeader(CPGFStream*) [clone ._omp_fn.0] (PGFimage.cpp:949) ==7668== by 0x10AAA809: ??? (in /usr/lib64/libgomp.so.1.0.0) ==7668== by 0xFCFFD17: start_thread (in /usr/lib64/libpthread-2.17.so) ==7668== by 0x10FB67CC: clone (in /usr/lib64/libc-2.17.so) ==7668== ==7668== Thread 14: ==7668== Conditional jump or move depends on uninitialised value(s) ==7668== at 0x80B8979: CEncoder::WriteValue(CSubband*, int) (Encoder.cpp:327) ==7668== by 0x80B8640: CEncoder::Partition(CSubband*, int, int, int, int) (Encoder.cpp:258) ==7668== by 0x80C49FB: CSubband::ExtractTile(CEncoder&, bool, unsigned int, unsigned int) (Subband.cpp:185) ==7668== by 0x80BC9C4: CPGFImage::WriteLevel() (PGFimage.cpp:1029) ==7668== by 0x80BCD55: CPGFImage::WriteImage(CPGFStream*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1103) ==7668== by 0x80BCEA1: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1148) ==7668== by 0x80C876E: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:280) ==7668== by 0x80C7BF0: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:178) ==7668== by 0x80A8E99: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==7668== by 0x80A6818: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:267) ==7668== by 0x80A63C5: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==7668== by 0x80B51F9: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) ==7668== ==7668== Conditional jump or move depends on uninitialised value(s) ==7668== at 0x80B8979: CEncoder::WriteValue(CSubband*, int) (Encoder.cpp:327) ==7668== by 0x80B872D: CEncoder::Partition(CSubband*, int, int, int, int) (Encoder.cpp:283) ==7668== by 0x80C49FB: CSubband::ExtractTile(CEncoder&, bool, unsigned int, unsigned int) (Subband.cpp:185) ==7668== by 0x80BC9C4: CPGFImage::WriteLevel() (PGFimage.cpp:1029) ==7668== by 0x80BCD55: CPGFImage::WriteImage(CPGFStream*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1103) ==7668== by 0x80BCEA1: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1148) ==7668== by 0x80C876E: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:280) ==7668== by 0x80C7BF0: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:178) ==7668== by 0x80A8E99: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==7668== by 0x80A6818: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:267) ==7668== by 0x80A63C5: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==7668== by 0x80B51F9: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) Sure, memory leak with PGF codec is lesser than thumb cache from digiKam core, but it exists... See my comments #16 about Coverity Scan source code reports (https://scan.coverity.com/) which are fully relevant of this memory problems... Gilles Caulier
Rex, libpgf maintainer is now in CC. Wait and see... Gilles
Thank you Gilles, I will try this and the compilation digikam with internal PGF. Give you the result today or monday. Greatings, Eric Le samedi 26 octobre 2013 à 05:09 +0000, Gilles Caulier a écrit : > https://bugs.kde.org/show_bug.cgi?id=326525 > > --- Comment #23 from Gilles Caulier <caulier.gilles@gmail.com> --- > There is a simple test to confirm that memory leak is relevant of PGF codec > implementation. > > In digiKam thumbnails database interface, we have code to change image > compression format to store in DB. Currently PGF is hardcoded in this line : > > https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/threadimageio/thumbnailcreator.cpp#L687 > > Just change : > > dbInfo.type = DatabaseThumbnail::PGF; > > by : > > dbInfo.type = DatabaseThumbnail::JPEG; > > ... and recompile and install digiKam. Re-run maintenance tool and look if > memory leak still here. > > Note, with this change, JPEG will be used instead PGF to store thumbnails in > DB. > > Gilles Caulier >
Re Memory leak: Please use a dedicated thumbnail load thread (*) for the catcher. The logic depends on a enqueue being called after immediately after requesting a thumbnail, it breaks if the thread receives requests from many places concurrently so that the "last request" is undefined when calling enqueue. Re PGF: Even if we switch thumbnail creation to another algorithm, we a) need to continue providing PGF read support for those gigabytes of thumbnails on our users machines b) we break backwards compatibility to use a new thumbnail db with an older version, so it should be done only with a major release (*) ThumbnailLoadThread* thumbnailLoadThread = new ThumbnailLoadThread; thumbnailLoadThread->setPixmapRequested(false); thumbnailLoadThread->setThumbnailSize(ThumbnailLoadThread::maximumThumbnailSize()); Note that switching off pixmap generation for this thread also avoids flooding the cache for the main UI.
Git commit 21eb0ed5e895742228d789077d6590af9ff7ab36 by Gilles Caulier. Committed on 26/10/2013 at 21:18. Pushed by cgilles into branch 'master'. Fix memory leak with ThumbnailLoadThread instance passed to ThumbnailImageCatcher. Pass a dedicated instace of thread to cacher, following Marcel tips. FIXED-IN: 4.0.0 M +6 -4 utilities/maintenance/thumbstask.cpp http://commits.kde.org/digikam/21eb0ed5e895742228d789077d6590af9ff7ab36
Marcel, Thanks for your tips. Huge memory leak due to ThumbnailImageThread instance is now fixed. It still PGF codec memory leak. I will wait feedback of Raphael Schweizer from LibPGF team before to take a decision about to continue to use or not PGF as Thumb DB image codec. I know that a change like this require a backward compatibility until few release... If PGF implementation can be fixed,, it will be great for the future. I hope that PGF team is not dead... Gilles
At last, news from the libPGF team: we will immediately look into this memory leak problem. @Gilles: please apologize for the long silence. Unlike the coverity reports your contributions towards CMake and better binary compatibility are not lost nor forgotten. We started renewed efforts to migrate to CMake last week and will (hopefully soon) have good news. I'll also check the indicated coverity reports. Thanks, Raphael
Hi Raphael, Great to hear you in this room... I don't know if you have a Coverity Scan account to be able to check reports. I send you an invitation using your email, like this, you will be able to review all information from Coverity web interface. Gilles Caulier
Raphael, Coverity said "The following email addresses are invalid: rschweizer@schweizer-informatik.ch" Probably it don't like minus in your email domain name... Please use this page : https://scan.coverity.com/projects Search "digiKam" entry and use "Add me to project" link... Gilles Caulier
Raphael, About my previous libpgf contributions not yet release, i recommend to use Cmake port in priority. Forget BC branch for the moment. I will replay my fixes in this branch later, including these future memory fixes. Also i want to pass more test about BC code from libpgf. Cmake branch do not touch in-deep libpgf implementation. Most of code are cmake scripts. It safe to release. BC branch can come later... Gilles Caulier
Raphael, As you can see in comment #13, OpenMp support from libpgf do not work in digiKam. PGF codec in digiKam will run into a separated thread (QThread class : http://qt-project.org/doc/qt-4.8/qthread.html) and when libpgf paralellize decoding to get PGF thumbnail from database, it crash violently. I written simple a code outside wholde digiKam implementation to reproduce the problem : https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/tests/qtpgftest.cpp When libpgf do not use OpenMp, all work fine. I can give you GDB backtrace after that you fix memory corruption/leak previously listed here. Perhaps openmp has rise up some side-effects. Gilles Caulier
Gilles, I synced our internal repository with SourceForce, now containing at least a bunch of ROI related out of bounds errors less. This will, however, not fix the issue reported here. I'll keep you posted on our progress. Raphael Schweizer
(In reply to comment #28) > Re PGF: > Even if we switch thumbnail creation to another algorithm, we > a) need to continue providing PGF read support for those gigabytes of > thumbnails on our users machines > b) we break backwards compatibility to use a new thumbnail db with an older > version, so it should be done only with a major release I agree with Marcel. Also, the WebP is inferior in some ways to PGF (vastly reduced color depth support), but superior in others (animation, color management). Might be nice to use WebP in certain cases, like to create animated thumbnails for animated gifs or videos, but I suppose most users won't want their thumbnail list to be animated. Perhaps it could be an option if the user would like it.
Hi Ananta, I agree with you that PGF is superior than WebP. But the topic here is not to use WebP to store photo, but only thumbnails in database, compressed with Wavelets algorithm. In this case WebP can be suitable. We don't need 16 bits color depth. As you can see, i recieve PGF maintainer feedback and it's good for the future. If PGF memory corruption/leak problems are fixed there is no reason to switch to WebP. There is also this problem with OpenMP support in PGF which don't work. I just re-tested here and crash still here. I wait to see if PGF memory fixes improve OPenMP support before look in-deep in this problem. Gilles
Ananta, Just to talk more about wavelets image format, there is another challenger named JPEG-XR done by ... Microsoft, and valided as a standard bu JPEG group to include it in digital camera to replace usual JPEG ! This format is now available to developers though an "open-source like" library named "jxlib" https://jxrlib.codeplex.com/ Gilles Caulier
Raphael, Do you see my comment #33 ? I can add you to digiKam Coverity Scan project. Like this you can analyse in-deep all Coverity entries about libPGF. Reports are really detailled and instructive... Gilles Caulier
(In reply to comment #40) Yes, I applied for access using a different mail address (minus' not allowed at coverity). Raphael
Raphael, I send you invitation with your new email... As i list in comment #16, Coverity Scan entries ID relevant to libPGF are : - 981290 - 981172 - 981291 - 981292 - 981188 - 981293 - 981497 - 981294 Gilles Caulier
*** Bug 327033 has been marked as a duplicate of this bug. ***
Raphael, Any progress about libpgf memory leak fixes ? Gilles Caulier
Gilles, at last: - regarding memory leak @CPGFImage::WriteHeader(CPGFStream*) (PGFimage.cpp:924): we believe we only leaked memory if we eventually ran out of it (fixed in latest sourceforfe repo version) (though I'm not sure how OMP is related here) - uninitialized values @CEncoder::WriteValue(CSubband*, int) (Encoder.cpp:327) are false positives - uninitialized values @CSubband::Quantize(int) (Subband.cpp:134) are also believed to be false positives We've just set up a new linux box for more testing (including your test from comment #35) - Raphael
Hello, I know the question is not for me. But what I can say is that use of "internal libpgf" reduce most of the memory leak. Before the use of memory was growing fast to 1 gb and go away to crash. Now, for scanning the all collection (same condition and same pictures) it's almost use 400 mo for digikam to the end of thumbnail scan. Is libpgf has something to do with face scan and/or recognition : - thumbnails for faces storage in digikam thumb db? - and other stuff in recognition,db ? Greatings, Eric
Raphael, I will backport last libpgf code into digiKam core and look valgrind report to see if any improvements have been done about memory leak. Gilles Caulier
Eric, libpgf has nothing releated to face recognition memory leak. All problem with face management are already reported in another report and problem is relevant of OpenCV bugs. https://bugs.kde.org/show_bug.cgi?id=323888 Gilles Caulier
Gilles, Does digiKam "internal libpgf" have other improvements (comment #5) besides disabling OpenMP regarding better "memory performance"? I'd then patch our sources before you do the work of backporting. Also, I'd like to fix some "uninitialized values in constructor" issued reported by coverity (though I don't think they actually have any effect). - Raphael
Raphael, No, as i remember, i don't patch internal libpgf code. I just only include a cmake switch to enable/disable OpenMP support. That all... Gilles Caulier
Created attachment 83399 [details] Build traces for digikam with external libpgf
Created attachment 83400 [details] Build traces for digikam with inrenal libpgf
Git commit 8c41f33ff8489e863130003a6f2bfea1bb6393e5 by Gilles Caulier. Committed on 07/11/2013 at 14:02. Pushed by cgilles into branch 'master'. update internal libpgf source code to last revision #r121 from SF repository M +1 -1 libs/3rdparty/libpgf/Decoder.cpp M +1 -1 libs/3rdparty/libpgf/Decoder.h M +1 -1 libs/3rdparty/libpgf/Encoder.h M +26 -11 libs/3rdparty/libpgf/PGFimage.cpp M +4 -4 libs/3rdparty/libpgf/PGFplatform.h M +4 -4 libs/3rdparty/libpgf/PGFstream.h M +4 -4 libs/3rdparty/libpgf/PGFtypes.h M +2 -2 libs/3rdparty/libpgf/Subband.cpp M +56 -5 libs/3rdparty/libpgf/WaveletTransform.cpp http://commits.kde.org/digikam/8c41f33ff8489e863130003a6f2bfea1bb6393e5
Raphael, As you can see in my last comment, i commited last code from libpgf repository and recompiled digiKam. Running it through valgrind report again some issue about memory problems. Below, it's not a memory leak : =2438== Thread 21: ==2438== Conditional jump or move depends on uninitialised value(s) ==2438== at 0x80BAA10: CEncoder::CMacroBlock::NumberOfBitplanes() (Encoder.cpp:740) ==2438== by 0x80BA233: CEncoder::CMacroBlock::BitplaneEncode() (Encoder.cpp:495) ==2438== by 0x80B9E7A: CEncoder::EncodeBuffer(ROIBlockHeader) (Encoder.cpp:347) ==2438== by 0x80B9D2B: CEncoder::Flush() (Encoder.cpp:313) ==2438== by 0x80BE2CD: CPGFImage::WriteImage(CPGFStream*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1128) ==2438== by 0x80BE381: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1163) ==2438== by 0x80CA106: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==2438== by 0x80C9562: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:184) ==2438== by 0x80AA279: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==2438== by 0x80A7BF8: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:267) ==2438== by 0x80A77A5: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==2438== by 0x80B6685: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) ==2438== ==2438== Conditional jump or move depends on uninitialised value(s) ==2438== at 0x80BAA41: CEncoder::CMacroBlock::NumberOfBitplanes() (Encoder.cpp:741) ==2438== by 0x80BA233: CEncoder::CMacroBlock::BitplaneEncode() (Encoder.cpp:495) ==2438== by 0x80B9E7A: CEncoder::EncodeBuffer(ROIBlockHeader) (Encoder.cpp:347) ==2438== by 0x80B9D2B: CEncoder::Flush() (Encoder.cpp:313) ==2438== by 0x80BE2CD: CPGFImage::WriteImage(CPGFStream*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1128) ==2438== by 0x80BE381: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1163) ==2438== by 0x80CA106: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==2438== by 0x80C9562: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:184) ==2438== by 0x80AA279: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==2438== by 0x80A7BF8: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:267) ==2438== by 0x80A77A5: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==2438== by 0x80B6685: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) ==2438== At end of application, valgrind report memory leak. It's not too huge, because i stopped thumbnail maintenance task 30 secondes after to start (1% completed)... ==2438== 9,200 bytes in 25 blocks are possibly lost in loss record 25,377 of 25,727 ==2438== at 0x4C26DFF: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==2438== by 0x4011008: _dl_allocate_tls (in /usr/lib64/ld-2.17.so) ==2438== by 0xFD028B8: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so) ==2438== by 0x10AACCBD: ??? (in /usr/lib64/libgomp.so.1.0.0) ==2438== by 0x80BD89F: CPGFImage::WriteHeader(CPGFStream*) (PGFimage.cpp:924) ==2438== by 0x80BE366: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1160) ==2438== by 0x80CA106: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==2438== by 0x80C9562: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:184) ==2438== by 0x80AA279: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==2438== by 0x80A7BF8: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:267) ==2438== by 0x80A77A5: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==2438== by 0x80B6685: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) ==2438== 2,208 bytes in 6 blocks are possibly lost in loss record 24,976 of 25,727 ==2438== at 0x4C26DFF: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==2438== by 0x4011008: _dl_allocate_tls (in /usr/lib64/ld-2.17.so) ==2438== by 0xFD028B8: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so) ==2438== by 0x10AACCBD: ??? (in /usr/lib64/libgomp.so.1.0.0) ==2438== by 0x80BC372: CPGFImage::Read(int, bool (*)(double, bool, void*), void*) (PGFimage.cpp:428) ==2438== by 0x80C8C51: Digikam::PGFUtils::readPGFImageData(QByteArray const&, QImage&, bool) (pgfutils.cpp:93) ==2438== by 0x80AAE7E: Digikam::ThumbnailCreator::loadFromDatabase(Digikam::ThumbnailInfo const&) const (thumbnailcreator.cpp:886) ==2438== by 0x80A7ACC: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:248) ==2438== by 0x80A77A5: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==2438== by 0x80B6685: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) ==2438== by 0x8091550: Digikam::LoadSaveThread::run() (loadsavethread.cpp:136) ==2438== by 0x80CF451: Digikam::DynamicThread::DynamicThreadPriv::run() (dynamicthread.cpp:186) Gilles Caulier
Raphael, My last comment is about digiKam compiled with internal libpgf with OpenMP support disabled. Gilles Caulier
Created attachment 83403 [details] Patch out OpenMP #pragma s Gilles, first two must be false positives. The value in question (m_maxAbsValue) is initialized in a method (CMacroBlock::Init) called from the CMacroBlock constructor. Memory leaks are reported on lines "#pragma omp parallel for default(shared)". Seemingly OpenMP leaks some memory here, even though being disabled. Searching the internet leads to other people having similar problems. As I'm not an OpenMP guru my best guess is to patch these lines out for the time being. I'll see for someone more into OpenMP to investigate this issue. - Raphael
Gilles, Do you compile using -fopenmp and disable OpenMP just for libPGF (how, which flag?) or are you using gcc default (AFAIK no OpenMP)? I'm puzzled by the fact that libPGF is calling into libgomp. Is there anything we can do to better support disabling OpenMP? - Raphael (In reply to comment #55) > Raphael, > > My last comment is about digiKam compiled with internal libpgf with OpenMP > support disabled. > > Gilles Caulier
Raphael, At cmake configuration time, this flag is turned off (default) : https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/CMakeLists.txt#L17 Which is managed here to set compilation flags : https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/CMakeLists.txt#L272 In libpgf code, this flag is used here : https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/3rdparty/libpgf/PGFplatform.h#L608 I don't know why libpgf call libgomp here... Gilles
Gilles, nostra culpa, I'll upload a fixed version tonight. - Raphael
Gilles, r122 @sourceforge ready with guarded OpenMP #pragma s and some improvements regarding the coverity issues. - Raphael
Git commit b494c2a021e3fd85eeb1a5eba5a9e449e4dbe114 by Gilles Caulier. Committed on 08/11/2013 at 07:07. Pushed by cgilles into branch 'master'. update internal libpgf source code to last revision #r121 from SF repository M +2 -1 libs/3rdparty/libpgf/Decoder.cpp M +4 -0 libs/3rdparty/libpgf/Decoder.h M +8 -1 libs/3rdparty/libpgf/Encoder.cpp M +5 -1 libs/3rdparty/libpgf/Encoder.h M +8 -1 libs/3rdparty/libpgf/PGFimage.cpp M +16 -16 libs/3rdparty/libpgf/PGFplatform.h M +2 -2 libs/3rdparty/libpgf/PGFtypes.h M +6 -1 libs/3rdparty/libpgf/Subband.cpp http://commits.kde.org/digikam/b494c2a021e3fd85eeb1a5eba5a9e449e4dbe114
Sorry, commit from comment #61 is libpgf revision #122 from SF repository (not #121) Gilles Caulier
Raphael, All memory leak from libpgf are disappear with revision #122, but multiple conditional jump or move depends on uninitialized values (more than revision #121). Look below. This can be false warnings, but this must be checked : ==8229== Thread 17: ==8229== Conditional jump or move depends on uninitialised value(s) ==8229== at 0x80C5B55: CSubband::Quantize(int) (Subband.cpp:137) ==8229== by 0x80C6B0E: CWaveletTransform::ForwardTransform(int, int) (WaveletTransform.cpp:164) ==8229== by 0x80BDBA7: CPGFImage::WriteHeader(CPGFStream*) (PGFimage.cpp:961) ==8229== by 0x80BE720: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1167) ==8229== by 0x80C9F2E: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==8229== by 0x80C938A: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:184) ==8229== by 0x80AA259: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==8229== by 0x80A7BD8: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:267) ==8229== by 0x80A7785: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==8229== by 0x80B6665: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) ==8229== by 0x8091530: Digikam::LoadSaveThread::run() (loadsavethread.cpp:136) ==8229== by 0x80CF279: Digikam::DynamicThread::DynamicThreadPriv::run() (dynamicthread.cpp:186) ==8229== Conditional jump or move depends on uninitialised value(s) ==8229== at 0x80B9DDD: CEncoder::WriteValue(CSubband*, int) (Encoder.cpp:328) ==8229== by 0x80B9AA4: CEncoder::Partition(CSubband*, int, int, int, int) (Encoder.cpp:259) ==8229== by 0x80C5D81: CSubband::ExtractTile(CEncoder&, bool, unsigned int, unsigned int) (Subband.cpp:190) ==8229== by 0x80BE25E: CPGFImage::WriteLevel() (PGFimage.cpp:1051) ==8229== by 0x80BE5EF: CPGFImage::WriteImage(CPGFStream*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1125) ==8229== by 0x80BE73B: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1170) ==8229== by 0x80C9F2E: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==8229== by 0x80C938A: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:184) ==8229== by 0x80AA259: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==8229== by 0x80A7BD8: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:267) ==8229== by 0x80A7785: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==8229== by 0x80B6665: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) ==8229== Conditional jump or move depends on uninitialised value(s) ==8229== at 0x80BA7B8: CEncoder::CMacroBlock::DecomposeBitplane(unsigned int, unsigned int, unsigned int, unsigned int*, unsigned int*, unsigned int*, unsigned int&, unsigned int&) (Encoder.cpp:654) ==8229== by 0x80BA307: CEncoder::CMacroBlock::BitplaneEncode() (Encoder.cpp:520) ==8229== by 0x80B9E52: CEncoder::EncodeBuffer(ROIBlockHeader) (Encoder.cpp:348) ==8229== by 0x80B9D74: CEncoder::WriteValue(CSubband*, int) (Encoder.cpp:324) ==8229== by 0x80B9AA4: CEncoder::Partition(CSubband*, int, int, int, int) (Encoder.cpp:259) ==8229== by 0x80C5D81: CSubband::ExtractTile(CEncoder&, bool, unsigned int, unsigned int) (Subband.cpp:190) ==8229== by 0x80BE25E: CPGFImage::WriteLevel() (PGFimage.cpp:1051) ==8229== by 0x80BE5EF: CPGFImage::WriteImage(CPGFStream*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1125) ==8229== by 0x80BE73B: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1170) ==8229== by 0x80C9F2E: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==8229== by 0x80C938A: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:184) ==8229== by 0x80AA259: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==8229== Thread 16: ==8229== Conditional jump or move depends on uninitialised value(s) ==8229== at 0x80BA9F6: CEncoder::CMacroBlock::NumberOfBitplanes() (Encoder.cpp:747) ==8229== by 0x80BA219: CEncoder::CMacroBlock::BitplaneEncode() (Encoder.cpp:502) ==8229== by 0x80B9E52: CEncoder::EncodeBuffer(ROIBlockHeader) (Encoder.cpp:348) ==8229== by 0x80B9D03: CEncoder::Flush() (Encoder.cpp:314) ==8229== by 0x80BE687: CPGFImage::WriteImage(CPGFStream*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1135) ==8229== by 0x80BE73B: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1170) ==8229== by 0x80C9F2E: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==8229== by 0x80C938A: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:184) ==8229== by 0x80AA259: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==8229== by 0x80A7BD8: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:267) ==8229== by 0x80A7785: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==8229== by 0x80B6665: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) ==8229== ==8229== Conditional jump or move depends on uninitialised value(s) ==8229== at 0x80BAA27: CEncoder::CMacroBlock::NumberOfBitplanes() (Encoder.cpp:748) ==8229== by 0x80BA219: CEncoder::CMacroBlock::BitplaneEncode() (Encoder.cpp:502) ==8229== by 0x80B9E52: CEncoder::EncodeBuffer(ROIBlockHeader) (Encoder.cpp:348) ==8229== by 0x80B9D03: CEncoder::Flush() (Encoder.cpp:314) ==8229== by 0x80BE687: CPGFImage::WriteImage(CPGFStream*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1135) ==8229== by 0x80BE73B: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1170) ==8229== by 0x80C9F2E: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==8229== by 0x80C938A: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:184) ==8229== by 0x80AA259: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==8229== by 0x80A7BD8: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:267) ==8229== by 0x80A7785: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==8229== by 0x80B6665: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) Gilles Caulier
Raphael, Next stages here will are : - to turn on OpenMP support for libpgf and check if crash is reproducible as explained in comment #35. - to run again Coverity Scan to see if all main reports are fixed now. Gilles Caulier
Raphael, There are few warnings reported by cppcheck. Nothing urgent, just informative and good to know : [gilles@localhost libpgf]$ cppcheck --enable=all . ... [Decoder.h:46]: (style) The class 'CDecoder' does not have a constructor. [Decoder.cpp:499]: (style) Throwing a copy of the caught exception instead of rethrowing the original exception. [Decoder.cpp:536]: (warning) Redundant assignment of 'wordLen' to itself. [Decoder.cpp:595]: (warning) Redundant assignment of 'wordLen' to itself. [BitStream.h:129]: (style) The scope of the variable 'hiCount' can be reduced. [Decoder.h:46]: (style) 'class CDecoder' does not have a copy constructor which is recommended since the class contains a pointer to allocated memory. ... [Encoder.h:46]: (style) The class 'CEncoder' does not have a constructor. [Encoder.cpp:405]: (style) Variable 'h' is assigned a value that is never used. [Encoder.cpp:744]: (style) The scope of the variable 'cnt' can be reduced. [Encoder.cpp:570]: (style) Unsigned variable 'wordPos' can't be negative so it is unnecessary to test it. [Encoder.cpp:583]: (style) Unsigned variable 'wordPos' can't be negative so it is unnecessary to test it. [Encoder.cpp:593]: (style) Unsigned variable 'wordPos' can't be negative so it is unnecessary to test it. [Encoder.cpp:605]: (style) Unsigned variable 'wordPos' can't be negative so it is unnecessary to test it. [Encoder.cpp:614]: (style) Unsigned variable 'm_codePos' can't be negative so it is unnecessary to test it. [Encoder.cpp:769]: (style) Unsigned variable 'codePos' can't be negative so it is unnecessary to test it. [Encoder.h:46]: (style) 'class CEncoder' does not have a copy constructor which is recommended since the class contains a pointer to allocated memory. ... [PGFimage.cpp:1294]: (error) syntax error [PGFimage.cpp:637]: (style) Unsigned variable 'len' can't be negative so it is unnecessary to test it. [PGFimage.cpp:680]: (style) Unsigned variable 'len' can't be negative so it is unnecessary to test it. ... [gilles@localhost libpgf]$ Gilles Caulier
Rapahel, I don't check your change about OpenMP support, but now, crash cannot be reproduced as explained in comment #35. I can run qtpgftest command line tool : [gilles@localhost tests]$ ./qtpgftest unnamed app(22268)/digikam (core) main: Using LibPGF version: "6.13.45" unnamed app(22268)/digikam (core) main: Using OpenMP with LibPGF : yes unnamed app(22268)/digikam (core) main: Generate PGF file using file stream unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: RGB => ARGB unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: PGF image settings: unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: width: 256 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: height: 170 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: nLevels: 0 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: quality: 0 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: bpp: 32 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: channels: 4 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: mode: 17 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: usedBitsPerChannel: 0 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageFile: file size written : 80806 unnamed app(22268)/digikam (core) main: Generate PGF file using data stream unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageData: PGF stream memory allocation in bytes: 174080 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: RGB => ARGB unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: PGF image settings: unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: width: 256 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: height: 170 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: nLevels: 0 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: quality: 0 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: bpp: 32 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: channels: 4 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: mode: 17 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageDataToStream: usedBitsPerChannel: 0 unnamed app(22268)/digikam (core) Digikam::PGFUtils::writePGFImageData: data size written : 80806 unnamed app(22268)/digikam (core) main: PGF Encoding time: 0.08 s unnamed app(22268)/digikam (core) main: Load PGF file generated by data stream unnamed app(22268)/digikam (core) Digikam::PGFUtils::readPGFImageData: image data stream size is : 80806 unnamed app(22268)/digikam (core) Digikam::PGFUtils::readPGFImageData: PGF image is open unnamed app(22268)/digikam (core) Digikam::PGFUtils::readPGFImageData: PGF image is read unnamed app(22268)/digikam (core) Digikam::PGFUtils::readPGFImageData: PGF image is decoded unnamed app(22268)/digikam (core) main: PGF Decoding time: 0.1 s unnamed app(22268)/digikam (core) main: Generate JPG file to compare performances unnamed app(22268)/digikam (core) main: JPG Encoding time: 0 s unnamed app(22268)/digikam (core) main: JPG Decoding time: 0 s unnamed app(22268)/digikam (core) main: Generate PNG file to compare performances unnamed app(22268)/digikam (core) main: PNG Encoding time: 0.01 s libpng warning: Unknown iTXt compression type or method unnamed app(22268)/digikam (core) main: PNG Decoding time: 0 s [gilles@localhost tests]$ But valgrind report few traces about memory leak when OpenMP support is enabled : [gilles@localhost tests]$ valgrind --tool=memcheck --leak-check=full --error-limit=no ./qtpgftest ==22448== Memcheck, a memory error detector ==22448== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al. ==22448== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info ==22448== Command: ./qtpgftest ==22448== ... ==22448== ==22448== HEAP SUMMARY: ==22448== in use at exit: 2,219,171 bytes in 470 blocks ==22448== total heap usage: 23,798 allocs, 23,328 frees, 39,735,306 bytes allocated ... ==22448== 2,464 bytes in 7 blocks are possibly lost in loss record 247 of 260 ==22448== at 0x4C26DFF: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==22448== by 0x4011008: _dl_allocate_tls (in /usr/lib64/ld-2.17.so) ==22448== by 0x59CB8B8: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so) ==22448== by 0x4E38CBD: ??? (in /usr/lib64/libgomp.so.1.0.0) ==22448== by 0x40AD03: CPGFImage::WriteHeader(CPGFStream*) (PGFimage.cpp:930) ==22448== by 0x40B7C0: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1167) ==22448== by 0x417611: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==22448== by 0x4167B3: Digikam::PGFUtils::writePGFImageFile(QImage const&, QString const&, int, bool) (pgfutils.cpp:152) ==22448== by 0x402992: main (qtpgftest.cpp:63) ... ==22448== 1,032,472 bytes in 7 blocks are definitely lost in loss record 259 of 260 ==22448== at 0x4C285F1: operator new(unsigned long) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==22448== by 0x40686A: CEncoder::CEncoder(CPGFStream*, PGFPreHeader, PGFHeader, PGFPostHeader const&, unsigned long long&, bool) (Encoder.cpp:101) ==22448== by 0x40AE60: CPGFImage::WriteHeader(CPGFStream*) (PGFimage.cpp:982) ==22448== by 0x40B7C0: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1167) ==22448== by 0x417611: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==22448== by 0x416ABD: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:184) ==22448== by 0x402AF3: main (qtpgftest.cpp:74) ==22448== ==22448== 1,032,472 bytes in 7 blocks are definitely lost in loss record 260 of 260 ==22448== at 0x4C285F1: operator new(unsigned long) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==22448== by 0x40686A: CEncoder::CEncoder(CPGFStream*, PGFPreHeader, PGFHeader, PGFPostHeader const&, unsigned long long&, bool) (Encoder.cpp:101) ==22448== by 0x40AE60: CPGFImage::WriteHeader(CPGFStream*) (PGFimage.cpp:982) ==22448== by 0x40B7C0: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1167) ==22448== by 0x417611: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==22448== by 0x4167B3: Digikam::PGFUtils::writePGFImageFile(QImage const&, QString const&, int, bool) (pgfutils.cpp:152) ==22448== by 0x402992: main (qtpgftest.cpp:63) ==22448== ==22448== LEAK SUMMARY: ==22448== definitely lost: 2,066,960 bytes in 35 blocks ==22448== indirectly lost: 7,540 bytes in 63 blocks ==22448== possibly lost: 2,464 bytes in 7 blocks ==22448== still reachable: 142,207 bytes in 365 blocks ==22448== suppressed: 0 bytes in 0 blocks ==22448== Reachable blocks (those to which a pointer was found) are not shown. ==22448== To see them, rerun with: --leak-check=full --show-reachable=yes ==22448== ==22448== For counts of detected and suppressed errors, rerun with: -v ==22448== ERROR SUMMARY: 6 errors from 6 contexts (suppressed: 2 from 2) [gilles@localhost tests]$
Raphael, running digiKam into valgrind with libpgf OpenMP support confirm memory leak : ==22590== 368 bytes in 1 blocks are possibly lost in loss record 22,738 of 25,730 ==22590== at 0x4C26DFF: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==22590== by 0x4011008: _dl_allocate_tls (in /usr/lib64/ld-2.17.so) ==22590== by 0xFD028B8: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so) ==22590== by 0x10AACCBD: ??? (in /usr/lib64/libgomp.so.1.0.0) ==22590== by 0x80BDA49: CPGFImage::WriteHeader(CPGFStream*) (PGFimage.cpp:930) ==22590== by 0x80BE510: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1167) ==22590== by 0x80CA2EE: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==22590== by 0x80C974A: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:184) ==22590== by 0x80AA2F9: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==22590== by 0x80A7C78: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:267) ==22590== by 0x80A7825: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==22590== by 0x80B6705: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) ==22590== 5,309,856 bytes in 36 blocks are possibly lost in loss record 25,727 of 25,730 ==22590== at 0x4C285F1: operator new(unsigned long) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==22590== by 0x80B9531: CEncoder::CEncoder(CPGFStream*, PGFPreHeader, PGFHeader, PGFPostHeader const&, unsigned long long&, bool) (Encoder.cpp:101) ==22590== by 0x80BDBAC: CPGFImage::WriteHeader(CPGFStream*) (PGFimage.cpp:982) ==22590== by 0x80BE510: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1167) ==22590== by 0x80CA2EE: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==22590== by 0x80C974A: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:184) ==22590== by 0x80AA2F9: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==22590== by 0x80A7C78: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:267) ==22590== by 0x80A7825: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==22590== by 0x80B6705: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) ==22590== by 0x80915D0: Digikam::LoadSaveThread::run() (loadsavethread.cpp:136) ==22590== by 0x80CF639: Digikam::DynamicThread::DynamicThreadPriv::run() (dynamicthread.cpp:186) ==22590== 59,735,880 bytes in 405 blocks are definitely lost in loss record 25,730 of 25,730 ==22590== at 0x4C285F1: operator new(unsigned long) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==22590== by 0x80B9531: CEncoder::CEncoder(CPGFStream*, PGFPreHeader, PGFHeader, PGFPostHeader const&, unsigned long long&, bool) (Encoder.cpp:101) ==22590== by 0x80BDBAC: CPGFImage::WriteHeader(CPGFStream*) (PGFimage.cpp:982) ==22590== by 0x80BE510: CPGFImage::Write(CPGFStream*, unsigned int*, bool (*)(double, bool, void*), void*) (PGFimage.cpp:1167) ==22590== by 0x80CA2EE: Digikam::PGFUtils::writePGFImageDataToStream(QImage const&, CPGFStream&, int, unsigned int&, bool) (pgfutils.cpp:288) ==22590== by 0x80C974A: Digikam::PGFUtils::writePGFImageData(QImage const&, QByteArray&, int, bool) (pgfutils.cpp:184) ==22590== by 0x80AA2F9: Digikam::ThumbnailCreator::storeInDatabase(Digikam::ThumbnailInfo const&, Digikam::ThumbnailImage const&) const (thumbnailcreator.cpp:695) ==22590== by 0x80A7C78: Digikam::ThumbnailCreator::load(QString const&, QRect const&, bool) const (thumbnailcreator.cpp:267) ==22590== by 0x80A7825: Digikam::ThumbnailCreator::load(QString const&) const (thumbnailcreator.cpp:199) ==22590== by 0x80B6705: Digikam::ThumbnailLoadingTask::execute() (thumbnailtask.cpp:172) ==22590== by 0x80915D0: Digikam::LoadSaveThread::run() (loadsavethread.cpp:136) ==22590== by 0x80CF639: Digikam::DynamicThread::DynamicThreadPriv::run() (dynamicthread.cpp:186) Ask you can see, memory leaks are not negligible if OPenMP support is enabled. I always run digiKam in same condition : I stop thumbnails generator at 1% (else it too long under valgrind). Nota : digiKam do not crash imediatly when openMp support is enabled, as i have seen in the pass. To resume : - libpgf without OpenMP support : no memory leak. - libpgf with OpenMP support : no crash but important memory leak. Gilles Caulier
Gilles, - UNINIT/JUMP warnings are false positives (I'll check and mark accordingly on coverity when new analysis is available). - Memory leak @CEncoder::CEncoder(Encoder.cpp:101) FIXED (soon on SF); other OpenMP memory leak must remain low-priority for the time being (we do not have sufficient resources to check with valgrind/gcc/libgomp, if time allows we'll at least check with VC++ [and Intel]) - cppcheck: fixed two issued (others are related to little/big endian conditionals), soon on SF Next steps: Dario will intermittently work on CMake toolchain and eventually OpenMP issues - Raphael
r123 with said FIXes now available @SF
Git commit 2a5aadd60a23bd61b74722125dc25677556422a4 by Gilles Caulier. Committed on 08/11/2013 at 10:59. Pushed by cgilles into branch 'master'. update internal libpgf source code to last revision #r123 from SF repository M +1 -1 libs/3rdparty/libpgf/Decoder.cpp M +8 -3 libs/3rdparty/libpgf/Encoder.cpp http://commits.kde.org/digikam/2a5aadd60a23bd61b74722125dc25677556422a4
Raphael, About CMake toolchain, you just need to sync svn libpgf cmakeport branch with trunk, and sync back this branch in trunk to complete job. Gilles Caulier
Gilles, unfortunately it's not that trivial (for us) as we have > 10 different build configurations in our Visual Studio project. We'd like to change everything to CMake but need to understand its configuration first (i.e. how to generate our configurations from CMake). Your CMake branch obviously is an excellent starting point. Thanks! - Raphael
Raphael, To configure cmake to build VisualC++ project, just run this script : http://sourceforge.net/p/libpgf/code/HEAD/tree/cmakeport/libpgf/bootstrap.msvc.bat this will make nmake rules to compile under Windows console. You can change cmake -G option to make VisualC++ project in the same way. For ex, to generate Visual C++ 2010 SLN files : cmake -G "Visual Studio 10" ... as explained here : http://stackoverflow.com/questions/395169/using-cmake-to-generate-visual-studio-c-project-files Gilles Caulier
Gilles, thanks for the SO link. I found the following there: "One more important thing to know about is the lack of support (afaik) for 'Solution Configurations' in CMake. As it stands, you have to generate two directories with projects/solutions - one for each build type (debug, release, etc.). There is no direct support for more sophisticated features - in other words: switching between configurations won't give you what you might expect." which is exactly our problem. As stated in my last post we have > 10 such "configurations" for release, debug, omp, 16bit, ROI/nonROI and combinations thereof. In Visual Studio we can easily switch between them by just selecting from a drop down menu. We'd like to keep this as convenient as possible... In principle we "just" need to translate our configurations to CMake, but there seem to be related issues with the VS generator (see CMake bugtracker) and we need to distil workarounds from their mailing list as the documentation is allegedly out of sync with the implementation... which in practice makes this a tedious task. - Raphael
Raphael, Well about cmakeport branch, i recommend to merge it in trunk, including multiple configuration framework support for a transition time. This will make cmake support official for the next release and permit to use other way to configure and compile the library. A new release must be done in a near future to include your precious fixes... I want also to set digiKam libpgf dependency restricted to this new release to be able to remove later all libpgf source code from digiKam core. Gilles Caulier
As with commit of libpgf revision 123 fix memory leak when OpenMP is not used, i can close this file now... Gilles Caulier
*** Bug 327815 has been marked as a duplicate of this bug. ***
(In reply to comment #76) > As with commit of libpgf revision 123 fix memory leak when OpenMP is not > used, i can close this file now... > > Gilles Caulier I just downloaded libpgf r123 from svn and installed it. I checked with ldd that the library was actually used. It was. Problem is still there, memory usage skyrockets. Any clue?
(In reply to comment #78) > I just downloaded libpgf r123 from svn and installed it. I checked with ldd > that the library was actually used. It was. > > Problem is still there, memory usage skyrockets. Any clue? Without further information, hard to say. I assume you did use the code from trunk and not by chance from bincomp or cmakeport? Did you compile with disabled OpenMP support? We still have confirmed memory leaks when using libgomp (though these shouldn't be "skyrockets"). - Raphael
*** Bug 328732 has been marked as a duplicate of this bug. ***
To Raphael, I continue to talk in this thread about future official release of libpgf including all fixes recently added. What's the plan for you. I would to see libpgf released before digiKam 4.0.0 to be sure that all distro will use clean code. Also, did you take a look to all ccpcheck and Coverty reports about your code ? I can see some entries from Jenkins/ccpcheck in KDE server : http://build.kde.org/view/extragear/job/digikam_master/1195/cppcheckResult/ The digiKam 4.0.0 plan is listed here : http://www.digikam.org/about/releaseplan Also, in the future, i would to drop libpgf code from digiKam code to only use external shared library instead. Thanks to your feedback Gilles Caulier