Bug 323888

Summary: Face recognition makes digikam fill all the available memory (Qt SQlite plugin relevant)
Product: digikam Reporter: Alberto <alberto.ferrante>
Component: Database-FacesAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: crash CC: alexander.s.m, bugs, caulier.gilles, dion, e.longuemare, fassadourian, felix.leif, g.rass, jean-martial, jeffnik, kde-bugs, kde-bugs, kde-bugtracking, leviatan1, mail, manuel, marcel.wiesweg, mathieu.clabaut, metzpinguin, micha.magen, mick, nico.kruber, ogni, olivier.becquaert, peteroschi, raving, rf-kde, richm+kde, rob.dean, scott_bugs, simonandric5, tomas.kloucek
Priority: NOR    
Version: 4.2.0   
Target Milestone: ---   
Platform: Fedora RPMs   
OS: Linux   
Latest Commit: Version Fixed In: 4.9.0
Attachments: Valgrind report
valgrind --tool=memcheck --leak-check=full --error-limit=no digikam 2> digikam.leaks.err
Crash after full fill the memory - valgrind log
Crash after full fill the memory - valgrind log - run 2
Crash after full fill the memory - valgrind log - run 2
New crash information added by DrKonqi
Graph of physical (RSS) and virtual memory (VSZ)
Second Graph of physical (RSS) and virtual memory (VSZ)
gdb log with break set to Digikam::DImg::DImg(QImage const&)
Graph of physical (RSS) and virtual memory (VSZ) with break set to Digikam::DImg::DImg(QImage const&) in dgb
New crash information added by DrKonqi
New crash information added by DrKonqi
Crash after full fill the memory when scanning a new folder with new pictures
Crash after full fill the memory - gdb log -
gdb full log - full fill memory - cont to SIGALRM -
Variables when SigFault at dimg.cpp ligne 165
digikam 4.0 still leaking memory

Description Alberto 2013-08-22 10:10:58 UTC
When recognizing faces on large albums, after a while the recognition.db starts growing a lot, up to 200-something MB. After that size face recognition becomes impossible: for each modification, while digikam updates the face DB, it fills all available memory (28GB on my system, including swap space) and then it gets terminated by the OS.

Reproducible: Always

Steps to Reproduce:
1. Use a large set of photos (15-20'000 in my case)
2. Perform face detection
3. Tag people
4. Perform face recognition
Actual Results:  
The program starts filling all available memory and gets terminated by the operating system

Expected Results:  
Update the face db and recognize faces
Comment 1 caulier.gilles 2013-08-22 10:20:00 UTC
*** Bug 277163 has been marked as a duplicate of this bug. ***
Comment 2 caulier.gilles 2013-08-22 10:22:14 UTC
Can you run digiKam into valgrind as explained here :

http://www.digikam.org/contrib

and report valgrind trace here for future investigations...

Also we need to know which OpenCV version you use. Look into Help/Components Info dialog for details.

Gilles Caulier
Comment 3 Alberto 2013-08-22 11:08:50 UTC
LibOpenCV release was 2.4.5, I now updated it to 2.4.6 with the same results.
Valgrind report will follow.
Comment 4 Alberto 2013-08-22 14:00:42 UTC
Created attachment 81852 [details]
Valgrind report
Comment 5 Alexander Meshcheryakov 2013-08-25 14:03:38 UTC
Affects me as well. Arch linux, opencv 2.4.6.1

$ du -sh .kde4/share/apps/libkface/database/recognition.db                                                                                                                                      
57M     .kde4/share/apps/libkface/database/recognition.db

Why is it so big?
$ sqlite3 ~/.kde4/share/apps/libkface/database/recognition.db 'select count(*) from opencvlbphistograms;'
6915

I tried to delete recognition.db and run face recognition wizard with option "Clear and rebuildall training data", and after this operation I got recognition.db < 1MiB.
$ sqlite3 ~/.kde4/share/apps/libkface/database/recognition.db 'select count(*) from opencvlbphistograms;'
116
And I have exactly 116 face tags.

Original 57 MiB is compressible by xz down to 2%, recreated - down to 72%. Therefore I conclude 57MiB recognition.db had lots of duplicates.

After recreating recognition.db digikam does not concume all RAM in face related opeartions anymore.
Comment 6 Alexander Meshcheryakov 2013-08-25 14:31:38 UTC
Steps to reproduce growing recognition.db with duplicate records
1) re-launch Digikam, switch to people view, select face tag "Unknown".
2) Manually assign one face to specific human.
3) Wait 30seconds (it appears that there is "sleep" before DB update)

Outcome:
before operation you had N records in table "opencvlbphistograms",
after operation number of entries in "opencvlbphistograms" becomes 2*N+1
Comment 7 Flu87 2013-09-07 12:34:46 UTC
Created attachment 82206 [details]
valgrind --tool=memcheck --leak-check=full --error-limit=no digikam 2> digikam.leaks.err

me too ...

with valgrind it took too long, so I canceled it and closed digikam
and killed it after another hour...

hope it still helps ^^
Comment 8 e.longuemare 2013-10-01 12:55:09 UTC
Hello,

Digikam 3.5 build from git. OpenCV 2.4.5 build from source.

Scanning a collection of 28000 pictures (jpg, raw ...) with digikam use in gdb - nothing else running except a top in terminal. 

Digikam crash sometimes with this error :

what digikam was doing :

digikam(3834)/digikam (core) Digikam::DetectionWorker::process: Found 0 faces in "IMG_2389.CR2" QSize(2848, 4272) QSize(2848, 4272)
digikam(3834)/digikam (core) Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
digikam(3834)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam(3834)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation => Exif.Image.Orientation =>  8
digikam(3834)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  533   800  searchIncrement  1.1  grouping  3  flags  1  min size  26   26
digikam(3834)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam(3834)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation => Exif.Image.Orientation =>  8
digikam(3834)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(3834)/digikam (core) Digikam::DetectionWorker::process: Found 0 faces in "IMG_2390.CR2" QSize(2848, 4272) QSize(2848, 4272)
digikam(3834)/digikam (core) Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
digikam(3834)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam(3834)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation => Exif.Image.Orientation =>  8

error :
------
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x94db5b40 (LWP 3876)]
0xb6efd89a in Digikam::DImg::DImg (this=0x94db5124, image=...)
    at /home/eric/digikamgit/dk/core/libs/dimg/dimg.cpp:165
165	                dptr[0] = qBlue(*sptr);

Cordialement,

Eric
Comment 9 e.longuemare 2013-10-01 12:55:51 UTC
*** This bug has been confirmed by popular vote. ***
Comment 10 e.longuemare 2013-10-01 13:59:55 UTC
Hello,

Here some more info :

Reproducible : yes
How : restart computer (freeing memory, cache ..), then launch digikam in gdb (nothing else except a top in terminal), wait a few minutes while scanning pictures for faces (skip images already scan).

Res memory grow to 1.2 1.5 gb (60%)  when scan is launched.
then swap is used, swap  and load grow (especially with raw files). Then digikam crash with error behind :

dbg :

[New Thread 0x95db7b40 (LWP 2642)]
digikam(2423)/digikam (core) Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
digikam(2423)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam(2423)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation => Exif.Image.Orientation =>  1
digikam(2423)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  800   533  searchIncrement  1.1  grouping  3  flags  1  min size  26   26
digikam(2423)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation => Exif.Image.Orientation =>  1
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2423)/digikam (core) Digikam::DetectionWorker::process: Found 0 faces in "IMG_2467.CR2" QSize(4272, 2848) QSize(4272, 2848)
digikam(2423)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam(2423)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation => Exif.Image.Orientation =>  1
digikam(2423)/digikam (core) Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
digikam(2423)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam(2423)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation => Exif.Image.Orientation =>  1
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  800   533  searchIncrement  1.1  grouping  3  flags  1  min size  26   26
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2423)/digikam (core) Digikam::DetectionWorker::process: Found 0 faces in "IMG_2468.CR2" QSize(4272, 2848) QSize(4272, 2848)
digikam(2423)/digikam (core) Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  800   533  searchIncrement  1.1  grouping  3  flags  1  min size  26   26
digikam(2423)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam(2423)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation => Exif.Image.Orientation =>  1
digikam(2423)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam(2423)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation => Exif.Image.Orientation =>  1
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2423)/digikam (core) Digikam::DetectionWorker::process: Found 0 faces in "IMG_2469.CR2" QSize(4272, 2848) QSize(4272, 2848)
digikam(2423)/digikam (core) Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  800   533  searchIncrement  1.1  grouping  3  flags  1  min size  26   26
digikam(2423)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam(2423)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation => Exif.Image.Orientation =>  1

error :
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x94db5b40 (LWP 2502)]
0xb6efd89a in Digikam::DImg::DImg (this=0x94db5124, image=...)
    at /home/eric/digikamgit/dk/core/libs/dimg/dimg.cpp:165
165	                dptr[0] = qBlue(*sptr);

(gdb) backtrace :

#0  0xb6efd89a in Digikam::DImg::DImg (this=0x94db5124, image=...)
    at /home/eric/digikamgit/dk/core/libs/dimg/dimg.cpp:165
#1  0xb70d5d7d in Digikam::PreviewLoadingTask::execute (this=0xaca8d38)
    at /home/eric/digikamgit/dk/core/libs/threadimageio/previewtask.cpp:288
#2  0xb70c5205 in Digikam::LoadSaveThread::run (this=0xb033c38)
    at /home/eric/digikamgit/dk/core/libs/threadimageio/loadsavethread.cpp:136
#3  0xb70ecd58 in Digikam::DynamicThread::DynamicThreadPriv::run (
    this=0xb033cd0)
    at /home/eric/digikamgit/dk/core/libs/threads/dynamicthread.cpp:186
#4  0xb53d446b in QThreadPoolThread::run (this=0x96bc2a0)
    at concurrent/qthreadpool.cpp:107
#5  0xb53e1eb0 in QThreadPrivate::start (arg=0x96bc2a0)
    at thread/qthread_unix.cpp:307
#6  0xb36c1d4c in start_thread (arg=0x94db5b40) at pthread_create.c:308
#7  0xb510fdde in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130
Comment 11 e.longuemare 2013-10-01 19:00:19 UTC
Created attachment 82594 [details]
Crash after full fill the memory - valgrind log

View line 4197 -> no crash but halt (about 1 hour) and fill the memory then continue.
View line 5835 -> crash
Comment 12 e.longuemare 2013-10-01 19:06:37 UTC
(In reply to comment #11)
> Created attachment 82594 [details]
> Crash after full fill the memory - valgrind log
> 
 View line 4197 -> no crash but halt (about 1 hour - before the first warning) and fill the memory then continue.

 View line 5835 -> crash
Comment 13 e.longuemare 2013-10-02 08:06:05 UTC
Created attachment 82602 [details]
Crash after full fill the memory - valgrind log - run 2

Hello,

an other run in valgrind with the same result.
Hang from digikam in line 4189 and continue with a warning (always to the same picture):
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave (QRect(32,224 217x217) )
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(32,224 217x217) using cascade 0
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  297   297  searchIncrement  1.1  grouping  3  flags  0  min size  130   130
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(32,224 217x217) using cascade 1
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  297   297  searchIncrement  1.1  grouping  3  flags  0  min size  130   130
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(32,224 217x217) using cascade 2
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(32,224 217x217) using cascade 3
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  297   297  searchIncrement  1.1  grouping  3  flags  0  min size  130   130
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(32,224 217x217) using cascade 4
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  297   297  searchIncrement  1.1  grouping  3  flags  0  min size  130   130
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(32,224 217x217) using cascade 5
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0,0 0.6x0.6) QRect(32,224 217x217) QRect(32,224 130x130)
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  130   130  searchIncrement  1.1  grouping  2  flags  0  min size  22   22
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(32,224 217x217) using cascade 6
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0.4,0 0.6x0.6) QRect(32,224 217x217) QRect(119,224 130x130)
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  130   130  searchIncrement  1.1  grouping  2  flags  0  min size  22   22
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(32,224 217x217) using cascade 7
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0.2,0.25 0.6x0.6) QRect(32,224 217x217) QRect(75,278 130x130)
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  130   130  searchIncrement  1.1  grouping  2  flags  0  min size  22   22
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(32,224 217x217) using cascade 8
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0.1,0.4 0.8x0.6) QRect(32,224 217x217) QRect(54,311 174x130)
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  174   130  searchIncrement  1.1  grouping  2  flags  0  min size  0   0
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2423)/digikam (core) Digikam::DetectionWorker::process: Found 0 faces in "IMG_7905.jpg" QSize(4272, 2848) QSize(4272, 2848)

<---HERE--> HANG FOR PAST AN HOUR and continue- 
==2423== Warning: set address range perms: large range [0x49dc6028, 0x5a827018) (undefined)
==2423== Warning: set address range perms: large range [0x49dc6018, 0x5a827028) (noaccess)

<---HERE-->


Crash to line 5839 :
digikam(2423)/digikam (core) Digikam::DetectionWorker::process: Found 0 faces in "IMG_2479.CR2" QSize(4272, 2848) QSize(4272, 2848)
digikam(2423)/digikam (core) Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
digikam(2423)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  800   533  searchIncrement  1.1  grouping  3  flags  1  min size  26   26
digikam(2423)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam(2423)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation => Exif.Image.Orientation =>  1

<--CRASH-->

==2423== Thread 7:
==2423== Invalid read of size 4
==2423==    at 0x4D7C89A: Digikam::DImg::DImg(QImage const&) (dimg.cpp:165)
==2423==    by 0x4F54D7C: Digikam::PreviewLoadingTask::execute() (previewtask.cpp:288)
==2423==    by 0x4F44204: Digikam::LoadSaveThread::run() (loadsavethread.cpp:136)
==2423==    by 0x4F6BD57: Digikam::DynamicThread::DynamicThreadPriv::run() (dynamicthread.cpp:186)
==2423==    by 0x6A1246A: QThreadPoolThread::run() (in /usr/lib/i386-linux-gnu/libQtCore.so.4.8.2)
==2423==    by 0x6A1FEAF: QThreadPrivate::start(void*) (in /usr/lib/i386-linux-gnu/libQtCore.so.4.8.2)
==2423==    by 0x96CCD4B: start_thread (pthread_create.c:308)
==2423==    by 0x6F34DDD: clone (clone.S:130)
==2423==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==2423== 
KCrash: Application 'digikam' crashing...
Comment 14 e.longuemare 2013-10-02 08:54:40 UTC
Created attachment 82603 [details]
Crash after full fill the memory - valgrind log - run 2
Comment 15 e.longuemare 2013-10-03 11:36:08 UTC
Hello,

I've continue to test.

I have disable zram on my ubuntu kernel ( see here :https://bugs.launchpad.net/ubuntu/+source/linux-lts-raring/+bug/1217189)  : I use Xubuntu 12.04 LTS with :
Linux version 3.2.0-54-generic-pae (buildd@batsu) (gcc version 4.
6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #82-Ubuntu SMP Tue Sep 10 20:29:22 UTC 2013
 (Ubuntu 3.2.0-54.82-generic-pae 3.2.50)

I was having those errors :
[ 1.864606] Adding 1922976k swap on /dev/zram0. Priority:100 extents:1 across:1922976k SS
[ 30.459194] Buffer I/O error on device zram0, logical block 480744
[ 30.459234] Buffer I/O error on device zram0, logical block 480744
[ 30.459313] Buffer I/O error on device zram0, logical block 480744
[ 30.459349] Buffer I/O error on device zram0, logical block 480744
[ 30.459384] Buffer I/O error on device zram0, logical block 480744

in dmesg

And Digikam doesn't full fill the memory nor Crash anymore : run a face scan to 100 %

How should I test again :

suppress face database file and rescan again ? more files to suppress ?

Greatings,

Eric
Comment 16 e.longuemare 2013-10-05 07:49:42 UTC
Hello,

Have a few more tests.

I was having some problem with ZRAM, but the bug still occur when do a
scan with option :  Clear unconfirmed results and rescan. (restart scan
for beginning except for result wich have been confirmed - I suppose).

Detection and recognition begin from root of the collection to the few 1
rst albums then crash with same error as my precedent posts :

GDB : 
error :
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x94db5b40 (LWP 2502)]
0xb6efd89a in Digikam::DImg::DImg (this=0x94db5124, image=...)
    at /home/eric/digikamgit/dk/core/libs/dimg/dimg.cpp:165
165                    dptr[0] = qBlue(*sptr);

(gdb) backtrace :

#0  0xb6efd89a in Digikam::DImg::DImg (this=0x94db5124, image=...)
    at /home/eric/digikamgit/dk/core/libs/dimg/dimg.cpp:165
#1  0xb70d5d7d in Digikam::PreviewLoadingTask::execute (this=0xaca8d38)

at /home/eric/digikamgit/dk/core/libs/threadimageio/previewtask.cpp:288
#2  0xb70c5205 in Digikam::LoadSaveThread::run (this=0xb033c38)

at /home/eric/digikamgit/dk/core/libs/threadimageio/loadsavethread.cpp:136
#3  0xb70ecd58 in Digikam::DynamicThread::DynamicThreadPriv::run (
    this=0xb033cd0)
    at /home/eric/digikamgit/dk/core/libs/threads/dynamicthread.cpp:186
#4  0xb53d446b in QThreadPoolThread::run (this=0x96bc2a0)
    at concurrent/qthreadpool.cpp:107
#5  0xb53e1eb0 in QThreadPrivate::start (arg=0x96bc2a0)
    at thread/qthread_unix.cpp:307
#6  0xb36c1d4c in start_thread (arg=0x94db5b40) at pthread_create.c:308
#7  0xb510fdde in clone ()
at ../sysdeps/unix/sysv/linux/i386/clone.S:130 


Valgrind :

digikam(2423)/digikam (core) Digikam::DMetadata::getIccProfile: Exif
color-space tag is sRGB. Using default sRGB ICC profile.
digikam(2423)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation:
Orientation =>
Exif.Image.Orientation =>  1

<--CRASH-->

==2423== Thread 7:
==2423== Invalid read of size 4
==2423==    at 0x4D7C89A: Digikam::DImg::DImg(QImage const&)
(dimg.cpp:165)
==2423==    by 0x4F54D7C: Digikam::PreviewLoadingTask::execute()
(previewtask.cpp:288)
==2423==    by 0x4F44204: Digikam::LoadSaveThread::run()
(loadsavethread.cpp:136)
==2423==    by 0x4F6BD57:
Digikam::DynamicThread::DynamicThreadPriv::run()
(dynamicthread.cpp:186)
==2423==    by 0x6A1246A: QThreadPoolThread::run() (in
/usr/lib/i386-linux-gnu/libQtCore.so.4.8.2)
==2423==    by 0x6A1FEAF: QThreadPrivate::start(void*) (in
/usr/lib/i386-linux-gnu/libQtCore.so.4.8.2)
==2423==    by 0x96CCD4B: start_thread (pthread_create.c:308)
==2423==    by 0x6F34DDD: clone (clone.S:130)
==2423==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==2423== 


This is on digikam 3.5 build from git with opencv 2.4.5 built from
opencv repository.

Build date: Sep 30 2013 (target: debugfull)
Rev.: 04ec199e75e8a6bc01f29bac3a9476e3b72960e0

Components :

digiKam version 3.5.0
Exiv2 can write to Jp2: Yes
Exiv2 can write to Jpeg: Yes
Exiv2 can write to Pgf: Yes
Exiv2 can write to Png: Yes
Exiv2 can write to Tiff: Yes
Exiv2 supports XMP metadata: Yes
LibCImg: 130
LibEigen: 3.0.5
LibExiv2: 0.22
LibJPEG: 80
LibJasper: 1.900.1
LibKDE: 4.11.1
LibKExiv2: 2.3.1
LibKGeoMap: 2.0.0
LibKdcraw: 2.3.1
LibLCMS: 2040
LibLensFun: 0.2.7-0
LibPGF: 6.12.24 - external shared library
LibPNG: 1.2.46
LibQt: 4.8.2
LibRaw: 0.15.4
LibTIFF: LIBTIFF, Version 3.9.5 Copyright (c) 1988-1996 Sam Leffler
Copyright (c) 1991-1996 Silicon Graphics, Inc.
Marble Widget: 0.16.1 (stable version)
Parallelized PGF codec: No
Parallelized demosaicing: Yes
RawSpeed codec support: Yes
Database backend: QMYSQL
Database internal server: No
Kipi-Plugins: 3.5.0
LibGphoto2: 2.4.14
LibKface: 3.0.0
LibKipi: 2.1.0
LibOpenCV: 2.4.5


digiKam version 3.5.0
Images: 
BMP: 5
GIF: 56
JP2: 16
JPG: 22730
PNG: 130
RAW-CR2: 3432
XCF: 209
total: 26578
: 
: 
Videos: 
AVI: 29
MOV: 185
MP4: 43
MPEG: 2
WMV: 1
total: 260
: 
: 
Total Items: 26838
Albums: 2749
Tags: 492
Database backend: QMYSQL
Database internal server: No





Greatings,

Eric
Comment 17 e.longuemare 2013-10-05 07:52:29 UTC
Hello,

Looking at this bug : https://bugs.kde.org/show_bug.cgi?id=277169


I look for 0 size files in album :

find -type f -size 0
./La creche/Kayak Fishing.flv

Delete this files but he wasn't in the album.

Look at the DB :

SELECT * FROM digikam.Images where name like "%.flv";

But nothing : .flv are not collected.

Greatings,

Eric
Comment 18 e.longuemare 2013-10-07 12:26:03 UTC
Created attachment 82701 [details]
New crash information added by DrKonqi

digikam (3.5.0) on KDE Platform 4.11.2 using Qt 4.8.2

- What I was doing when the application crashed: scanning faces RAW files - memory fill up and digikam crash.

I've updated libkdcraw from kde 12.04 backport repository.

-- Backtrace (Reduced):
#8  0xb48661df in __GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#9  0xb4869825 in __GI_abort () at abort.c:91
[...]
#14 0xb4beb5ee in QThreadPoolThread::run (this=0x99df7e0) at concurrent/qthreadpool.cpp:114
#15 0xb4bf8eb0 in QThreadPrivate::start (arg=0x99df7e0) at thread/qthread_unix.cpp:307
#16 0xb2c5cd4c in start_thread (arg=0x9b5c6b40) at pthread_create.c:308
Comment 19 e.longuemare 2013-10-07 12:44:38 UTC
Digikam components :

digiKam version 3.5.0
Exiv2 can write to Jp2: Yes
Exiv2 can write to Jpeg: Yes
Exiv2 can write to Pgf: Yes
Exiv2 can write to Png: Yes
Exiv2 can write to Tiff: Yes
Exiv2 supports XMP metadata: Yes
LibCImg: 130
LibEigen: 3.0.5
LibExiv2: 0.22
LibJPEG: 80
LibJasper: 1.900.1
LibKDE: 4.11.2
LibKExiv2: 2.3.1
LibKGeoMap: 2.0.0
LibKdcraw: 2.3.1-2 from kubuntu backport repository
LibLCMS: 2040
LibLensFun: 0.2.7-0
LibPGF: 6.12.24 - external shared library
LibPNG: 1.2.46
LibQt: 4.8.2
LibRaw: 0.15.3
LibTIFF: LIBTIFF, Version 3.9.5 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc.
Marble Widget: 0.16.1 (stable version)
Parallelized PGF codec: No
Parallelized demosaicing: No
RawSpeed codec support: No
Database backend: QMYSQL
Database internal server: No
Kipi-Plugins: 3.5.0
LibGphoto2: 2.4.14
LibKface: 3.0.0
LibKipi: 2.1.0
LibOpenCV: 2.4.5
Comment 20 e.longuemare 2013-10-07 15:39:06 UTC
Hello,

After updates it's still the same error in Valgring :

Thread 5:
==3494== Invalid read of size 4
==3494==    at 0x4D0489A: Digikam::DImg::DImg(QImage const&) (dimg.cpp:165)
==3494==    by 0x4EDCD7C: Digikam::PreviewLoadingTask::execute() (previewtask.cpp:288)
==3494==    by 0x4ECC204: Digikam::LoadSaveThread::run() (loadsavethread.cpp:136)
==3494==    by 0x4EF3D57: Digikam::DynamicThread::DynamicThreadPriv::run() (dynamicthread.cpp:186)
==3494==    by 0x699A46A: QThreadPoolThread::run() (in /usr/lib/i386-linux-gnu/libQtCore.so.4.8.2)
==3494==    by 0x69A7EAF: QThreadPrivate::start(void*) (in /usr/lib/i386-linux-gnu/libQtCore.so.4.8.2)
==3494==    by 0x98D5D4B: start_thread (pthread_create.c:308)
==3494==    by 0x6EBCDDD: clone (clone.S:130)
==3494==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
Comment 21 e.longuemare 2013-10-07 16:03:14 UTC
Note that those lines disappeared from logs (digikam in terminal, gdb, and valgrind) :
digikam(2423)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation => Exif.Image.Orientation => 1

Another point is that face scan (finding face but without recognition) was running before : there was faces in unknown but they were not automatically tagged.

So I think this is an issue from modifications since activation of face recognition or recognition.db and perhaps related to the camera RAW files (libkdcraw, libraw issue ?).

Process for jpg files and raw files is different : no file generation for jpg.

RAW files are from a canon EOS 1100D.

Greatings,

Eric
Comment 22 e.longuemare 2013-10-08 14:01:35 UTC
Hello,

Runing GDB with breakpoints ... I have those lines :

Digikam::LoadSaveThread::run (this=0xa0f4c20) at /home/eric/digikamgit/dk/core/libs/threadimageio/loadsavethread.cpp:106
106	    while (runningFlag())
(gdb) 
139	}
(gdb) 
Digikam::DynamicThread::DynamicThreadPriv::run (this=0x9d601d8) at /home/eric/digikamgit/dk/core/libs/threads/dynamicthread.cpp:193
193	    if (emitSignals)
(gdb) 
198	    transitionToInactive();
(gdb) 
200	}
(gdb) 
QThreadPoolThread::run (this=0x96aec70) at concurrent/qthreadpool.cpp:117
117	concurrent/qthreadpool.cpp: Aucun fichier ou dossier de ce type.

between each (thread/picture) for face scan (RAW files).

Is it usual or not ?

PS : i'm able to locate qthreadpool.h but no qthreadpoll.cpp
Comment 23 caulier.gilles 2013-10-08 14:38:32 UTC
Hi,

Typically, loadsavethread will load Raw image data in a separated thread to be able to parse image contents for face.

qthreadpoll.cpp come from Qt4. GDB cannot found it because you don't have installed Qt source package on your computer.

In simple way, just download Qt4 source code and look cpp file accordingly

Gilles Caulier
Comment 24 e.longuemare 2013-10-09 13:24:07 UTC
Created attachment 82746 [details]
Graph of physical (RSS) and virtual memory (VSZ)

Hello,

A graph (with bash script on ps and gnuplot) of the memory usage for digikam when scanning face of one album  with option "clear unconfirmed results and rescan".

The increase is when the scan is run. Never end and digikam crash. The size of recognition.db doesn't move.

Thanks,

Eric

PS :
Comment 25 e.longuemare 2013-10-09 13:25:59 UTC
Created attachment 82747 [details]
Second Graph of physical (RSS) and virtual memory (VSZ)

Same conditions
Comment 26 e.longuemare 2013-10-09 16:09:48 UTC
Created attachment 82751 [details]
gdb log with break set to Digikam::DImg::DImg(QImage const&)
Comment 27 e.longuemare 2013-10-09 16:11:00 UTC
Created attachment 82752 [details]
Graph of physical (RSS) and virtual memory (VSZ) with break set to Digikam::DImg::DImg(QImage const&) in dgb
Comment 28 e.longuemare 2013-10-09 16:31:26 UTC
Hello,

I run digikam in dgb with a break point set to Digikam::DImg::DImg(QImage const&).

Clearly, when this is executed, no memory is freed :
1 --- first stair 
(gdb) c
Continuing.
[Thread 0x9be24b40 (LWP 13898) exited]
digikam(8415)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.

Breakpoint 2, Digikam::DImg::DImg (this=0xa2cf1124, image=...)
    at /home/eric/digikamgit/dk/core/libs/dimg/dimg.cpp:140
140	    : m_priv(new Private)


It's the "stair effect". Memory grow and doesn't go down.

2 -- second stair go down to same level before executed (1-- level)

When this is executed :

(gdb) c
Continuing.
digikam(8415)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam(8415)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(8415)/digikam (core) Digikam::DetectionWorker::process: Found 0 faces in "IMG_2793.CR2" QSize(4272, 2848) QSize(4272, 2848)
digikam(8415)/digikam (core) Digikam::DatabaseWriter::process: Removing old entries ()
digikam(8415)/digikam (core) Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
digikam(8415)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  800   533  searchIncrement  1.1  grouping  3  flags  1  min size  26   26

Breakpoint 2, Digikam::DImg::DImg (this=0x9be24124, image=...)
    at /home/eric/digikamgit/dk/core/libs/dimg/dimg.cpp:140
140	    : m_priv(new Private)

Memory grow and go down 

3 --- downstairs - more memory than "lost" in 1-- is back 

When this occure (only 2 times) :

(gdb) c
Continuing.
[New Thread 0xa36ceb40 (LWP 14797)]
digikam(8415)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(8415)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam(8415)/digikam (core) Digikam::DetectionWorker::process: Found 0 faces in "IMG_2814.CR2" QSize(4272, 2848) QSize(4272, 2848)
digikam(8415)/digikam (core) Digikam::DatabaseWriter::process: Removing old entries ()
digikam(8415)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  800   533  searchIncrement  1.1  grouping  3  flags  1  min size  26   26
digikam(8415)/digikam (core) Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
digikam(8415)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(8415)/digikam (core) Digikam::DetectionWorker::process: Found 0 faces in "IMG_2815.CR2" QSize(4272, 2848) QSize(4272, 2848)
digikam(8415)/digikam (core) Digikam::DatabaseWriter::process: Removing old entries ()
[New Thread 0xa2cf1b40 (LWP 14808)]
digikam(8415)/digikam (core) Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
digikam(8415)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  800   533  searchIncrement  1.1  grouping  3  flags  1  min size  26   26

Breakpoint 2, Digikam::DImg::DImg (this=0x9be24124, image=...)
    at /home/eric/digikamgit/dk/core/libs/dimg/dimg.cpp:140
140	    : m_priv(new Private)
(gdb) c

Two pictures are proceed and memory go down more.
Those are the U on the graph.

So as step 1-- and 2-- occure more than step 3-- memory use go up and up, and digikam crash.

Greatings,

Eric

Ps : each stair on the graph is a continue on gdb, then wait and do another continue.
Comment 29 e.longuemare 2013-10-09 16:32:59 UTC
PS : look at this graph : https://bugs.kde.org/attachment.cgi?id=82752
gdb logs are here : https://bugs.kde.org/attachment.cgi?id=82751
Comment 30 e.longuemare 2013-10-10 22:44:58 UTC
Created attachment 82781 [details]
New crash information added by DrKonqi

digikam (3.5.0) on KDE Platform 4.11.2 using Qt 4.8.2

- What I was doing when the application crashed:

Scanning an album with option  Clear unconfirmed results and rescan

-- Backtrace (Reduced):
#8  0xb48b61df in __GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#9  0xb48b9825 in __GI_abort () at abort.c:91
[...]
#14 0xb4c3b5ee in QThreadPoolThread::run (this=0x988a018) at concurrent/qthreadpool.cpp:114
#15 0xb4c48eb0 in QThreadPrivate::start (arg=0x988a018) at thread/qthread_unix.cpp:307
#16 0xb2cacd4c in start_thread (arg=0x9de19b40) at pthread_create.c:308
Comment 31 e.longuemare 2013-10-13 08:38:47 UTC
Created attachment 82819 [details]
New crash information added by DrKonqi

digikam (3.5.0) on KDE Platform 4.11.2 using Qt 4.8.2

- What I was doing when the application crashed:

Scanning a new folders with new pictures (RAW and JPG), with option skip already scanned image ( I try to scan this folder one time before and digikam crashed with same traces).

-- Backtrace (Reduced):
#8  0xb48331df in __GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#9  0xb4836825 in __GI_abort () at abort.c:91
[...]
#14 0xb4bb85ee in QThreadPoolThread::run (this=0xa9d3270) at concurrent/qthreadpool.cpp:114
#15 0xb4bc5eb0 in QThreadPrivate::start (arg=0xa9d3270) at thread/qthread_unix.cpp:307
#16 0xb2c29d4c in start_thread (arg=0x9ed97b40) at pthread_create.c:308
Comment 32 caulier.gilles 2013-10-13 08:43:48 UTC
As Marcel said previously, there is a no catched exception when digiKam crash

You backtrace is uncomplete because you don't get the exception properly.

To do it, follow this tutorial and report relevant and suitable backtrace :

http://techbase.kde.org/Development/Tutorials/Debugging/How_to_create_useful_crash_reports#Retrieving_a_backtrace_when_an_uncaught_exception_is_causing_a_crash

Gilles Caulier
Comment 33 e.longuemare 2013-10-13 08:48:21 UTC
Hello,

All the traces for the crash described in precedent post (scan faces in a new folder with option : skip already scanned pictures):

digikam
digikam(2380)/digikam (core) Digikam::KInotify::Private::open: Successfully opened connection to inotify: 15
digikam(2380)/digikam (core) Digikam::AlbumManager::setDatabase: DatabaseParameters: [ Type "QMYSQL", Name "digikam" (Thumbnails Name "digikam_thumb"); Host Name and Port: "192.168.1.2" 3306; Username and Password: "***", "****"]
QDBusConnection: name 'org.freedesktop.UDisks2' had owner '' but we thought it was ':1.45'
QSqlDatabasePrivate::removeDatabase: connection 'ConnectionTest' is still in use, all queries will cease to work.
digikam(2380)/digikam (core) Digikam::DatabaseConfigElementLoader::readConfig: Loading SQL code from config file "/usr/share/kde4/apps/digikam/database/dbconfig.xml"
digikam(2380)/digikam (core) Digikam::DatabaseConfigElementLoader::readConfig: false "1" 1 1
digikam(2380)/digikam (core) Digikam::SchemaUpdater::update: SchemaUpdater update
digikam(2380)/digikam (core) Digikam::SchemaUpdater::startUpdates: Have a database structure version  7
digikam(2380)/digikam (core) Digikam::SchemaUpdater::makeUpdates: makeUpdates  7  to  7
"/org/freedesktop/UDisks2/drives/SAMSUNG_HM160HC_S12TJDZB314313" : property "DeviceNumber" does not exist 
"/org/freedesktop/UDisks2/drives/SAMSUNG_HM160HC_S12TJDZB314313" : property "Device" does not exist 
digikam(2380)/digikam (core) Digikam::AlbumRootLocation::AlbumRootLocation: Creating new Location  "/"  uuid  "networkshareid:?mountpath=%2Fmedia%2FDiskphotos"
digikam(2380)/digikam (core) Digikam::CollectionManager::updateLocations: location for  "/media/Diskphotos"  is available  true
digikam(2380)/digikam (core) Digikam::KMemoryInfo::update: Platform identified :  "LINUX"
digikam(2380)/digikam (core) Digikam::KMemoryInfo::bytes: TotalRam:  2110857216
digikam(2380)/digikam (core) Digikam::LoadingCache::setCacheSize: Allowing a cache size of 100 MB
digikam(2380)/digikam (core) Digikam::ThumbnailSchemaUpdater::startUpdates: Have a thumbnail database structure version  "2"
digikam(2380)/digikam (core) Digikam::ThumbnailLoadThread::initializeThumbnailDatabase: Thumbnail db ready for use
digikam(2380)/digikam (core) Digikam::CollectionScanner::completeScan: Complete scan (file scanning deferred) took: 13646 msecs.
digikam(2380)/digikam (core) Digikam::IccSettings::Private::scanDirectories: ("/usr/share/color/icc", "/home/eric/.color/icc")
digikam(2380)/KFACE KFaceIface::SchemaUpdater::startUpdates: Have a database structure version  "1"
digikam(2380)/KGEOMAP KGeoMap::KGeoMapWidget::setBackend: "setting backend marble"
digikam(2380)/KGEOMAP KGeoMap::KGeoMapWidget::setBackend: "setting backend marble"
digikam(2380)/digikam (core) Digikam::LensFunCameraSelector::populateLensCombo: variant:  QVariant(Digikam::LensFunIface::DevicePtr, )
digikam(2380)/digikam (core) Digikam::LensFunCameraSelector::populateLensCombo: dev:  Asahi Optical Co.,Ltd  ::  PENTAX Optio 430  ::  4.85
digikam(2380)/digikam (core) Digikam::LensFunIface::findCamera: Search for camera  "" - ""  ==> false
digikam(2380)/digikam (core) Digikam::LensFunIface::findLens: Search for lens  ""  ==> false
digikam(2380)/digikam (core) Digikam::CurvesWidget::updateData: updating data
digikam(2380)/digikam (core) Digikam::BlurFilter::cimgBlurImage: Radius:  0.1
digikam(2380)/digikam (core) Digikam::BlurFilter::cimgBlurImage: BlurFilter::Process Computation...
digikam(2380)/digikam (core) Digikam::BlurFilter::cimgBlurImage: Radius:  0.2
digikam(2380)/digikam (core) Digikam::BlurFilter::cimgBlurImage: BlurFilter::Process Computation...
digikam(2380)/digikam (core) Digikam::BlurFilter::cimgBlurImage: Radius:  0.5
digikam(2380)/digikam (core) Digikam::BlurFilter::cimgBlurImage: BlurFilter::Process Computation...
digikam(2380)/digikam (core) Digikam::BlurFilter::cimgBlurImage: Radius:  0.3
digikam(2380)/digikam (core) Digikam::BlurFilter::cimgBlurImage: BlurFilter::Process Computation...
digikam(2380)/digikam (core) Digikam::BlurFilter::cimgBlurImage: BlurFilter::Finalization...
digikam(2380)/digikam (core) Digikam::BlurFilter::cimgBlurImage: BlurFilter::Finalization...
digikam(2380)/digikam (core) Digikam::BlurFilter::cimgBlurImage: BlurFilter::Finalization...
digikam(2380)/digikam (core) Digikam::BlurFilter::cimgBlurImage: BlurFilter::Finalization...
digikam(2380)/digikam (core) Digikam::CurvesWidget::updateData: updating data
digikam(2380)/digikam (core) Digikam::AlbumFilterModel::setSearchTextSettings: new search text settings:  "" : hasResult =  false , validRows =  0
digikam(2380)/digikam (core) Digikam::WorkflowManager::load: Cannot open XML file to load Workflow
digikam(2380)/digikam (core) Digikam::ThemeManager::slotChangePalette: ""  ::  ""
digikam(2380)/digikam (core) Digikam::ThemeManager::slotChangePalette: ""  ::  ""
digikam(2380)/digikam (core) Digikam::KInotify::Private::open: Successfully opened connection to inotify: 66
digikam(2380)/KGEOMAP KGeoMap::BackendGoogleMaps::setMapType: "ROADMAP"
digikam(2380)/KGEOMAP KGeoMap::BackendGoogleMaps::setMapType: "ROADMAP"
digikam(2380)/KGEOMAP KGeoMap::KGeoMapWidget::setBackend: "setting backend marble"
digikam(2380)/KGEOMAP KGeoMap::BackendGoogleMaps::setMapType: "ROADMAP"
digikam(2380)/KGEOMAP KGeoMap::ItemMarkerTiler::slotSourceModelReset: ----
digikam(2380)/KGEOMAP KGeoMap::ItemMarkerTiler::slotSourceModelReset: ----
digikam(2380)/KGEOMAP KGeoMap::BackendGoogleMaps::setMapType: "ROADMAP"
digikam(2380)/KIPI (general) KIPIImageshackExportPlugin::Plugin_ImageshackExport::Plugin_ImageshackExport: Plugin_ImageshackExport plugin loaded
digikam(2380)/KIPI (general) KIPI::Plugin::Private::XMLParser::removeDisabledActions: Plugin action ' "batch_border_images" ' is disabled.
digikam(2380)/KIPI (general) KIPI::Plugin::Private::XMLParser::removeDisabledActions: Plugin action ' "batch_color_images" ' is disabled.
digikam(2380)/KIPI (general) KIPI::Plugin::Private::XMLParser::removeDisabledActions: Plugin action ' "batch_convert_images" ' is disabled.
digikam(2380)/KIPI (general) KIPI::Plugin::Private::XMLParser::removeDisabledActions: Plugin action ' "batch_filter_images" ' is disabled.
digikam(2380)/KIPI (general) KIPI::Plugin::Private::XMLParser::removeDisabledActions: Plugin action ' "batch_rename_images" ' is disabled.
digikam(2380)/KIPI (general) KIPI::Plugin::Private::XMLParser::removeDisabledActions: Plugin action ' "batch_recompress_images" ' is disabled.
digikam(2380)/KIPI (general) KIPIGalleryExportPlugin::Gallery::load: Reading data from kipirc file..
digikam(2380)/KIPI (general) KIPIJAlbumExportPlugin::JAlbum::load: Reading data from kipirc file..
digikam(2380)/KIPI (general) KIPIPhotoLayoutsEditor::Plugin_PhotoLayoutsEditor::Plugin_PhotoLayoutsEditor: Plugin_PhotoLayoutsEditor plugin loaded
digikam(2380)/KIPI (general) KIPI::Plugin::addAction: Action ' "batch_border_images" ' is disabled.
digikam(2380)/KIPI (general) KIPI::Plugin::addAction: Action ' "batch_color_images" ' is disabled.
digikam(2380)/KIPI (general) KIPI::Plugin::addAction: Action ' "batch_convert_images" ' is disabled.
digikam(2380)/KIPI (general) KIPI::Plugin::addAction: Action ' "batch_filter_images" ' is disabled.
digikam(2380)/KIPI (general) KIPI::Plugin::addAction: Action ' "batch_rename_images" ' is disabled.
digikam(2380)/KIPI (general) KIPI::Plugin::addAction: Action ' "batch_recompress_images" ' is disabled.
digikam(2380)/KIPI (general) KIPIPiwigoExportPlugin::Piwigo::load: Reading data from kipirc file..
digikam(2380)/KGEOMAP KGeoMap::ItemMarkerTiler::slotSourceModelReset: ----
digikam(2380)/KGEOMAP KGeoMap::ItemMarkerTiler::slotSourceModelReset: ----
digikam(2380)/digikam (editor plugins) DigikamColorImagePlugin::ImagePlugin_Color::ImagePlugin_Color: ImagePlugin_Color plugin loaded
digikam(2380)/digikam (core) Digikam::ImagePluginLoader::loadPluginsFromList: ImagePluginLoader: Loaded plugin  "ImagePlugin_Color"
digikam(2380)/digikam (editor plugins) DigikamDecorateImagePlugin::ImagePlugin_Decorate::ImagePlugin_Decorate: ImagePlugin_Decorate plugin loaded
digikam(2380)/digikam (core) Digikam::ImagePluginLoader::loadPluginsFromList: ImagePluginLoader: Loaded plugin  "ImagePlugin_Decorate"
digikam(2380)/digikam (editor plugins) DigikamEnhanceImagePlugin::ImagePlugin_Enhance::ImagePlugin_Enhance: ImagePlugin_Enhance plugin loaded
digikam(2380)/digikam (core) Digikam::ImagePluginLoader::loadPluginsFromList: ImagePluginLoader: Loaded plugin  "ImagePlugin_Enhance"
digikam(2380)/digikam (editor plugins) DigikamFxFiltersImagePlugin::ImagePlugin_FxFilters::ImagePlugin_FxFilters: ImagePlugin_FxFilters plugin loaded
digikam(2380)/digikam (core) Digikam::ImagePluginLoader::loadPluginsFromList: ImagePluginLoader: Loaded plugin  "ImagePlugin_FxFilters"
digikam(2380)/digikam (editor plugins) DigikamTransformImagePlugin::ImagePlugin_Transform::ImagePlugin_Transform: ImagePlugin_Transform plugin loaded
digikam(2380)/digikam (core) Digikam::ImagePluginLoader::loadPluginsFromList: ImagePluginLoader: Loaded plugin  "ImagePlugin_Transform"
digikam(2380)/KGEOMAP KGeoMap::KGeoMapWidget::setBackend: "setting backend marble"
digikam(2380)/KGEOMAP KGeoMap::BackendGoogleMaps::setMapType: "ROADMAP"
digikam(2380)/KGEOMAP KGeoMap::KGeoMapWidget::setBackend: "setting backend marble"
digikam(2380)/KGEOMAP KGeoMap::BackendGoogleMaps::setMapType: "ROADMAP"
digikam(2380)/KGEOMAP KGeoMap::KGeoMapWidget::setBackend: "setting backend marble"
digikam(2380)/KGEOMAP KGeoMap::BackendGoogleMaps::setMapType: "ROADMAP"
digikam(2380)/digikam (core) Digikam::NewItemsFinder::slotStart: scan mode: ScanDeferredFiles
digikam(2380)/digikam (core) Digikam::DImg::load: "/media/Diskphotos/mormal octobre 2013/MVI_2868.MOV"  : QIMAGE file identified
digikam(2380)/digikam (core) Digikam::QImageLoader::load: Can not load " "/media/Diskphotos/mormal octobre 2013/MVI_2868.MOV" " using DImg::QImageLoader!
digikam(2380)/digikam (core) Digikam::JPEGUtils::isJpegImage: mimetype =  ""  ext =  "MOV"
digikam(2380)/digikam (core) Digikam::ThumbnailCreator::createThumbnail: Cannot create thumbnail for  "/media/Diskphotos/mormal octobre 2013/MVI_2868.MOV" 
digikam(2380)/digikam (core) Digikam::ThumbnailCreator::load: Thumbnail is null for  "/media/Diskphotos/mormal octobre 2013/MVI_2868.MOV" 
digikam(2380)/KGEOMAP KGeoMap::ItemMarkerTiler::slotSourceModelReset: ----
digikam(2380)/KGEOMAP KGeoMap::ItemMarkerTiler::slotSourceModelReset: ----
digikam(2380)/KGEOMAP KGeoMap::ItemMarkerTiler::slotSourceModelReset: ----
digikam(2380)/KGEOMAP KGeoMap::ItemMarkerTiler::slotSourceModelReset: ----
digikam(2380)/digikam (core) Digikam::NewItemsFinder::slotTotalFilesToScan: total scan value :  28165
digikam(2380)/digikam (core) Digikam::FaceScanDialog::doLoadState: "Face Detection Dialog"
digikam(2380)/digikam (core) Digikam::DImg::load: "/media/Diskphotos/archive materiel peche/aero technium/eclaté_galet-shimano_aero_technium_et_ultegra_modifiée.JPG"  : JPEG file identified
digikam(2380)/digikam (core) Digikam::ImageScanner::prepareAddImage: Adding new item "/media/Diskphotos/archive materiel peche/aero technium/eclaté_galet-shimano_aero_technium_et_ultegra_modifiée.JPG"
digikam(2380)/digikam (core) Digikam::ImageScanner::scanTags: Pick Label found :  0
digikam(2380)/digikam (core) Digikam::ImageScanner::scanTags: Assigned Pick Label Tag  :  12
digikam(2380)/digikam (core) Digikam::ImageScanner::scanTags: Color Label found :  0
digikam(2380)/digikam (core) Digikam::ImageScanner::scanTags: Assigned Color Label Tag  :  2
digikam(2380)/digikam (core) Digikam::DMetadata::getImageHistory: Loading image history  ""
digikam(2380)/digikam (core) Digikam::ImageScanner::commit: Scanning took 54 ms
digikam(2380)/digikam (core) Digikam::ImageScanner::~ImageScanner: Finishing took 18 ms
digikam(2380)/digikam (core) Digikam::FaceScanDialog::doSaveState: "Face Detection Dialog"
digikam(2380)/digikam (core) Digikam::FaceDetector::slotStart: Total is 124165
digikam(2380)/digikam (core) Digikam::FaceDetector::slotContinueAlbumListing: false false
digikam(2380)/digikam (core) Digikam::FaceDetector::slotContinueAlbumListing: false true
digikam(2380)/digikam (core) Digikam::FacePipeline::Private::checkFinished: Check for finish:  0 packages, 34 infos to filter, hasFinished() false

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed
digikam(2380)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed

(digikam:2380): Gtk-CRITICAL **: IA__gtk_progress_configure: assertion `value >= min && value <= max' failed
digikam(2380)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.

---- START of face scan ---
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  800   533  searchIncrement  1.1  grouping  3  flags  1  min size  26   26
digikam(2380)/digikam (core) Digikam::DImg::load: "/media/Diskphotos/vendee 2010/bretignolle juin 2010 immersion/lune_bretignolles.jpg"  : JPEG file identified
digikam(2380)/digikam (core) Digikam::ImageScanner::prepareAddImage: Adding new item "/media/Diskphotos/vendee 2010/bretignolle juin 2010 immersion/lune_bretignolles.jpg"
digikam(2380)/digikam (core) Digikam::ImageScanner::scanTags: Pick Label found :  0
digikam(2380)/digikam (core) Digikam::ImageScanner::scanTags: Assigned Pick Label Tag  :  12
digikam(2380)/digikam (core) Digikam::ImageScanner::scanTags: Color Label found :  0
digikam(2380)/digikam (core) Digikam::ImageScanner::scanTags: Assigned Color Label Tag  :  2
digikam(2380)/digikam (core) Digikam::DMetadata::getImageHistory: Loading image history  "<?xml version="1.0"?>
<history version="1">
    <file uuid="3b6a2a35ffeb7720318bf7910e7f2265da8767f8944cc68cde090b7fe3abddd3" type="original">
        <fileParams fileName="lune_bretignolles.JPG" filePath="/media/Diskphotos/vendee 2010/bretignolle juin 2010 immersion/" fileHash="da8767f8944cc68cde090b7fe3abddd3" fileSize="2159724" creationDate="2010-06-21T23:55:11"/>
    </file>
</history>
"
digikam(2380)/digikam (core) Digikam::ImageScanner::commit: Scanning took 73 ms
digikam(2380)/digikam (core) Digikam::ImageScanner::~ImageScanner: Finishing took 41 ms
digikam(2380)/digikam (core) Digikam::CollectionScanner::completeHistoryScanning: items to tag (22180)
digikam(2380)/digikam (core) Digikam::ImageScanner::tagImageHistoryGraph: Graph with 2 vertices:
"{ Id: 22180 UUID: 3b6a2a... } -> { Id: 60303 UUID: ac2c01... }"
digikam(2380)/digikam (core) Digikam::ImageScanner::tagImageHistoryGraph: Image 60303 type QFlags(0x8)
digikam(2380)/digikam (core) Digikam::ImageScanner::tagImageHistoryGraph: Image 22180 type QFlags(0x1|0x8)
digikam(2380)/digikam (core) Digikam::CollectionScanner::completeScan: Complete scan took: 27180 msecs.
digikam(2380)/digikam (core) Digikam::DImg::load: "/media/Diskphotos/archive materiel peche/aero technium/eclaté_galet-shimano_aero_technium_et_ultegra_modifiee.jpg"  : JPEG file identified
digikam(2380)/digikam (core) Digikam::ImageScanner::prepareAddImage: Adding new item "/media/Diskphotos/archive materiel peche/aero technium/eclaté_galet-shimano_aero_technium_et_ultegra_modifiee.jpg"
digikam(2380)/digikam (core) Digikam::ImageScanner::commit: Scanning took 40 ms
digikam(2380)/digikam (core) Digikam::ImageScanner::~ImageScanner: Finishing took 293 ms
digikam(2380)/digikam (core) Digikam::DImg::load: "/media/Diskphotos/vendee 2010/bretignolle juin 2010 immersion/lune_bretignolles.JPG"  : JPEG file identified
digikam(2380)/digikam (core) Digikam::ImageScanner::prepareAddImage: Adding new item "/media/Diskphotos/vendee 2010/bretignolle juin 2010 immersion/lune_bretignolles.JPG"
digikam(2380)/digikam (core) Digikam::ImageScanner::scanFromIdenticalFile: Recognized "/media/Diskphotos/vendee 2010/bretignolle juin 2010 immersion/lune_bretignolles.JPG" as identical to item 22180
digikam(2380)/digikam (core) Digikam::ImageScanner::commit: Scanning took 25 ms
digikam(2380)/digikam (core) Digikam::ImageScanner::~ImageScanner: Finishing took 302 ms
digikam(2380)/digikam (core) Digikam::CollectionScanner::completeHistoryScanning: items to tag ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave (QRect(73,84 82x82) ,  QRect(66,132 93x93) )
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(73,84 82x82) using cascade 0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  162   162  searchIncrement  1.1  grouping  3  flags  0  min size  49   49
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave (QRect(45,40 80x80) )
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(73,84 82x82) using cascade 1
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  162   162  searchIncrement  1.1  grouping  3  flags  0  min size  49   49
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave (QRect(39,40 82x82) )
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(73,84 82x82) using cascade 2
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(73,84 82x82) using cascade 3
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  162   162  searchIncrement  1.1  grouping  3  flags  0  min size  49   49
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(73,84 82x82) using cascade 4
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  162   162  searchIncrement  1.1  grouping  3  flags  0  min size  49   49
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(73,84 82x82) using cascade 5
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0,0 0.6x0.6) QRect(73,84 82x82) QRect(73,84 49x49)
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  49   49  searchIncrement  1.1  grouping  2  flags  0  min size  0   0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(73,84 82x82) using cascade 6
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0.4,0 0.6x0.6) QRect(73,84 82x82) QRect(106,84 49x49)
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  49   49  searchIncrement  1.1  grouping  2  flags  0  min size  0   0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(73,84 82x82) using cascade 7
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0.2,0.25 0.6x0.6) QRect(73,84 82x82) QRect(89,105 49x49)
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  49   49  searchIncrement  1.1  grouping  2  flags  0  min size  0   0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(73,84 82x82) using cascade 8
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0.1,0.4 0.8x0.6) QRect(73,84 82x82) QRect(81,117 66x49)
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  66   49  searchIncrement  1.1  grouping  2  flags  0  min size  0   0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(66,132 93x93) using cascade 0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  173   173  searchIncrement  1.1  grouping  3  flags  0  min size  56   56
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave (QRect(36,35 97x97) )
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(66,132 93x93) using cascade 1
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  173   173  searchIncrement  1.1  grouping  3  flags  0  min size  56   56
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave (QRect(50,49 70x70) )
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(66,132 93x93) using cascade 2
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(66,132 93x93) using cascade 3
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  173   173  searchIncrement  1.1  grouping  3  flags  0  min size  56   56
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(66,132 93x93) using cascade 4
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  173   173  searchIncrement  1.1  grouping  3  flags  0  min size  56   56
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(66,132 93x93) using cascade 5
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0,0 0.6x0.6) QRect(66,132 93x93) QRect(66,132 56x56)
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  56   56  searchIncrement  1.1  grouping  2  flags  0  min size  0   0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(66,132 93x93) using cascade 6
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0.4,0 0.6x0.6) QRect(66,132 93x93) QRect(103,132 56x56)
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  56   56  searchIncrement  1.1  grouping  2  flags  0  min size  0   0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(66,132 93x93) using cascade 7
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0.2,0.25 0.6x0.6) QRect(66,132 93x93) QRect(85,155 56x56)
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  56   56  searchIncrement  1.1  grouping  2  flags  0  min size  0   0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(66,132 93x93) using cascade 8
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0.1,0.4 0.8x0.6) QRect(66,132 93x93) QRect(75,169 74x56)
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  74   56  searchIncrement  1.1  grouping  2  flags  0  min size  0   0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/digikam (core) Digikam::DetectionWorker::process: Found 2 faces in "IMG_2860.CR2" QSize(4272, 2848) QSize(4272, 2848)
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  800   533  searchIncrement  1.1  grouping  3  flags  1  min size  26   26
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave (QRect(16,257 53x53) )
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(16,257 53x53) using cascade 0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  133   133  searchIncrement  1.1  grouping  3  flags  0  min size  32   32
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(16,257 53x53) using cascade 1
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  133   133  searchIncrement  1.1  grouping  3  flags  0  min size  32   32
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(16,257 53x53) using cascade 2
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(16,257 53x53) using cascade 3
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  133   133  searchIncrement  1.1  grouping  3  flags  0  min size  32   32
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(16,257 53x53) using cascade 4
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  133   133  searchIncrement  1.1  grouping  3  flags  0  min size  32   32
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(16,257 53x53) using cascade 5
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0,0 0.6x0.6) QRect(16,257 53x53) QRect(16,257 32x32)
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  32   32  searchIncrement  1.1  grouping  2  flags  0  min size  0   0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(16,257 53x53) using cascade 6
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0.4,0 0.6x0.6) QRect(16,257 53x53) QRect(37,257 32x32)
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  32   32  searchIncrement  1.1  grouping  2  flags  0  min size  0   0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(16,257 53x53) using cascade 7
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0.2,0.25 0.6x0.6) QRect(16,257 53x53) QRect(27,270 32x32)
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  32   32  searchIncrement  1.1  grouping  2  flags  0  min size  0   0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: Verifying face QRect(16,257 53x53) using cascade 8
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::verifyFace: feature QRectF(0.1,0.4 0.8x0.6) QRect(16,257 53x53) QRect(21,278 42x32)
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size  42   32  searchIncrement  1.1  grouping  2  flags  0  min size  0   0
digikam(2380)/KFACE KFaceIface::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
digikam(2380)/digikam (core) Digikam::DetectionWorker::process: Found 0 faces in "IMG_2861.CR2" QSize(4272, 2848) QSize(4272, 2848)

-- big HALT (long time, fill the memory  and crash)

OpenCV Error: Insufficient memory (Failed to allocate 262144 bytes) in OutOfMemoryError, file /home/eric/opencv-2-4-5-SH/OpenCV/opencv-2.4.5/modules/core/src/alloc.cpp, line 52
digikam(2380)/KFACE: cv::Exception: /home/eric/opencv-2-4-5-SH/OpenCV/opencv-2.4.5/modules/core/src/alloc.cpp:52: error: (-4) Failed to allocate 262144 bytes in function OutOfMemoryError
 
Qt has caught an exception thrown from an event handler. Throwing
exceptions from an event handler is not supported in Qt. You must
reimplement QApplication::notify() and catch all exceptions there.

Qt Concurrent has caught an exception thrown from a worker thread.
This is not supported, exceptions thrown in worker threads must be
caught before control returns to Qt Concurrent.
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
KCrash: Application 'digikam' crashing...
KCrash: Attempting to start /usr/lib/kde4/libexec/drkonqi from kdeinit
sock_file=/home/eric/.kde/socket-eric-laptop/kdeinit4__0
QSocketNotifier: Invalid socket 13 and type 'Read', disabling...
QSocketNotifier: Invalid socket 16 and type 'Read', disabling...
QSocketNotifier: Invalid socket 22 and type 'Read', disabling...
digikam: Fatal IO error 9 (Mauvais descripteur de fichier) on X server :0.0.

[1]+  Stoppé                 digikam
eric@eric-laptop:~$ QSocketNotifier: Invalid socket 17 and type 'Read', disabling...

[1]+  Termine 253             digikam
Comment 34 e.longuemare 2013-10-13 08:49:22 UTC
(In reply to comment #32)
> As Marcel said previously, there is a no catched exception when digiKam crash
> 
> You backtrace is uncomplete because you don't get the exception properly.
> 
> To do it, follow this tutorial and report relevant and suitable backtrace :
> 
> http://techbase.kde.org/Development/Tutorials/Debugging/
> How_to_create_useful_crash_reports#Retrieving_a_backtrace_when_an_uncaught_ex
> ception_is_causing_a_crash
> 
> Gilles Caulier

Ok Gilles, 

I will try.

Thank you.
Comment 35 e.longuemare 2013-10-13 11:11:28 UTC
Created attachment 82823 [details]
Crash after full fill the memory when scanning a new folder with new pictures

Hello,

attach the gdb logs as explain here : http://techbase.kde.org/Development/Tutorials/Debugging/How_to_create_useful_crash_reports

set break point two "to __GI_raise" and breakpoint three to "__GI_abort".

What I was doing : scanning a new folder with new pictures with option : skip already scanned.

Thank you,

Eric
Comment 36 e.longuemare 2013-10-13 13:17:05 UTC
Created attachment 82826 [details]
Crash after full fill the memory - gdb log -

Crash after full fill the memory when scanning a  folder 

 Hello, 

Set break point two "to __GI_raise" and breakpoint three to "__GI_abort". 

What I was doing : scanning a  folder with  pictures(PS1) with option : Clear unconfirmed results and rescan. 

Thank you, 

Eric

PS 1: I rescan the folder used for testing in precedent post ( not the one used this morning)
PS 2 : this message from opencv doesn't appear in gdb :
OpenCV Error: Insufficient memory (Failed to allocate 262144 bytes) in OutOfMemoryError, file /home/eric/opencv-2-4-5-SH/OpenCV/opencv-2.4.5/modules/core/src/alloc.cpp, line 52 digikam(2380)/KFACE: cv::Exception: /home/eric/opencv-2-4-5-SH/OpenCV/opencv-2.4.5/modules/core/src/alloc.cpp:52: error: (-4) Failed to allocate 262144 bytes in function OutOfMemoryError
Comment 37 e.longuemare 2013-10-13 13:57:39 UTC
Created attachment 82827 [details]
gdb full log - full fill memory - cont to SIGALRM -

- same folder as first test with option Clear unconfirmed results and rescan. 
- breakpoint to "__GI_raise" and  to "__GI_abort". 
- "thread apply all bt" then "cont" in gdb until SIGALRM
Comment 38 e.longuemare 2013-10-18 10:18:58 UTC
Hello,

I have a new try with digikam 4.0 beta, compiled from git for eclipse and debug in eclipse+gdb.
I have made a test with two folders move from original source albums to the folder image then tag only one person in those folders (recognition.db about 86k). I use  2 sqlite db for digikam and digikam_thumb. I should do anything without problem. Memory usage remain stable.

Then I clone my working databases (mysql) into two new mysql databases : one for the thumb and the other for digikam, clone all the albums collection too. I configure digikam for using those clones dbs and the clone collection. Do a learning pass with facetagging tools wich was a succes (as in precedent tests - digikam 3.3, 3.4, 3.5) and do a new face scan  in the same folders that precedent test : memory usage grow, then digikam crash with segfault at :

digikam [5359] [cores: 0]	
	Thread [63] 12305 [core: 0] (Suspended : Signal : SIGSEGV:Segmentation fault)	
		Digikam::DImg::DImg() at dimg.cpp:165 0xe3e148	
		Digikam::PreviewLoadingTask::execute() at previewtask.cpp:288 0x104d2ac	
		Digikam::LoadSaveThread::run() at loadsavethread.cpp:136 0x1038e58	
		Digikam::DynamicThread::DynamicThreadPriv::run() at dynamicthread.cpp:186 0x106c11e	
		QThreadPoolThread::run() at qthreadpool.cpp:107 0x30f346b	
		QThreadPrivate::start() at qthread_unix.cpp:307 0x3100eb0	
		start_thread() at pthread_create.c:308 0x495ad4c	
		clone() at clone.S:130 0x3614dde	
	Thread [60] 12011 [core: 0] (Suspended : Container)	
		__kernel_vsyscall() at 0x132416	
		__GI___poll() at poll.c:87 0x3606690	
		g_poll() at 0x516aa3b	
		0x515d06e	
		g_main_context_iteration() at 0x515d1c1	
		QEventDispatcherGlib::processEvents() at qeventdispatcher_glib.cpp:424 0x3248d87	
		QEventLoop::processEvents() at qeventloop.cpp:149 0x32146ad	
		QEventLoop::exec() at qeventloop.cpp:204 0x3214949	
		Digikam::WorkerObjectRunnable::run() at threadmanager.cpp:196 0x1069a8e	
		QThreadPoolThread::run() at qthreadpool.cpp:107 0x30f346b	
		QThreadPrivate::start() at qthread_unix.cpp:307 0x3100eb0	
		start_thread() at pthread_create.c:308 0x495ad4c	
		clone() at clone.S:130 0x3614dde	
	Thread [59] 12010 [core: 0] (Suspended : Container)	
		__kernel_vsyscall() at 0x132416	
		__GI___poll() at poll.c:87 0x3606690	
		g_poll() at 0x516aa3b	
		0x515d06e	
		g_main_context_iteration() at 0x515d1c1	
		QEventDispatcherGlib::processEvents() at qeventdispatcher_glib.cpp:424 0x3248d87	
		QEventLoop::processEvents() at qeventloop.cpp:149 0x32146ad	
		QEventLoop::exec() at qeventloop.cpp:204 0x3214949	
		Digikam::WorkerObjectRunnable::run() at threadmanager.cpp:196 0x1069a8e	
		QThreadPoolThread::run() at qthreadpool.cpp:107 0x30f346b	
		QThreadPrivate::start() at qthread_unix.cpp:307 0x3100eb0	
		start_thread() at pthread_create.c:308 0x495ad4c	
		clone() at clone.S:130 0x3614dde	
	Thread [58] 12009 [core: 0] (Suspended : Container)	
		icvEvalHidHaarClassifier() at haar.cpp:812 0x3f5bd36	
		cvRunHaarClassifierCascadeSum() at haar.cpp:1 258 0x3f5d3ff	
		cvRunHaarClassifierCascade() at haar.cpp:1 274 0x3f5d60d	
		cv::HaarDetectObjects_ScaleCascade_Invoker::operator() at haar.cpp:1 463 0x3f5df5c	
		() at parallel.cpp:142 0x16dbbff	
		() at parallel.cpp:162 0x16dbcd2	
		tbb::interface6::internal::start_for<tbb::blocked_range<int>, {anonymous}::ProxyLoopBody, tbb::auto_partitioner>::run_body() at parallel_for.h:110 0x16dcc46	
		tbb::interface6::internal::partition_type_base<tbb::interface6::internal::auto_partition_type>::execute<tbb::interface6::internal::start_for<tbb::blocked_range<int>, {anonymous}::ProxyLoopBody, tbb::auto_partitioner>, tbb::blocked_range<int> >() at partitioner.h:265 0x16dc9b5	
		tbb::interface6::internal::start_for<tbb::blocked_range<int>, {anonymous}::ProxyLoopBody, tbb::auto_partitioner>::execute() at parallel_for.h:116 0x16dc500	
		tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::local_wait_for_all() at custom_scheduler.h:449 0x4e538c9	
		tbb::internal::generic_scheduler::local_spawn_root_and_wait() at scheduler.cpp:627 0x4e51ea0	
		tbb::internal::generic_scheduler::spawn_root_and_wait() at scheduler.h:535 0x4e52509	
		tbb::task::spawn_root_and_wait() at task.h:692 0x16db51b	
		tbb::interface6::internal::start_for<tbb::blocked_range<int>, {anonymous}::ProxyLoopBody, tbb::auto_partitioner>::run() at parallel_for.h:94 0x16dc239	
		tbb::parallel_for<tbb::blocked_range<int>, {anonymous}::ProxyLoopBody>() at parallel_for.h:162 0x16dc0cc	
		cv::parallel_for_() at parallel.cpp:238 0x16dbd78	
		cvHaarDetectObjectsForROC() at haar.cpp:1 720 0x3f5ff4c	
		cv::CascadeClassifier::detectMultiScale() at cascadedetect.cpp:1 109 0x3f77c32	
		cv::CascadeClassifier::detectMultiScale() at cascadedetect.cpp:1 204 0x3f7875e	
		KFaceIface::OpenCVFaceDetector::cascadeResult() at opencvfacedetector.cpp:419 0xa218ab	
		KFaceIface::OpenCVFaceDetector::detectFaces() at opencvfacedetector.cpp:699 0xa266ad	
		KFaceIface::FaceDetector::detectFaces() at facedetector.cpp:152 0xa1e712	
		Digikam::DetectionWorker::process() at facepipeline.cpp:482 0x834745b	
		Digikam::DetectionWorker::qt_static_metacall() at facepipeline_p.moc:318 0x8347cdc	
		QMetaCallEvent::placeMetaCall() at qobject.cpp:525 0x3228101	
		QObject::event() at qobject.cpp:1 195 0x323117b	
		Digikam::WorkerObject::event() at workerobject.cpp:160 0x106a762	
		notify_helper() at qapplication.cpp:4 556 0x242adf4	
		QApplicationPrivate::notify_helper() at qapplication.cpp:4 528 0x242adf4	
		QApplication::notify() at qapplication.cpp:4 285 0x243015d	
		KApplication::notify() at kapplication.cpp:311 0x2099ed1	
		QCoreApplication::notifyInternal() at qcoreapplication.cpp:915 0x3215e0e	
		sendEvent() at qcoreapplication.h:231 0x3219f68	
		QCoreApplicationPrivate::sendPostedEvents() at qcoreapplication.cpp:1 539 0x3219f68	
		QCoreApplication::sendPostedEvents() at qcoreapplication.cpp:1 432 0x321a29c	
		sendPostedEvents() at qcoreapplication.h:236 0x3248994	
		postEventSourceDispatch() at qeventdispatcher_glib.cpp:279 0x3248994	
		g_main_context_dispatch() at 0x515cd46	
		0x515d0e5	
		g_main_context_iteration() at 0x515d1c1	
		QEventDispatcherGlib::processEvents() at qeventdispatcher_glib.cpp:424 0x3248d87	
		QEventLoop::processEvents() at qeventloop.cpp:149 0x32146ad	
		QEventLoop::exec() at qeventloop.cpp:204 0x3214949	
		Digikam::WorkerObjectRunnable::run() at threadmanager.cpp:196 0x1069a8e	
		QThreadPoolThread::run() at qthreadpool.cpp:107 0x30f346b	
		QThreadPrivate::start() at qthread_unix.cpp:307 0x3100eb0	
		start_thread() at pthread_create.c:308 0x495ad4c	
		clone() at clone.S:130 0x3614dde	
	Thread [7] 5640 [core: 0] (Suspended : Container)	
	Thread [3] 5552 [core: 0] (Suspended : Container)	
	Thread [2] 5550 [core: 0] (Suspended : Container)	
	Thread [1] 5359 [core: 0] (Suspended : Container)

Please view attached variables

Thanks,

Eric
Comment 39 e.longuemare 2013-10-18 10:21:30 UTC
Created attachment 82913 [details]
Variables when SigFault at dimg.cpp ligne 165

Variables when digikam crash on dimg.cpp line 165 (scanning faces)
Comment 40 Olivier Becquaert 2013-11-03 19:40:26 UTC
Hello great Digikam team,

please fix this bug...this is a huge regression since last 2 versions : face recognition is not usable anymore :-(.

Regards,
Olivier
Comment 41 caulier.gilles 2013-11-03 21:34:52 UTC
The huge memory leak is already fixed. I miss libPGF memory corruption/leakto be fixed. I waiting libpgf feedback for that. 

Note: libpgf problems already exist in previous versions...

Gilles Caulier
Comment 42 Olivier Becquaert 2013-11-05 19:53:55 UTC
Thank you Gilles for your quick feedback. Hope you'll be able to publish a release soon :-).

Olivier
Comment 43 caulier.gilles 2013-11-06 06:40:02 UTC
*** Bug 327197 has been marked as a duplicate of this bug. ***
Comment 44 caulier.gilles 2014-01-06 13:08:52 UTC
*** Bug 325712 has been marked as a duplicate of this bug. ***
Comment 45 caulier.gilles 2014-01-06 13:09:02 UTC
*** Bug 329651 has been marked as a duplicate of this bug. ***
Comment 46 Olivier Becquaert 2014-01-06 21:35:46 UTC
(In reply to comment #41)
> The huge memory leak is already fixed. I miss libPGF memory
> corruption/leakto be fixed. I waiting libpgf feedback for that. 
> 
> Note: libpgf problems already exist in previous versions...
> 
> Gilles Caulier

Hi Gilles,

I wish you all the best for you & Digikam in 2014 !

is libpgf bug fixed ?

Regards
Olivier
Comment 47 caulier.gilles 2014-01-06 22:16:23 UTC
Thanks for your 2014 hopes. 

libpgf is fixed into digiKam core, but stand alone library is not yet released with fixes. This is why libpgf still in digiKam core for the moment...

Gilles Caulier
Comment 48 Olivier Becquaert 2014-01-08 19:44:20 UTC
Hi Gilles,

you mean it is fixed in digikam core in current version (3.5.0) ? Or in a version to be published ? because I have this bug in version 3.5.0 (on archlinux).

regards
Olivier
Comment 49 e.longuemare 2014-01-08 20:14:37 UTC
Hello,

You must build digikam 3.5 with internal libpgf (given in digikam 3.5.0 sources).

For this, you must remove the libpgf package from distribution then rebuild digikam. I will answer you more in detail tomorrow, but if i well remenber, i think remove lipgf package from dist is enougth.

So remove libpgf, then make an make install for digikam.

Regards,

Eric
Comment 50 caulier.gilles 2014-03-09 13:59:32 UTC
*** Bug 331912 has been marked as a duplicate of this bug. ***
Comment 51 kde-bugtracking 2014-03-20 07:51:56 UTC
This bug stil occurs in Digikam 4.11.3 within OpenSUSE 13.1, linux 3.11.10-7-desktop.
Comment 52 rob.dean 2014-03-24 11:40:21 UTC
I also get this bug in digiKam Version 4.0.0-beta3 (installed from the ppa at http://ppa.launchpad.net/msylwester/digikam-beta/ubuntu). I use this PPA because I don't believe that in this day and age I should have to compile the code myself. Why is there no "official" PPA for Digikam betas? Would this not help with testing? Is there plans to resolve this bug properly? It seems that face recognition is a major draw card for Digikam and the fact that it makes the program grind to a halt isn't a good look. Is there a workaround? With easily followed instructions?
Comment 53 Scott 2014-04-03 23:43:19 UTC
I am also still getting this in digikam 4.0.0.beta4 from msylwester's ppa.

is it at all possible to compile the internal libpgf and install this on the system, or stick it in a ppa somewhere to save having to wait for an update to a package that has not been updated upstream in 3 years?

I get error after error trying to compile digikam using the flaky instructions provided on the digikam website and just dont have the time to sort the issues out, though I do love digikam.

This is a shame that one of the major features that digikam offers is completely unusable...
Comment 54 Thomas Käfer 2014-04-19 09:30:50 UTC
I'm experiencing this bug in digikam 4.0.0.beta4 (compiled it myself)
After killing my "unresponsive" (all changes to face-tags are ignored and undone) digikam I've found that my recognition.db was 11GB in size. After deleting (renaming) it the tagging now works brilliantly :)
Comment 55 rob.dean 2014-04-19 12:17:14 UTC
(In reply to comment #54)
> I'm experiencing this bug in digikam 4.0.0.beta4 (compiled it myself)
> After killing my "unresponsive" (all changes to face-tags are ignored and
> undone) digikam I've found that my recognition.db was 11GB in size. After
> deleting (renaming) it the tagging now works brilliantly :)

Can anyone else confirm this works? And provide some clear "how to" steps if it does?
Comment 56 Ritesh Raj Sarraf 2014-04-30 22:05:05 UTC
(In reply to comment #41)
> The huge memory leak is already fixed. I miss libPGF memory
> corruption/leakto be fixed. I waiting libpgf feedback for that. 
> 
> Note: libpgf problems already exist in previous versions...
> 
> Gilles Caulier

@Gilles, I don't think the problem lies with libPGF.

First based on the advise, I recompiled libPGF as was recommended. That did not help.

Then, I recompiled Digikam (4.0 Beta 4) with internal libPGF support. It still loses memory at the same insane rate.


digiKam version 4.0.0-beta4
Exiv2 can write to Jp2: Yes
Exiv2 can write to Jpeg: Yes
Exiv2 can write to Pgf: Yes
Exiv2 can write to Png: Yes
Exiv2 can write to Tiff: Yes
Exiv2 supports XMP metadata: Yes
LibCImg: 130
LibEigen: 3.2.1
LibExiv2: 0.23
LibJPEG: 80
LibJasper: 1.900.1
LibKDE: 4.12.4
LibKExiv2: 2.3.1
LibKGeoMap: 2.0.0
LibKdcraw: 2.3.1
LibLCMS: 2060
LibLensFun: 0.2.8-0
LibPGF: 6.13.45 - internal library
LibPNG: 1.2.50
LibQt: 4.8.6
LibRaw: 0.15.3
LibTIFF: LIBTIFF, Version 4.0.3 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc.
Marble Widget: 0.16.5 (stable version)
Parallelized PGF codec: No
Parallelized demosaicing: No
RawSpeed codec support: No
Database backend: QSQLITE
Kipi-Plugins: 4.0.0-beta4
LibGphoto2: 2.5.4
LibKface: 3.0.0
LibKipi: 2.1.0
LibOpenCV: 2.4.8
Comment 57 Scott 2014-05-01 00:51:42 UTC
I can confirm that this bug has been fixed in libpgf upstream, and therefore Digikam.  Of course though, the libpgf package still hasn't made it to the Debian/Ubuntu repo's as yet...

The steps that I took to resolve this (on x64 Ubuntu 13.10, running digikam from the package maintained by msylwester's ppa), and I am trying to make this as accessible to the everyday user as possible: 

Firstly: Don't uninstall the official package if you installed digikam from a repository!! It is a dependency of digikam, so you want the system to think it is there still!  As I am still a reasonably new user to compiling from source, I know no better way than to fool the system into thinking it has one version, when it has a newer...

Get the package source from here:  http://sourceforge.net/projects/libpgf/files/libpgf/6.14.12-latest/libpgf-src-6.14.12.tar.gz/download

unpack it in a terminal using:   
tar -xzvf libpgf-src-6.14.12.tar.gz 

Get the required essentials for building packages if you havent already:    
sudo apt-get install build-essential checkinstall  

On top of build-essentials etc. you also need libtools and automake.   Get those too: 
sudo apt-get install libtools automake

Once you've got those, cd to the directory where libpgf extracted to, then run 
./autogen.sh

Once that has finished, you need to run 
./configure

For me, there seemed to be an error in the generated config file due to a malformed command.  I kept getting the error: ".in'ig.status: error: cannot find input file: `Makefile".  To resolve this, open the configure file using your favourite text editor, for me it's: 
gedit configure

Search for a line that has "cat >$CONFIG_STATUS <<_ASEOF || as_write_fail=1" (the easiest way is just to search for ">$C" and it should come up.  Add a space in between the > and $ so the line then reads: "cat > $CONFIG_STATUS <<_ASEOF || as_write_fail=1".  Save and close the file and rerun:
./configure

If all goes well, proceed as normal: 
make
sudo make install

There is one final step though.  This technique puts the libraries in the wrong position for digikam.  It puts them in the /usr/local/lib/, while digikam looks for them in /usr/lib/x86_64-linux-gnu/ but this is easily fixed by copying them to the correct location running:
sudo cp -f /usr/local/lib/libpgf.so.6.0.11 /usr/lib/x86_64-linux-gnu/libpgf.so.6 && sudo cp -f /usr/local/lib/libpgf.so.6.0.11 /usr/lib/x86_64-linux-gnu/libpgf.so.6.0.7 && sudo cp -f /usr/local/lib/libpgf.so.6.0.11 /usr/lib/x86_64-linux-gnu/libpgf.so.6.0.11

This forces any other programs that may be looking for libpgf.so.6.0.7 to use libpgf.so.6.0.11 also.  And for those others out there, yes I know I could have made symlinks, but every time I tried, the links were broken...  I have no idea what I was doing wrong, but copying worked, so I'm sticking with it...

Once I had done this, the first tag took forever still, but after the first went much, much quicker than previously.  Good work libpgf team!

Hope this helps a bit for some...  Now just to wait for the official repos to catch up. :)
Comment 58 Ritesh Raj Sarraf 2014-05-01 02:38:08 UTC
(In reply to comment #57)
> I can confirm that this bug has been fixed in libpgf upstream, and therefore
> Digikam.  Of course though, the libpgf package still hasn't made it to the
> Debian/Ubuntu repo's as yet...
> 
> The steps that I took to resolve this (on x64 Ubuntu 13.10, running digikam
> from the package maintained by msylwester's ppa), and I am trying to make
> this as accessible to the everyday user as possible: 
> 
> Firstly: Don't uninstall the official package if you installed digikam from
> a repository!! It is a dependency of digikam, so you want the system to
> think it is there still!  As I am still a reasonably new user to compiling
> from source, I know no better way than to fool the system into thinking it
> has one version, when it has a newer...
> 
> Get the package source from here: 
> http://sourceforge.net/projects/libpgf/files/libpgf/6.14.12-latest/libpgf-
> src-6.14.12.tar.gz/download
> 
> unpack it in a terminal using:   
> tar -xzvf libpgf-src-6.14.12.tar.gz 
> 
> Get the required essentials for building packages if you havent already:    
> sudo apt-get install build-essential checkinstall  
> 
> On top of build-essentials etc. you also need libtools and automake.   Get
> those too: 
> sudo apt-get install libtools automake
> 
> Once you've got those, cd to the directory where libpgf extracted to, then
> run 
> ./autogen.sh
> 
> Once that has finished, you need to run 
> ./configure
> 
> For me, there seemed to be an error in the generated config file due to a
> malformed command.  I kept getting the error: ".in'ig.status: error: cannot
> find input file: `Makefile".  To resolve this, open the configure file using
> your favourite text editor, for me it's: 
> gedit configure
> 

I saw this error too. From what it looks, the generated configure file as invalid characters in it. Given that this file is generated at runtime, it makes me curious on where lies the bug.

> Search for a line that has "cat >$CONFIG_STATUS <<_ASEOF || as_write_fail=1"
> (the easiest way is just to search for ">$C" and it should come up.  Add a
> space in between the > and $ so the line then reads: "cat > $CONFIG_STATUS
> <<_ASEOF || as_write_fail=1".  Save and close the file and rerun:
> ./configure
> 

This alone did not solve the problem. You still get errors like config.h.in not present etc... Open the configure script and look for many instances of the word "Makefile". It has some invalid characters in it. Remove them and then the configure scirpt runs fine..

But.....

> If all goes well, proceed as normal: 
> make
> sudo make install
> 
> There is one final step though.  This technique puts the libraries in the
> wrong position for digikam.  It puts them in the /usr/local/lib/, while
> digikam looks for them in /usr/lib/x86_64-linux-gnu/ but this is easily
> fixed by copying them to the correct location running:
> sudo cp -f /usr/local/lib/libpgf.so.6.0.11
> /usr/lib/x86_64-linux-gnu/libpgf.so.6 && sudo cp -f
> /usr/local/lib/libpgf.so.6.0.11 /usr/lib/x86_64-linux-gnu/libpgf.so.6.0.7 &&
> sudo cp -f /usr/local/lib/libpgf.so.6.0.11
> /usr/lib/x86_64-linux-gnu/libpgf.so.6.0.11
> 
> This forces any other programs that may be looking for libpgf.so.6.0.7 to
> use libpgf.so.6.0.11 also.  And for those others out there, yes I know I
> could have made symlinks, but every time I tried, the links were broken... 
> I have no idea what I was doing wrong, but copying worked, so I'm sticking
> with it...
> 
> Once I had done this, the first tag took forever still, but after the first
> went much, much quicker than previously.  Good work libpgf team!
> 
> Hope this helps a bit for some...  Now just to wait for the official repos
> to catch up. :)

But... The problem remains the same. Digikam still leaks memory. Can you please double check if it has sorted the problem for you ?
Comment 59 caulier.gilles 2014-05-01 08:04:07 UTC
The other part where memory issue can be relevant of this file is OpenCV shared library. Please check which version is installed on your system....

Gilles Caulier
Comment 60 Ritesh Raj Sarraf 2014-05-01 08:45:20 UTC
(In reply to comment #59)
> The other part where memory issue can be relevant of this file is OpenCV
> shared library. Please check which version is installed on your system....
> 
> Gilles Caulier

The version I have is: 2.4.8+dfsg1-2

Does it have known issues ?
Comment 61 e.longuemare 2014-05-01 08:56:51 UTC
Hello,

I don't know if openCV have an issue, but I was having a problem with it too.

Should you look here to check some openCV compilation feature and your hardware (for me disable SSE3 feature - Intel pentium M):

https://bugs.kde.org/show_bug.cgi?id=325309

Greatings,

Eric
Comment 62 Scott 2014-05-01 10:18:12 UTC
> But... The problem remains the same. Digikam still leaks memory. Can you
> please double check if it has sorted the problem for you ?

I can confirm that for me digikam uses alot of memory, but no longer leaks memory.  I have just tagged 200+ of my 31,000+ outstanding face tags (I have ALOT of portrait photography dating back to 2003) and the memory usage is consistantly between 780-850MB, with an average of 800MB.

Btw devs, should I post a link the the solution (for me) on the digikam website, as a comment to the beta4 release?  Or should I let this flow downstream first?
Comment 63 Ritesh Raj Sarraf 2014-05-01 11:36:58 UTC
17:00:58 rrs@zan:~$ dstat -am
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- ------memory-usage-----
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw | used  buff  cach  free
  2   1  97   0   0   0|2179k  103k|   0     0 |   0     0 | 559  2702 |2513M  131M 1038M 3895M
 11   2  87   0   0   0|  54M    0 |   0     0 |   0     0 |1111  1800 |2895M  131M 1091M 3459M
 11   1  87   0   0   0|  53M   88k|   0     0 |   0     0 |1057  1501 |3272M  131M 1145M 3029M
 11   2  87   0   0   0|  54M 1240k|   0     0 |   0     0 |1171  1436 |3659M  131M 1198M 2589M
 11   2  87   0   0   0|  46M    0 | 428B    0 |   0     0 | 906  1316 |3988M  131M 1244M 2215M
 12   1  87   0   0   0|  46M    0 |   0     0 |   0     0 | 948  1731 |4321M  131M 1290M 1835M
 11   1  87   0   0   0|  38M    0 |   0    65B|   0     0 | 885  1872 |4593M  131M 1328M 1525M
 12   1  87   0   0   0|  44M    0 |   0     0 |   0     0 | 929  1745 |4908M  131M 1372M 1166M
 11   2  87   0   0   0|  46M    0 |   0     0 |   0     0 | 932  1386 |5236M  131M 1418M  792M
 12   1  87   0   0   0|  47M   36k|   0     0 |   0     0 | 965  1349 |5575M  131M 1465M  406M
 11   3  86   1   0   0|  49M    0 |   0     0 |   0     0 |1682  1666 |5927M 89.3M 1397M  163M
 11   3  86   0   0   0|  48M   88k|  42B   60B|   0    88k|  12k 2628 |6266M 38.9M 1128M  144M
  5   9  76   9   0   0|  11M   35M|   0     0 |   0    80M|  25k 3218 |7319M  328k  157M  100M
  1   4  69  26   0   0|3872k   79M|   0     0 |   0    98M|  16k  938 |7332M  328k  142M  102M
  1   3  73  22   0   0|3196k   86M| 428B    0 |  48k   83M|  12k  967 |7330M  328k  145M  102M
  1   3  72  25   0   0| 976k   77M|   0    65B| 160k   84M|  13k  885 |7328M  328k  144M  104M
  1   2  64  33   0   0|9024k   36M|   0     0 | 816k   25M|8423  1742 |7328M  328k  147M  101M
  1   1  60  39   0   0|4856k 9916k|   0     0 |1080k   16M| 622  1476 |7327M  328k  148M  101M
  0   3  45  52   0   0|1024k   63M|   0     0 | 188k   63M|  12k  920 |7322M  328k  146M  108M
  1   2  74  23   0   0|1120k   56M|   0     0 | 152k   52M|7944   809 |7329M  328k  148M  100M
  0   3  63  33   0   0|1256k   74M|   0     0 |  88k   79M|  12k  881 |7330M  328k  146M  100M
  0   2  56  42   0   0| 404k   46M|   0     0 | 272k   57M|5198   995 |7324M  352k  148M  104M
  0   1  55  44   0   0| 912k   34M|   0     0 | 908k   21M|4964  1115 |7326M  352k  148M  103M
  0   1  50  48   0   0|2136k   13M|   0     0 | 680k   15M|5817   980 |7328M  352k  148M  101M
  0   0  62  37   0   0|5072k 2048k|   0     0 |1212k    0 | 605  1207 |7327M  352k  149M  101M
  0   1  64  35   0   0| 820k   30M|   0    65B| 808k   32M|4568  1209 |7320M  348k  148M  109M
  1   1  78  20   0   0|1792k   19M|   0     0 | 688k   20M| 941  1638 |7326M  352k  148M  102M
  0   2  54  44   0   0|1096k   32M|   0     0 | 496k   38M|4716  1035 |7324M  348k  149M  104M
  0   1  64  34   0   0|5708k   14M|   0     0 |1940k   10M|2101  1601 |7331M  492k  145M  100M
  0   1  47  51   0   0| 760k   37M|   0     0 | 608k   32M|1192   868 |7330M  492k  146M  101M
  1   1  61  37   0   0|1416k   13M|   0     0 |1408k   20M|1587  1108 |7324M  492k  145M  107M
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- ------memory-usage-----
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw | used  buff  cach  free
  0   0  53  46   0   0| 900k 2560k|   0     0 | 896k    0 | 865   999 |7325M  492k  145M  106M
  1   1  58  41   0   0|1620k 8212k|   0     0 |1152k 6148k|3741  1193 |7330M  492k  146M  101M
  0   2  67  32   0   0| 468k   58M|   0     0 | 296k   63M|1338   718 |7323M  492k  146M  108M
  0   1  63  36   0   0| 520k   24M|   0    65B| 524k   23M|1973   780 |7324M  500k  146M  107M
  0   1  63  36   0   0|1756k   25M|   0     0 |1372k   24M|7324   987 |7324M  516k  146M  106M
  1   0  66  34   0   0|3056k    0 |   0     0 |1200k    0 |1026  1494 |7327M  516k  148M  102M
  0   0  64  35   0   0|2104k    0 |   0     0 |1356k    0 | 643  1255 |7327M  516k  149M  101M
  0   1  71  28   0   0|2464k   21M|   0     0 | 872k   15M|4744  1087 |7326M  516k  150M  101M
  0   2  59  39   0   0|2260k   39M|   0     0 | 548k   36M|  10k  960 |7325M  504k  151M  101M
  0   1  64  35   0   0|2796k 3584k|   0     0 |1500k   12M| 499  1133 |7320M  504k  151M  105M
  1   0  63  36   0   0|3804k    0 |   0     0 |1340k    0 | 843  1453 |7322M  644k  153M  101M
  0   7  53  40   0   0|3520k    0 |4363B    0 |1392k    0 |1212  1148 |7322M  644k  154M  101M
  0   0  77  23   0   0|2148k 4132k|   0     0 |1528k    0 | 835   928 |7322M  548k  153M  101M
  0   0  75  25   0   0|2940k    0 |   0    65B|1520k    0 | 528   964 |7323M  496k  153M  100M
  0   0  69  31   0   0|3972k 4096B|   0     0 |1252k    0 | 565  1198 |7323M  496k  153M  101M
  0   0  60  39   0   0|3296k 2048k|   0     0 |1060k    0 | 602  1403 |7324M  488k  147M  105M
  0   0  74  25   0   0|3960k    0 |   0     0 |1200k    0 |1084  1312 |7325M  488k  150M  101M
  0   0  75  25   0   0|1644k 2060k|   0     0 | 952k    0 | 942  1055 |7325M  496k  150M  101M
  0   3  75  23   0   0|3216k 2048k|  42B   60B|1096k  104k|1210  1284 |5440M  376k  142M 1995M
  0   3  48  49   0   0|5372k    0 |   0     0 |1380k    0 |1311  1561 | 514M  644k  140M 6922M
  0   1  78  21   0   0|4812k    0 |   0     0 |1908k    0 |1319  2009 | 516M  652k  143M 6917M
  0   0  81  19   0   0|4372k    0 |   0     0 |1500k    0 |1163  1518 | 518M  652k  146M 6913M
  0   0  75  25   0   0|6104k  516k|   0    65B|1748k    0 |1324  1766 | 520M  928k  150M 6906M
  0   0  74  26   0   0|5268k    0 |   0   212B|1796k    0 |1161  1576 | 522M  928k  153M 6901M
  0   0  82  17   0   0|4004k    0 |   0     0 |1416k    0 |1069  1417 | 523M  928k  156M 6897M
  0   1  87  12   0   0|5808k   32k|   0     0 |1956k    0 |1211  1875 | 525M  936k  159M 6892M
  0   0  88  12   0   0|2220k   50M| 428B    0 | 384k    0 | 821   837 | 520M  936k  161M 6895M
  1   0  88  11   0   0|4304k 6252k|   0     0 |1056k    0 | 517  1615 | 519M  936k  164M 6892M
  1   0  88  10   0   0|3052k    0 |   0     0 |1108k    0 | 599  1660 | 520M  936k  166M 6890M^C



@Giles,

Do you have any idea why Digikam is reading disk so bad ? Right when the leak is triggered, you can see, it reads 500+ MiB of data and then similary writes it down again.
Comment 64 Mick Sulley 2014-05-23 16:22:30 UTC
I tried the fix in comments 57 & 58 and the problem still exists for me.
I am running DK Version 3.5.0 Using KDE Development Platform 4.13.0
on Linux Mint 17 64 bit
The problem occurs when manually adding a single face tag.
Please let me know if I can help by providing logs or other info

Thanks
Mick
Comment 65 caulier.gilles 2014-05-23 16:30:13 UTC
Hi,

We need a fresh feedback using 4.0.0 just released.

Also, to have a log using valgrind about memory leak can help. Use this command in CLI, and report console trace to investiguate :

"valgrind --tool=memcheck --leak-check=full --error-limit=no digikam"

Gilles Caulier
Comment 66 Ritesh Raj Sarraf 2014-05-24 08:46:56 UTC
Created attachment 86795 [details]
digikam 4.0 still leaking memory
Comment 67 Ritesh Raj Sarraf 2014-05-24 08:47:59 UTC
(In reply to comment #65)
> Hi,
> 
> We need a fresh feedback using 4.0.0 just released.
> 
> Also, to have a log using valgrind about memory leak can help. Use this
> command in CLI, and report console trace to investiguate :
> 
> "valgrind --tool=memcheck --leak-check=full --error-limit=no digikam"
> 
> Gilles Caulier

Comment #66 has what you asked. This now is the pristine copy of Digikam as uploaded in Debian, i.e. the libPGF in use is external.
Comment 68 Scott 2014-05-26 03:08:31 UTC
Sorry for the long delay,

I can confirm that his bug had regressed when I upgraded Ubuntu from 13.10 to 14.04 and from 4.0.0beta to stable.  But for me, after deleting recognition.db it fixed the issue again, so I am starting to think there is more than meets the eye.

The problem with libPGF is gone, but clearly one with libkface remains...

Scott.
Comment 69 kde-bugs 2014-05-27 13:47:20 UTC
I can confirm that nothing has changed for me with 4.0.0 - running face detection even on one single picture will make digikam fill 70GB memory and more (killed it after that). I'm using MariaDB as backend.
Comment 70 caulier.gilles 2014-05-27 13:48:53 UTC
What's about this memory leak, if SQlite DB is used instead ?

Gilles Caulier
Comment 71 kde-bugs 2014-05-27 13:53:31 UTC
(In reply to comment #70)
> What's about this memory leak, if SQlite DB is used instead ?

If I read it correctly, the valgrind log from comment #66 uses SQLite, so that doesn't seem to make a difference.
Comment 72 Alexander Meshcheryakov 2014-05-27 14:00:39 UTC
It seems that everyone ignored my comments where I have investigated the issue a bit

https://bugs.kde.org/show_bug.cgi?id=323888#c5
https://bugs.kde.org/show_bug.cgi?id=323888#c6

In a nutshell my findings show that in some cases recognition.db becomes two times larger, so if this is repeated several times recognition.db becomes unbearable for digikam.

I have described steps to reproduce, please try to reproduce according to my guide. It takes less than 3 minutes.
Comment 73 Ritesh Raj Sarraf 2014-05-27 14:04:02 UTC
(In reply to comment #68)
> Sorry for the long delay,
> 
> I can confirm that his bug had regressed when I upgraded Ubuntu from 13.10
> to 14.04 and from 4.0.0beta to stable.  But for me, after deleting
> recognition.db it fixed the issue again, so I am starting to think there is
> more than meets the eye.
> 
> The problem with libPGF is gone, but clearly one with libkface remains...


Thank you for sharing this informaion Scott. Based on your feedback I went and did the same. After removing the recognition.db file, it is working a charm. But I still don't know if we have covered all the cases where the bug was reproducible.
Comment 74 kde-bugs 2014-05-27 14:14:43 UTC
I'll check my MariaDB instance for the face recognition DB size. However, will deleting the DB mean losing the face tags?
Comment 75 Alexander Meshcheryakov 2014-05-27 14:18:42 UTC
(In reply to comment #74)
> I'll check my MariaDB instance for the face recognition DB size. However,
> will deleting the DB mean losing the face tags?

recognition.db contains just "fingerprints" for auto face detection. It can be re-created according to your face tags after deletion.
Comment 76 Scott 2014-05-27 14:31:54 UTC
(In reply to comment #73)
> (In reply to comment #68)
> > Sorry for the long delay,
> > 
> > I can confirm that his bug had regressed when I upgraded Ubuntu from 13.10
> > to 14.04 and from 4.0.0beta to stable.  But for me, after deleting
> > recognition.db it fixed the issue again, so I am starting to think there is
> > more than meets the eye.
> > 
> > The problem with libPGF is gone, but clearly one with libkface remains...
> 
> 
> Thank you for sharing this informaion Scott. Based on your feedback I went
> and did the same. After removing the recognition.db file, it is working a
> charm. But I still don't know if we have covered all the cases where the bug
> was reproducible.

I totally agree.  While in my specific case recompiling libPGF, putting it in the right places and removing recognition.db when the bug regressed works, it's clear that it did not for everyone.

Either steps were not followed, or there is something more.  But either way, it would be nice to know just what is corrupting recognition.db in the first place...

On trying to repeat the corruption (I think it may be related to the detection and recognition option), i am now getting a crash when detecting, but tagging is still working.

As for deleting face tags, as said before, that recognition.db is for "fingerprints" only.  The tags and the face boxes themselves seem to be kept in digikam4.db, if not in the metadata itself (that is of course if the write metadata to images option is on!).  For my piece of mind, and portability, I personally leave the metadata writing on...
Comment 77 caulier.gilles 2014-06-11 21:13:59 UTC
Git commit fa909aa96e7e41225551cd9e866618717f1ea546 by Gilles Caulier.
Committed on 11/06/2014 at 21:06.
Pushed by cgilles into branch 'master'.

use a better C++ exception wrapper to handle non OpenCV exceptions (as pure C++ one)
Related: bug 335624, bug 330342, bug 329873, bug 326742, bug 326586, bug 326585, bug 324774, bug 323361, bug 320812, bug 312440, bug 309027, bug 308645, bug 301611, bug 297558, bug 285517

M  +4    -0    libkface/facedetector.cpp
M  +17   -2    libkface/recognitiondatabase.cpp

http://commits.kde.org/libkface/fa909aa96e7e41225551cd9e866618717f1ea546
Comment 78 caulier.gilles 2014-06-11 21:30:49 UTC
With next digiKam 4.1.0, i fixed libkface to handle all C++ exception (and not only OpenCV exception).

So, at least, digiKam must crash lesser now. If you want to review this entry again, use current implementation from git/master, or wait next 4.1.0 release...

Thanks to update your feedback

Gilles Caulier
Comment 79 caulier.gilles 2014-06-18 14:26:18 UTC
Git commit 4662dd94102f8144bc65ce1cb66d6b6cb1d500fd by Gilles Caulier.
Committed on 18/06/2014 at 14:22.
Pushed by cgilles into branch 'master'.

Libkface now depand of last stable OpenCV library version 2.4.9
This prevent internal crash int Cv:Algorithm that we cannot handle in libkface as exception.
Now, Face Recognition do not crash but still report Exception at training operations, especially about wrong Cv:Matrix size :

digikam(8673)/digikam (core) Digikam::DImg::load:"/mnt/data2/photos/GILLES/NEW/Adrien/2010-04-13/20100413_009.jpg"  :
JPEG file identified
OpenCV Error: Assertion failed (0 <= _dims && _dims <= CV_MAX_DIM) in setSize, file /mnt/devel/opencv/modules/core/src/matrix.cpp, line 89
digikam(8673)/KFACE: cv::Exception training LBPH: /mnt/devel/opencv/modules/core/src/matrix.cpp:89: error: (-215) 0 <=_dims && _dims <= CV_MAX_DIM in function setSize

It still a problem somwhere, but it's better than previous state.
Related: bug 335624, bug 330342, bug 329873, bug 326742, bug 326586, bug 326585, bug 324774, bug 323361, bug 320812, bug 312440, bug 309027, bug 308645, bug 301611, bug 297558, bug 285517

M  +1    -1    CMakeLists.txt

http://commits.kde.org/libkface/4662dd94102f8144bc65ce1cb66d6b6cb1d500fd
Comment 80 caulier.gilles 2014-06-24 12:43:29 UTC
Git commit 1fff86f31e3bf47a2a2cfa6eaa98bb1bdf1a863b by Gilles Caulier.
Committed on 24/06/2014 at 12:32.
Pushed by cgilles into branch 'master'.

add more test before to commit/checkout compressed histogram data in Face database, to prevent crashes, especially if data are corrupted from database.
Related: bug 335624, bug 330342, bug 329873, bug 326742, bug 326586, bug 326585, bug 324774, bug 320812, bug 312440, bug 309027, bug 308645, bug 301611, bug 297558, bug 285517

M  +62   -28   libkface/database/trainingdb.cpp

http://commits.kde.org/libkface/1fff86f31e3bf47a2a2cfa6eaa98bb1bdf1a863b
Comment 81 caulier.gilles 2014-06-24 13:10:25 UTC
I have bad news for this entry.

My investigations result sound to conclude that there are huge memory liberation performed outside digiKam database interface.

I tested it with SQLite.

It's not memory leak. allocated memory is liberated at end of digiKam session.

But database transactions take memory again and again. There is no reason for that. This is true for main digiKam database, and of course with Face database which store image compressed histogram data processed by faces training process.

I'm sure to not have seen it in the pass. I suspect a regression in libsqlite or Qt4-sqlite plugin. 

To resume, i suspect a UPSTREAM bug.

Gilles Caulier
Comment 82 caulier.gilles 2014-06-24 13:14:20 UTC
Note : 

I suspect a similar problem with QSlite about main digiKAm database at scanning process, especially at first time run, when collection to parse is huge. I remember to see report about crash at start up when swap become active.

Here i cannot reproduce this typical crash, because my computer use 16 Gb of RAM. It's of course reproducible with a VM using limited RAM.

Gilles Caulier
Comment 83 kde-bugs 2014-06-24 13:15:09 UTC
I think Alexander has a point in #5 #6 and #72 - when the recognition.db is deleted, everything works fine. So the real issue is that it must be verified that it cannot grow as large as it had for those of us who had been affected by this bug.
Comment 84 caulier.gilles 2014-06-24 13:22:17 UTC
In a VM, i can reproduce the memory leak from SQlite. I take a similar fingerprint with valgrind here. Look log from attachment "digikam 4.0 still leaking memory" :

==26673== 305,920 bytes in 239 blocks are possibly lost in loss record 120,501 of 120,538
==26673==    at 0x4C274A0: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26673==    by 0x21076F46: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21050AE9: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21059217: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2105CEBA: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2105D125: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x210569B0: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21083B35: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21083F1C: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21086102: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21086179: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21086257: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673== 
==26673== 1,332,480 bytes in 1,041 blocks are possibly lost in loss record 120,527 of 120,538
==26673==    at 0x4C274A0: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26673==    by 0x21076F46: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21050AE9: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21059217: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2105CEBA: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2105D125: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x210569B0: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21083B35: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21083F1C: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21086102: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21086179: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2108B2AB: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673== 
==26673== 2,752,344 bytes in 43 blocks are possibly lost in loss record 120,533 of 120,538
==26673==    at 0x4C274A0: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26673==    by 0x21076F46: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21050AE9: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21059217: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21062FD1: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x210D7F9D: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x36ED985E: QSQLiteDriver::open(QString const&, QString const&, QString const&, QString const&, int, QString const&) (in /usr/lib/x86_64-linux-gnu/qt4/plugins/sqldrivers/libqsqlite.so)
==26673==    by 0x4E44D60: QSqlDatabase::open() (in /usr/lib/x86_64-linux-gnu/libQtSql.so.4.8.6)
==26673==    by 0x7516600: Digikam::DatabaseCoreBackendPrivate::open(QSqlDatabase&) (in /usr/lib/digikam/libdigikamcore.so.4.0.0)
==26673==    by 0x7516BCA: Digikam::DatabaseCoreBackendPrivate::databaseForThread() (in /usr/lib/digikam/libdigikamcore.so.4.0.0)
==26673==    by 0x7516E99: Digikam::DatabaseCoreBackend::getQuery() (in /usr/lib/digikam/libdigikamcore.so.4.0.0)
==26673==    by 0x7519391: Digikam::DatabaseCoreBackend::prepareQuery(QString const&) (in /usr/lib/digikam/libdigikamcore.so.4.0.0)
==26673== 
==26673== 4,497,920 bytes in 3,514 blocks are possibly lost in loss record 120,535 of 120,538
==26673==    at 0x4C274A0: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26673==    by 0x21076F46: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21050AE9: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21059217: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2105CEBA: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2105D125: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x210569B0: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21083B35: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2108AA23: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2108C55E: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x210B72E8: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x210B834E: sqlite3_step (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673== 

==26673== 305,920 bytes in 239 blocks are possibly lost in loss record 120,501 of 120,538
==26673==    at 0x4C274A0: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26673==    by 0x21076F46: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21050AE9: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21059217: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2105CEBA: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2105D125: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x210569B0: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21083B35: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21083F1C: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21086102: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21086179: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21086257: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673== 
==26673== 1,332,480 bytes in 1,041 blocks are possibly lost in loss record 120,527 of 120,538
==26673==    at 0x4C274A0: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26673==    by 0x21076F46: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21050AE9: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21059217: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2105CEBA: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2105D125: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x210569B0: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21083B35: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21083F1C: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21086102: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21086179: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2108B2AB: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673== 
==26673== 2,752,344 bytes in 43 blocks are possibly lost in loss record 120,533 of 120,538
==26673==    at 0x4C274A0: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26673==    by 0x21076F46: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21050AE9: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21059217: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21062FD1: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x210D7F9D: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x36ED985E: QSQLiteDriver::open(QString const&, QString const&, QString const&, QString const&, int, QString const&) (in /usr/lib/x86_64-linux-gnu/qt4/plugins/sqldrivers/libqsqlite.so)
==26673==    by 0x4E44D60: QSqlDatabase::open() (in /usr/lib/x86_64-linux-gnu/libQtSql.so.4.8.6)
==26673==    by 0x7516600: Digikam::DatabaseCoreBackendPrivate::open(QSqlDatabase&) (in /usr/lib/digikam/libdigikamcore.so.4.0.0)
==26673==    by 0x7516BCA: Digikam::DatabaseCoreBackendPrivate::databaseForThread() (in /usr/lib/digikam/libdigikamcore.so.4.0.0)
==26673==    by 0x7516E99: Digikam::DatabaseCoreBackend::getQuery() (in /usr/lib/digikam/libdigikamcore.so.4.0.0)
==26673==    by 0x7519391: Digikam::DatabaseCoreBackend::prepareQuery(QString const&) (in /usr/lib/digikam/libdigikamcore.so.4.0.0)
==26673== 
==26673== 4,497,920 bytes in 3,514 blocks are possibly lost in loss record 120,535 of 120,538
==26673==    at 0x4C274A0: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26673==    by 0x21076F46: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21050AE9: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21059217: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2105CEBA: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2105D125: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x210569B0: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x21083B35: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2108AA23: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x2108C55E: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x210B72E8: ??? (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673==    by 0x210B834E: sqlite3_step (in /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6)
==26673== 

Memory leak from SQlite is just enormous...

Gilles Caulier
Comment 85 caulier.gilles 2014-06-24 13:53:34 UTC
With next digiKam 4.1.0, a lots of improvements/fixes have be done around face management. Please give us a fresh feedback.

Note i recommend to delete face recognition database to prevent dysfunction due to possible wrong data store in this container. Look where file is located in my computer :

[gilles@localhost database]$ pwd
/home/gilles/.kde4/share/apps/libkface/database
[gilles@localhost database]$ ls -al
total 397028
drwx------ 2 gilles gilles      4096 juin  24 14:22 ./
drwx------ 3 gilles gilles      4096 juin  18 19:08 ../
-rw-r--r-- 1 gilles gilles 406543360 juin  24 14:22 recognition.db
[gilles@localhost database]$
Comment 86 caulier.gilles 2014-07-08 08:11:48 UTC
I just check again digiKam 4.2.0 (current git/master implementation), and i can confirm the huge memory leak done by Qt Sqlite plugin when face are detected from image and registered to digiKam Database.

Sound like there is a big problem in Qt Sqlite plugin. See end of valgrind backtrace below :

==7348== 34,560 bytes in 27 blocks are possibly lost in loss record 26,533 of 26,604
==7348==    at 0x4C266ED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==7348==    by 0x1F8569A6: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F832189: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83A317: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83CDCA: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83D021: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F8378E8: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F862255: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F86250C: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F864701: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F8649EE: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F8660E4: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348== 
==7348== 44,800 bytes in 35 blocks are possibly lost in loss record 26,540 of 26,604
==7348==    at 0x4C266ED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==7348==    by 0x1F8569A6: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F832189: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83A317: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83CDCA: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83D021: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F8378E8: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F862255: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F86250C: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F864701: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F864779: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F864867: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348== 
==7348== 64,008 bytes in 1 blocks are possibly lost in loss record 26,549 of 26,604
==7348==    at 0x4C266ED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==7348==    by 0x1F8569A6: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F832189: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83A317: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F840AE1: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F8B42A6: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x2D04581E: ??? (in /usr/lib64/qt4/plugins/sqldrivers/libqsqlite.so)
==7348==    by 0x4E42D60: QSqlDatabase::open() (in /usr/lib64/libQtSql.so.4.8.6)
==7348==    by 0x7C179D6: Digikam::DatabaseCoreBackendPrivate::open(QSqlDatabase&) (databasecorebackend.cpp:214)
==7348==    by 0x7C172C0: Digikam::DatabaseCoreBackendPrivate::databaseForThread() (databasecorebackend.cpp:120)
==7348==    by 0x7C1A7A3: Digikam::DatabaseCoreBackend::open(Digikam::DatabaseParameters const&) (databasecorebackend.cpp:769)
==7348==    by 0x81D2BD1: Digikam::DatabaseAccess::checkReadyForUse(Digikam::InitializationObserver*) (databaseaccess.cpp:286)
==7348== 
==7348== 602,880 bytes in 471 blocks are possibly lost in loss record 26,596 of 26,604
==7348==    at 0x4C266ED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==7348==    by 0x1F8569A6: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F832189: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83A317: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83CDCA: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83D021: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F8378E8: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F862255: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F86250C: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F864701: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F864779: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F866338: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348== 
==7348== 768,096 bytes in 12 blocks are possibly lost in loss record 26,600 of 26,604
==7348==    at 0x4C266ED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==7348==    by 0x1F8569A6: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F832189: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83A317: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F840AE1: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F8B42A6: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x2D04581E: ??? (in /usr/lib64/qt4/plugins/sqldrivers/libqsqlite.so)
==7348==    by 0x4E42D60: QSqlDatabase::open() (in /usr/lib64/libQtSql.so.4.8.6)
==7348==    by 0x7C179D6: Digikam::DatabaseCoreBackendPrivate::open(QSqlDatabase&) (databasecorebackend.cpp:214)
==7348==    by 0x7C172C0: Digikam::DatabaseCoreBackendPrivate::databaseForThread() (databasecorebackend.cpp:120)
==7348==    by 0x7C1D2DD: Digikam::DatabaseCoreBackend::getQuery() (databasecorebackend.cpp:1512)
==7348==    by 0x7C1CF12: Digikam::DatabaseCoreBackend::prepareQuery(QString const&) (databasecorebackend.cpp:1467)
==7348== 
==7348== 2,291,200 bytes in 1,790 blocks are possibly lost in loss record 26,603 of 26,604
==7348==    at 0x4C266ED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==7348==    by 0x1F8569A6: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F832189: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83A317: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83CDCA: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F83D021: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F8378E8: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F862255: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F865B53: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F866B26: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F895911: ??? (in /usr/lib64/libsqlite3.so.0.8.6)
==7348==    by 0x1F89655E: sqlite3_step (in /usr/lib64/libsqlite3.so.0.8.6)
==7348== 
==7348== LEAK SUMMARY:
==7348==    definitely lost: 246,004 bytes in 1,710 blocks
==7348==    indirectly lost: 584,784 bytes in 10,817 blocks
==7348==      possibly lost: 4,004,231 bytes in 4,642 blocks
==7348==    still reachable: 21,369,113 bytes in 64,209 blocks
==7348==         suppressed: 0 bytes in 0 blocks
==7348== Reachable blocks (those to which a pointer was found) are not shown.
==7348== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==7348== 
==7348== For counts of detected and suppressed errors, rerun with: -v
==7348== Use --track-origins=yes to see where uninitialised values come from
==7348== ERROR SUMMARY: 1474 errors from 1450 contexts (suppressed: 4 from 3)

Gilles Caulier
Comment 87 caulier.gilles 2014-07-08 08:20:42 UTC
Note : Linux Mageia 4 64 bits, Qt 4.8.2, libsqlite 3.8.0.2

I forward this file to qt components for future investigations from Qt developers.

Gilles Caulier
Comment 88 caulier.gilles 2014-07-08 08:28:24 UTC
Without to run whole digiKam with Face Management feature, you can reproduce memory leak with libkface "kfacegui" test program. When tool is started and closed imediatly, whitout too processs any image to detect or recognize faces, the face database is open by Qt sqlite plugin and valgrind report this trace about :

==15618== 476 (96 direct, 380 indirect) bytes in 1 blocks are definitely lost in loss record 635 of 697
==15618==    at 0x4C26BF5: operator new(unsigned long) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==15618==    by 0x6218FA2: ??? (in /usr/lib64/libQtCore.so.4.8.6)
==15618==    by 0x6213567: QFactoryLoader::updateDir(QString const&, QSettings&) (in /usr/lib64/libQtCore.so.4.8.6)
==15618==    by 0x62154C6: QFactoryLoader::update() (in /usr/lib64/libQtCore.so.4.8.6)
==15618==    by 0x62156E9: QFactoryLoader::QFactoryLoader(char const*, QString const&, Qt::CaseSensitivity) (in /usr/lib64/libQtCore.so.4.8.6)
==15618==    by 0x8A94844: ??? (in /usr/lib64/libQtSql.so.4.8.6)
==15618==    by 0x8A9551F: QSqlDatabase::drivers() (in /usr/lib64/libQtSql.so.4.8.6)
==15618==    by 0x4E812B8: KFaceIface::DatabaseAccess::checkReadyForUse(KFaceIface::DatabaseAccessData*, KFaceIface::InitializationObserver*) (databaseaccess.cpp:201)
==15618==    by 0x4E583C0: KFaceIface::RecognitionDatabase::Private::Private(QString const&) (recognitiondatabase.cpp:215)
==15618==    by 0x4E5814F: KFaceIface::RecognitionDatabaseStaticPriv::database(QString const&) (recognitiondatabase.cpp:192)
==15618==    by 0x4E588DE: KFaceIface::RecognitionDatabase::addDatabase(QString const&) (recognitiondatabase.cpp:276)
==15618==    by 0x40A5DE: MainWindow::MainWindow(QWidget*) (mainwindow.cpp:159)

Of course, it's not memory leak due to DB data registration as with digiKam face management, but it's also abnormal to leak memory just to open database file.

libkface implementation is available here : 

https://projects.kde.org/projects/extragear/libs/libkface/repository

Gilles Caulier
Comment 89 caulier.gilles 2014-07-10 10:11:26 UTC
I progressed well in this entry about investigations.

Here under Mageia4, i use Qt 4.8.2 and libsqlite 3.8.0.2

I written a small CLI program to check digiKam database init with valgrind. Installing qt and sqlite debug package, i can identify now where memory is leak.

In fact it's a know problem already reported in this entry :

https://bugs.kde.org/show_bug.cgi?id=329697

... where users complain that digiKam crash into libsqlite memory management functions.

So now with valgrind, we can see where memory is leak on my computer :
==32332== 7,680 bytes in 6 blocks are possibly lost in loss record 2,049 of 2,066
==32332==    at 0x4C266ED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332==    by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332==    by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332==    by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332==    by 0x1DDE5DCA: pcache1Alloc (sqlite3.c:36875)
==32332==    by 0x1DDE6021: pcache1Fetch (sqlite3.c:36959)
==32332==    by 0x1DDE08E8: sqlite3PcacheFetch (sqlite3.c:36286)
==32332==    by 0x1DE0B255: sqlite3PagerAcquire (sqlite3.c:43632)
==32332==    by 0x1DE0B50C: btreeGetPage (sqlite3.c:51175)
==32332==    by 0x1DE0D701: getAndInitPage (sqlite3.c:51230)
==32332==    by 0x1DE0D9EE: moveToRoot (sqlite3.c:53987)
==32332==    by 0x1DE0F0E4: sqlite3BtreeMovetoUnpacked (sqlite3.c:54204)
==32332== 
==32332== 9,000 bytes in 45 blocks are possibly lost in loss record 2,056 of 2,066
==32332==    at 0x4C266ED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332==    by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332==    by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332==    by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332==    by 0x1DDE3450: sqlite3DbMallocRaw (sqlite3.c:19406)
==32332==    by 0x1DDF06DB: exprDup (sqlite3.c:76262)
==32332==    by 0x1DE2EEA3: sqlite3Parser (sqlite3.c:76350)
==32332==    by 0x1DE34459: sqlite3RunParser (sqlite3.c:115140)
==32332==    by 0x1DE34A81: sqlite3Prepare (sqlite3.c:96189)
==32332==    by 0x1DE34D64: sqlite3LockAndPrepare (sqlite3.c:96280)
==32332==    by 0x1DE34DF4: sqlite3_prepare (sqlite3.c:96344)
==32332==    by 0x1DE34EAE: sqlite3InitCallback (sqlite3.c:95658)
==32332== 
==32332== 9,564 (1,920 direct, 7,644 indirect) bytes in 20 blocks are definitely lost in loss record 2,058 of 2,066
==32332==    at 0x4C26BF5: operator new(unsigned long) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332==    by 0x5C39FA2: ??? (in /usr/lib64/libQtCore.so.4.8.6)
==32332==    by 0x5C34567: QFactoryLoader::updateDir(QString const&, QSettings&) (in /usr/lib64/libQtCore.so.4.8.6)
==32332==    by 0x5C364C6: QFactoryLoader::update() (in /usr/lib64/libQtCore.so.4.8.6)
==32332==    by 0x5C366E9: QFactoryLoader::QFactoryLoader(char const*, QString const&, Qt::CaseSensitivity) (in /usr/lib64/libQtCore.so.4.8.6)
==32332==    by 0x6456754: ??? (in /usr/lib64/libQtGui.so.4.8.6)
==32332==    by 0x6456BBF: ??? (in /usr/lib64/libQtGui.so.4.8.6)
==32332==    by 0x6458891: ??? (in /usr/lib64/libQtGui.so.4.8.6)
==32332==    by 0x6459DE7: QImageReader::read(QImage*) (in /usr/lib64/libQtGui.so.4.8.6)
==32332==    by 0x6459FB3: QImageReader::read() (in /usr/lib64/libQtGui.so.4.8.6)
==32332==    by 0x644E5D8: operator>>(QDataStream&, QImage&) (in /usr/lib64/libQtGui.so.4.8.6)
==32332==    by 0x64676D8: operator>>(QDataStream&, QPixmap&) (in /usr/lib64/libQtGui.so.4.8.6)
==32332== 
==32332== 15,288 bytes in 39 blocks are possibly lost in loss record 2,059 of 2,066
==32332==    at 0x4C266ED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332==    by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332==    by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332==    by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332==    by 0x1DDE3450: sqlite3DbMallocRaw (sqlite3.c:19406)
==32332==    by 0x1DE31C06: sqlite3Parser (sqlite3.c:83487)
==32332==    by 0x1DE34459: sqlite3RunParser (sqlite3.c:115140)
==32332==    by 0x1DE34A81: sqlite3Prepare (sqlite3.c:96189)
==32332==    by 0x1DE34D64: sqlite3LockAndPrepare (sqlite3.c:96280)
==32332==    by 0x1DE34DF4: sqlite3_prepare (sqlite3.c:96344)
==32332==    by 0x1DE34EAE: sqlite3InitCallback (sqlite3.c:95658)
==32332==    by 0x1DE35258: sqlite3_exec (sqlite3.c:92433)
==32332== 
==32332== 38,400 bytes in 30 blocks are possibly lost in loss record 2,063 of 2,066
==32332==    at 0x4C266ED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332==    by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332==    by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332==    by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332==    by 0x1DDE5DCA: pcache1Alloc (sqlite3.c:36875)
==32332==    by 0x1DDE6021: pcache1Fetch (sqlite3.c:36959)
==32332==    by 0x1DDE08E8: sqlite3PcacheFetch (sqlite3.c:36286)
==32332==    by 0x1DE0B255: sqlite3PagerAcquire (sqlite3.c:43632)
==32332==    by 0x1DE0B50C: btreeGetPage (sqlite3.c:51175)
==32332==    by 0x1DE0D701: getAndInitPage (sqlite3.c:51230)
==32332==    by 0x1DE0D779: moveToChild (sqlite3.c:53869)
==32332==    by 0x1DE0D867: moveToLeftmost (sqlite3.c:54049)
==32332== 
==32332== 64,008 bytes in 1 blocks are possibly lost in loss record 2,064 of 2,066
==32332==    at 0x4C266ED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332==    by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332==    by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332==    by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332==    by 0x1DDE9AE1: setupLookaside.part.235 (sqlite3.c:116154)
==32332==    by 0x1DE5D2A6: openDatabase (sqlite3.c:93720)
==32332==    by 0x2BA5A81E: ??? (in /usr/lib64/qt4/plugins/sqldrivers/libqsqlite.so)
==32332==    by 0x6E8CD60: QSqlDatabase::open() (in /usr/lib64/libQtSql.so.4.8.6)
==32332==    by 0x5173AA8: Digikam::DatabaseCoreBackendPrivate::open(QSqlDatabase&) (databasecorebackend.cpp:214)
==32332==    by 0x51733D0: Digikam::DatabaseCoreBackendPrivate::databaseForThread() (databasecorebackend.cpp:120)
==32332==    by 0x5176861: Digikam::DatabaseCoreBackend::open(Digikam::DatabaseParameters const&) (databasecorebackend.cpp:769)
==32332==    by 0x572FBD1: Digikam::DatabaseAccess::checkReadyForUse(Digikam::InitializationObserver*) (databaseaccess.cpp:286)
==32332== 
==32332== 64,008 bytes in 1 blocks are possibly lost in loss record 2,065 of 2,066
==32332==    at 0x4C266ED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332==    by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332==    by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332==    by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332==    by 0x1DDE9AE1: setupLookaside.part.235 (sqlite3.c:116154)
==32332==    by 0x1DE5D2A6: openDatabase (sqlite3.c:93720)
==32332==    by 0x2BA5A81E: ??? (in /usr/lib64/qt4/plugins/sqldrivers/libqsqlite.so)
==32332==    by 0x6E8CD60: QSqlDatabase::open() (in /usr/lib64/libQtSql.so.4.8.6)
==32332==    by 0x5173AA8: Digikam::DatabaseCoreBackendPrivate::open(QSqlDatabase&) (databasecorebackend.cpp:214)
==32332==    by 0x51733D0: Digikam::DatabaseCoreBackendPrivate::databaseForThread() (databasecorebackend.cpp:120)
==32332==    by 0x517939B: Digikam::DatabaseCoreBackend::getQuery() (databasecorebackend.cpp:1512)
==32332==    by 0x5178FD0: Digikam::DatabaseCoreBackend::prepareQuery(QString const&) (databasecorebackend.cpp:1467)
==32332== 
==32332== 64,008 bytes in 1 blocks are possibly lost in loss record 2,066 of 2,066
==32332==    at 0x4C266ED: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==32332==    by 0x1DDFF9A6: sqlite3MemMalloc (sqlite3.c:15739)
==32332==    by 0x1DDDB189: mallocWithAlarm (sqlite3.c:19037)
==32332==    by 0x1DDE3317: sqlite3Malloc (sqlite3.c:19070)
==32332==    by 0x1DDE9AE1: setupLookaside.part.235 (sqlite3.c:116154)
==32332==    by 0x1DE5D2A6: openDatabase (sqlite3.c:93720)
==32332==    by 0x2BA5A81E: ??? (in /usr/lib64/qt4/plugins/sqldrivers/libqsqlite.so)
==32332==    by 0x6E8CD60: QSqlDatabase::open() (in /usr/lib64/libQtSql.so.4.8.6)
==32332==    by 0x5173AA8: Digikam::DatabaseCoreBackendPrivate::open(QSqlDatabase&) (databasecorebackend.cpp:214)
==32332==    by 0x51733D0: Digikam::DatabaseCoreBackendPrivate::databaseForThread() (databasecorebackend.cpp:120)
==32332==    by 0x5176861: Digikam::DatabaseCoreBackend::open(Digikam::DatabaseParameters const&) (databasecorebackend.cpp:769)
==32332==    by 0x5181716: Digikam::ThumbnailDatabaseAccess::checkReadyForUse(Digikam::InitializationObserver*) (thumbnaildatabaseaccess.cpp:216)
==32332== 
==32332== LEAK SUMMARY:
==32332==    definitely lost: 8,000 bytes in 690 blocks
==32332==    indirectly lost: 9,358 bytes in 81 blocks
==32332==      possibly lost: 391,691 bytes in 1,652 blocks
==32332==    still reachable: 336,972 bytes in 4,708 blocks
==32332==         suppressed: 0 bytes in 0 blocks
==32332== Reachable blocks (those to which a pointer was found) are not shown.
==32332== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==32332== 
==32332== For counts of detected and suppressed errors, rerun with: -v
==32332== ERROR SUMMARY: 957 errors from 957 contexts (suppressed: 2 from 2)
[gilles@localhost tests]$ 

So i suspect this entry to be fully relevant of Sqlite implementation as it's have already reported as UPSTREAM #329697

Note the difference : #329697 is about libsqlite:: sqlite3MemCompare() when this file is about libsqlite::sqlite3MemMalloc().

#329697 have been fixed with a patch release of libsqlite has it's explained in this Ubuntu report :

https://bugs.launchpad.net/ubuntu/+source/sqlite3/+bug/1317449

- Packagers and Users must update libsqlite to last stable 3.8.5 (http://www.sqlite.org/news.html) 
- Users must try to run FaceManagement again and look if memory leak still present or not.
- If yes, this file must be reported as UPSTREAM to SQLite bug tracker.

Giles Caulier
Comment 90 Mathieu Clabaut 2014-08-01 17:08:24 UTC
For information, the problem still present with digikam 4.1.0 and sqlite 3.8.5.
Comment 91 Nico Kruber 2014-08-10 23:30:00 UTC
if it really is fixed upstream, can you link the commit with the patch that fixes this issue if there is one?
Comment 92 caulier.gilles 2014-08-11 08:58:01 UTC
Nico,

I never said that problem is already fixed as UPSTREAM fixes

I closed this file as UPSTREAM because i suspect that problem is located in libsqlite.

This problem must be reported to SQlite bugzilla to be analysed by Sqlite team.

It can be a problem also in Qt sqlite plugin, but i'm not sure... 

Gilles Caulier
Comment 93 Nico Kruber 2014-08-11 10:58:17 UTC
oh, ok - I just wondered about the status change without any further information.

Actually, I resolved the problem by deleting my ~700MB face recognition DB for now. I kept the file in case it helps for debugging. Since I'm still using the same SQLite version, I suspect either the huge file size (and in turn any indexes etc.) or something went wrong during an update (?)
Comment 94 caulier.gilles 2014-08-11 11:30:59 UTC
I can details more some technicals stuff here :

- Face detection store region of image with face in main digiKam database. This point mangle memory.

- Face recognition is another algorithm which store face histograms in a dedicated database (the famous file that you have deleted). Recognition with previous version crash digiKam, but do not mangle memory. This is due to wrong storage of histogram data in database blob, and some incompatibility with previous OpenCV version which compute histogram. Removing this database will prevent crash. With 4.2.0, histograms generated will be done in a more secure way. I also add more protection in code to prevent wrong context. 

So, this entry is about face detection only.

Gilles Caulier
Comment 95 Nico Kruber 2014-08-11 14:53:38 UTC
From the bug title and description in the first entry, I guess this bug is actually about recognition.db and face recognition.

IIRC I had the crash problem you mentioned with the upgrade to 4.1.0 and re-created the DB with the appropriate GUI-Entry at that time. Now with 4.2.0, face detection works (al always) but whenever I gave an unknown face a name, digikam kept growing in memory size until it was killed. Re-creating the DB from the GUI did result in the same error - but deleting the DB file manually worked.
Comment 96 caulier.gilles 2014-08-13 21:56:20 UTC
*** Bug 338249 has been marked as a duplicate of this bug. ***
Comment 97 caulier.gilles 2014-08-27 13:51:23 UTC
*** Bug 335013 has been marked as a duplicate of this bug. ***
Comment 98 fassadourian 2014-11-30 16:59:48 UTC
*** This bug has been confirmed by popular vote. ***
Comment 99 caulier.gilles 2014-12-15 12:53:26 UTC
Git commit f6bcec9d07f0f6530835498370a6042f9b1e1675 by Marcel Wiesweg.
Committed on 14/11/2014 at 09:56.
Pushed by mwiesweg into branch 'master'.

Some fixes in face progress calculation

M  +1    -0    utilities/facemanagement/facepipeline.cpp
M  +2    -0    utilities/facemanagement/facepipeline.h
M  +31   -14   utilities/maintenance/facedetector.cpp

http://commits.kde.org/digikam/f6bcec9d07f0f6530835498370a6042f9b1e1675

diff --git a/utilities/facemanagement/facepipeline.cpp b/utilities/facemanagement/facepipeline.cpp
index a655385..d512a92 100644
--- a/utilities/facemanagement/facepipeline.cpp
+++ b/utilities/facemanagement/facepipeline.cpp
@@ -1171,6 +1171,7 @@ void FacePipeline::Private::send(FacePipelineExtendedPackage::Ptr package)
 {
     start();
     ++totalPackagesAdded;
+    emit(q->processing(*package));

     if (senderFlowControl(package))
     {
diff --git a/utilities/facemanagement/facepipeline.h b/utilities/facemanagement/facepipeline.h
index 4b381ca..45e733a 100644
--- a/utilities/facemanagement/facepipeline.h
+++ b/utilities/facemanagement/facepipeline.h
@@ -292,6 +292,8 @@ Q_SIGNALS:
     /// Emitted when processing has started
     void started(const QString& message);

+    /// Emitted when one package begins processing
+    void processing(const FacePipelinePackage& package);
     /// Emitted when one package has finished processing
     void processed(const FacePipelinePackage& package);
     void progressValueChanged(float progress);
diff --git a/utilities/maintenance/facedetector.cpp b/utilities/maintenance/facedetector.cpp
index 7bd8c0e..222652b 100644
--- a/utilities/maintenance/facedetector.cpp
+++ b/utilities/maintenance/facedetector.cpp
@@ -96,18 +96,27 @@ class FaceDetector::Private
 public:

     Private()
+        : benchmark(false),
+          total(0),
+          progressValue(0),
+          currentProgressChunk(0),
+          currentScheduled(0),
+          currentFinished(0)
     {
-        benchmark  = false;
-        total      = 0;
     }

-    bool               benchmark;
+    bool                benchmark;

-    int                total;
+    int                 total;

-    AlbumPointerList<> albumTodoList;
-    ImageInfoJob       albumListing;
-    FacePipeline       pipeline;
+    AlbumPointerList<>  albumTodoList;
+    ImageInfoJob        albumListing;
+    FacePipeline        pipeline;
+    QMap<Album*,double> relativeProgressValue;
+    double              progressValue;
+    double              currentProgressChunk;
+    int                 currentScheduled;
+    int                 currentFinished;
 };

 FaceDetector::FaceDetector(const FaceScanSettings& settings, ProgressItem* const parent)
@@ -275,25 +284,33 @@ void FaceDetector::slotStart()
         QApplication::restoreOverrideCursor();
     }

-    d->total = 0;
-
+    // first, we use the relativeProgressValue map to store absolute counts
     foreach(Album* const album, d->albumTodoList)
     {
         if (album->type() == Album::PHYSICAL)
         {
-            d->total += palbumCounts.value(album->id());
+            d->relativeProgressValue[album] = palbumCounts.value(album->id());
         }
         else
             // this is possibly broken of course because we do not know if images have multiple tags,
             // but there's no better solution without expensive operation
         {
-            d->total += talbumCounts.value(album->id());
+            d->relativeProgressValue[album] = talbumCounts.value(album->id());
         }
     }
-
-    kDebug() << "Total is" << d->total;
-
+    // second, calculate (approximate) overall sum
+    d->total = 0;
+    foreach (double count, d->relativeProgressValue)
+    {
+        d->total += (int)count;
+    }
     d->total = qMax(1, d->total);
+    kDebug() << "Total is" << d->total;
+    // third, break absolute to relative values
+    for (QMap<Album*,double>::iterator it = d->relativeProgressValue.begin(); it != d->relativeProgressValue.end(); ++it)
+    {
+        it.value() /= double(d->total);
+    }

     setUsesBusyIndicator(false);
     setTotalItems(d->total);
Comment 100 caulier.gilles 2014-12-15 12:54:19 UTC
Git commit 59f0bdb6b2709171354418ed297d4aa1d667ebad by Marcel Wiesweg.
Committed on 15/11/2014 at 13:47.
Pushed by mwiesweg into branch 'master'.

Add database cleanup calls to have a clean shutdown of SQLite data at application termination

M  +3    -0    app/main/main.cpp
M  +17   -3    tests/testdatabase.cpp

http://commits.kde.org/digikam/59f0bdb6b2709171354418ed297d4aa1d667ebad

diff --git a/app/main/main.cpp b/app/main/main.cpp
index 3564d23..85fcbc7 100644
--- a/app/main/main.cpp
+++ b/app/main/main.cpp
@@ -62,6 +62,7 @@
 #include "databaseparameters.h"
 #include "digikamapp.h"
 #include "scancontroller.h"
+#include "thumbnaildatabaseaccess.h"
 #include "version.h"

 using namespace Digikam;
@@ -235,6 +236,8 @@ int main(int argc, char* argv[])

     int ret = app.exec();

+    DatabaseAccess::cleanUpDatabase();
+    ThumbnailDatabaseAccess::cleanUpDatabase();
     KExiv2Iface::KExiv2::cleanupExiv2();

     return ret;
diff --git a/tests/testdatabase.cpp b/tests/testdatabase.cpp
index 10c14ad..0930029 100644
--- a/tests/testdatabase.cpp
+++ b/tests/testdatabase.cpp
@@ -30,6 +30,7 @@
 #include <QSqlDatabase>
 #include <QDBusConnection>
 #include <QString>
+#include <QTimer>

 // KDE includes

@@ -47,6 +48,7 @@
 #include "databaseparameters.h"
 #include "scancontroller.h"
 #include "setup.h"
+#include "thumbnaildatabaseaccess.h"
 #include "version.h"

 namespace Digikam
@@ -76,16 +78,28 @@ int main(int argc, char** argv)
     KCmdLineArgs::init(argc, argv, &aboutData);
     KApplication app;

-    DatabaseParameters params = DatabaseParameters::parametersFromConfig(KGlobal::config());
+    DatabaseParameters params;
+    params.databaseType = DatabaseParameters::SQLiteDatabaseType();
+    params.setDatabasePath(QDir::currentPath() + "/digikam-test.db");
+    params.setThumbsDatabasePath(QDir::currentPath() + "/digikam-thumbs-test.db");
+
     params.legacyAndDefaultChecks();

     QDBusConnection::sessionBus().registerService("org.kde.digikam.startup-" +
                      QString::number(QCoreApplication::instance()->applicationPid()));

     // initialize database
-    bool b = AlbumManager::instance()->setDatabase(params, false);
+    bool b = AlbumManager::instance()->setDatabase(params, false, "/media/fotos/Digikam Sample/");

     kDebug() << "Database initialization done: " << b;
-
+
+    QTimer::singleShot(500, &app, SLOT(quit()));
+    app.exec();
+
+    ScanController::instance()->shutDown();
+
+    DatabaseAccess::cleanUpDatabase();
+    ThumbnailDatabaseAccess::cleanUpDatabase();
+
     return 0;
 }
Comment 101 caulier.gilles 2014-12-15 12:55:03 UTC
Git commit b9f8dbfe470609ef31c5442cb2a4c97e02344233 by Marcel Wiesweg.
Committed on 15/11/2014 at 13:45.
Pushed by mwiesweg into branch 'master'.

Rewrite per-thread database connection cleanup
Use QThreadStorage of a per-thread DatabaseThreadData object which is destroyed when the thread finishes.

M  +103  -113  libs/database/core/databasecorebackend.cpp
M  +0    -5    libs/database/core/databasecorebackend.h
M  +22   -12   libs/database/core/databasecorebackend_p.h

http://commits.kde.org/digikam/b9f8dbfe470609ef31c5442cb2a4c97e02344233

diff --git a/libs/database/core/databasecorebackend.cpp b/libs/database/core/databasecorebackend.cpp
index c553bb1..76a958a 100644
--- a/libs/database/core/databasecorebackend.cpp
+++ b/libs/database/core/databasecorebackend.cpp
@@ -79,22 +79,64 @@ public:
     }
 };

+DatabaseThreadData::DatabaseThreadData()
+    : valid(0),
+      transactionCount(0)
+{
+}
+
+DatabaseThreadData::~DatabaseThreadData()
+{
+    if (transactionCount)
+    {
+        kDebug() << "WARNING !!! Transaction count is" << transactionCount << "when destroying database!!!";
+    }
+    closeDatabase();
+}
+
+void DatabaseThreadData::closeDatabase()
+{
+    QString connectionToRemove;
+    if (database.isOpen())
+    {
+        connectionToRemove = database.connectionName();
+    }
+
+    // Destroy object
+    database = QSqlDatabase();
+
+    valid            = 0;
+    transactionCount = 0;
+    lastError        = QSqlError();
+
+    // Remove connection
+    if (!connectionToRemove.isNull())
+    {
+        QSqlDatabase::removeDatabase(connectionToRemove);
+    }
+}
+
 DatabaseCoreBackendPrivate::DatabaseCoreBackendPrivate(DatabaseCoreBackend* const backend)
-    : q(backend)
+    : currentValidity(0),
+      isInTransaction(false),
+      status(DatabaseCoreBackend::Unavailable),
+      lock(0),
+      operationStatus(DatabaseCoreBackend::ExecuteNormal),
+      errorLockOperationStatus(DatabaseCoreBackend::ExecuteNormal),
+      errorHandler(0),
+      q(backend)
 {
-    status                   = DatabaseCoreBackend::Unavailable;
-    isInTransaction          = false;
-    operationStatus          = DatabaseCoreBackend::ExecuteNormal;
-    errorHandler             = 0;
-    lock                     = 0;
-    errorLockOperationStatus = DatabaseCoreBackend::ExecuteNormal;
 }

-void DatabaseCoreBackendPrivate::init(const QString& name, DatabaseLocking* const l)
+DatabaseCoreBackendPrivate::~DatabaseCoreBackendPrivate()
 {
-    QObject::connect(QCoreApplication::instance(), SIGNAL(aboutToQuit()),
-                     q, SLOT(slotMainThreadFinished()));
+    // Must be shut down from the main thread.
+    // Clean up the QThreadStorage. It deletes any stored data.
+    threadDataStorage.setLocalData(0);
+}

+void DatabaseCoreBackendPrivate::init(const QString& name, DatabaseLocking* const l)
+{
     backendName = name;
     lock        = l;

@@ -110,82 +152,43 @@ void DatabaseCoreBackendPrivate::init(const QString& name, DatabaseLocking* cons
 // finishing of the thread.
 QSqlDatabase DatabaseCoreBackendPrivate::databaseForThread()
 {
-    QThread* const thread = QThread::currentThread();
-    QSqlDatabase db       = threadDatabases[thread];
-    int isValid           = databasesValid[thread];
-
-    if (!isValid || !db.isOpen())
+    DatabaseThreadData* threadData = 0;
+    if (!threadDataStorage.hasLocalData())
     {
-        // need to open a db for thread
-        bool success = open(db);
-
-        if (!success)
-        {
-            kDebug() << "Error while opening the database. Details: [" << db.lastError() << "]";
-        }
-
-        QObject::connect(thread, SIGNAL(finished()),
-                         q, SLOT(slotThreadFinished()));
+        threadData = new DatabaseThreadData;
+        threadDataStorage.setLocalData(threadData);
     }
-
-#ifdef DATABASCOREBACKEND_DEBUG
     else
     {
-        kDebug() << "Database ["<< connectionName(thread) <<"] already open for thread ["<< thread <<"].";
+        threadData = threadDataStorage.localData();
     }

-#endif
-
-    return db;
-}
-
-void DatabaseCoreBackendPrivate::closeDatabaseForThread()
-{
-    QThread* const thread = QThread::currentThread();
+    // do we need to reopen the database because parameter changed and validity was increased?
+    if (threadData->valid && threadData->valid < currentValidity)
+    {
+        threadData->closeDatabase();
+    }

-    // scope, so that db is destructed when calling removeDatabase
+    if (!threadData->valid || !threadData->database.isOpen())
     {
-        QSqlDatabase db = threadDatabases[thread];
+        threadData->database = createDatabaseConnection();

-        if (db.isValid())
+        if (threadData->database.open())
+        {
+            threadData->valid = currentValidity;
+        }
+        else
         {
-            db.close();
+            kDebug() << "Error while opening the database. Error was" << threadData->database.lastError();
         }
     }

-    threadDatabases.remove(thread);
-    databaseErrors.remove(thread);
-    databasesValid[thread] = 0;
-    transactionCount.remove(thread);
-    QSqlDatabase::removeDatabase(connectionName(thread));
-}
-
-QSqlError DatabaseCoreBackendPrivate::databaseErrorForThread()
-{
-    QThread* const thread = QThread::currentThread();
-    return databaseErrors[thread];
-}
-
-void DatabaseCoreBackendPrivate::setDatabaseErrorForThread(const QSqlError& lastError)
-{
-    QThread* const thread = QThread::currentThread();
-    databaseErrors.insert(thread, lastError);
-}
-
-QString DatabaseCoreBackendPrivate::connectionName(QThread* const thread)
-{
-    return backendName + QString::number((quintptr)thread);
+    return threadData->database;
 }

-bool DatabaseCoreBackendPrivate::open(QSqlDatabase& db)
+QSqlDatabase DatabaseCoreBackendPrivate::createDatabaseConnection()
 {
-    if (db.isValid())
-    {
-        db.close();
-    }
-
-    QThread* const thread  = QThread::currentThread();
-    db                     = QSqlDatabase::addDatabase(parameters.databaseType, connectionName(thread));
+    QSqlDatabase db        = QSqlDatabase::addDatabase(parameters.databaseType, connectionName());
     QString connectOptions = parameters.connectOptions;

     if (parameters.isSQLite())
@@ -211,46 +214,47 @@ bool DatabaseCoreBackendPrivate::open(QSqlDatabase& db)
     db.setUserName(parameters.userName);
     db.setPassword(parameters.password);

-    bool success = db.open();
+    return db;
+}

-    if (success==false)
+void DatabaseCoreBackendPrivate::closeDatabaseForThread()
+{
+    if (threadDataStorage.hasLocalData())
     {
-        kDebug() << "Error while opening the database. Error was <" << db.lastError() << ">";
+        threadDataStorage.localData()->closeDatabase();
     }
-
-    threadDatabases[thread]  = db;
-    databasesValid[thread]   = 1;
-    transactionCount[thread] = 0;
-
-    return success;
 }

-bool DatabaseCoreBackendPrivate::incrementTransactionCount()
+QSqlError DatabaseCoreBackendPrivate::databaseErrorForThread()
 {
-    QThread* const thread = QThread::currentThread();
-    return (!transactionCount[thread]++);
+    if (threadDataStorage.hasLocalData())
+    {
+        return threadDataStorage.localData()->lastError;
+    }
+    return QSqlError();
 }

-bool DatabaseCoreBackendPrivate::decrementTransactionCount()
+void DatabaseCoreBackendPrivate::setDatabaseErrorForThread(const QSqlError& lastError)
 {
-    QThread* const thread = QThread::currentThread();
-    return (!--transactionCount[thread]);
+    if (threadDataStorage.hasLocalData())
+    {
+        threadDataStorage.localData()->lastError = lastError;
+    }
 }

-bool DatabaseCoreBackendPrivate::isInTransactionInOtherThread() const
+QString DatabaseCoreBackendPrivate::connectionName()
 {
-    QThread* const thread = QThread::currentThread();
-    QHash<QThread*, int>::const_iterator it;
+    return backendName + QString::number((quintptr)QThread::currentThread());
+}

-    for (it = transactionCount.constBegin(); it != transactionCount.constEnd(); ++it)
-    {
-        if (it.key() != thread && it.value())
-        {
-            return true;
-        }
-    }
+bool DatabaseCoreBackendPrivate::incrementTransactionCount()
+{
+    return (!threadDataStorage.localData()->transactionCount++);
+}

-    return false;
+bool DatabaseCoreBackendPrivate::decrementTransactionCount()
+{
+    return (!--threadDataStorage.localData()->transactionCount);
 }

 bool DatabaseCoreBackendPrivate::isInMainThread() const
@@ -740,18 +744,6 @@ void DatabaseCoreBackend::setDatabaseErrorHandler(DatabaseErrorHandler* const ha
     d->errorHandler = handler;
 }

-void DatabaseCoreBackend::slotThreadFinished()
-{
-    Q_D(DatabaseCoreBackend);
-    d->closeDatabaseForThread();
-}
-
-void DatabaseCoreBackend::slotMainThreadFinished()
-{
-    Q_D(DatabaseCoreBackend);
-    d->closeDatabaseForThread();
-}
-
 bool DatabaseCoreBackend::isCompatible(const DatabaseParameters& parameters)
 {
     return QSqlDatabase::drivers().contains(parameters.databaseType);
@@ -761,10 +753,8 @@ bool DatabaseCoreBackend::open(const DatabaseParameters& parameters)
 {
     Q_D(DatabaseCoreBackend);
     d->parameters = parameters;
-
-    // Force possibly opened thread dbs to re-open with new parameters.
-    // They are not accessible from this thread!
-    d->databasesValid.clear();
+    // This will make possibly opened thread dbs reload at next access
+    d->currentValidity++;

     int retries = 0;

@@ -1634,7 +1624,7 @@ DatabaseCoreBackend::QueryState DatabaseCoreBackend::commitTransaction()
 bool DatabaseCoreBackend::isInTransaction() const
 {
     Q_D(const DatabaseCoreBackend);
-    return d->isInTransactionInOtherThread();
+    return d->isInTransaction;
 }

 void DatabaseCoreBackend::rollbackTransaction()
diff --git a/libs/database/core/databasecorebackend.h b/libs/database/core/databasecorebackend.h
index 41e47da..fa4dba4 100644
--- a/libs/database/core/databasecorebackend.h
+++ b/libs/database/core/databasecorebackend.h
@@ -472,11 +472,6 @@ public:
             LastInsertId
     */

-private Q_SLOTS:
-
-    void slotThreadFinished();
-    void slotMainThreadFinished();
-
 protected:

     DatabaseCoreBackendPrivate* const d_ptr;
diff --git a/libs/database/core/databasecorebackend_p.h b/libs/database/core/databasecorebackend_p.h
index 2078509..ff3a3fa 100644
--- a/libs/database/core/databasecorebackend_p.h
+++ b/libs/database/core/databasecorebackend_p.h
@@ -29,6 +29,7 @@
 #include <QHash>
 #include <QSqlDatabase>
 #include <QThread>
+#include <QThreadStorage>
 #include <QWaitCondition>

 // Local includes
@@ -40,25 +41,38 @@
 namespace Digikam
 {

+class DatabaseThreadData
+{
+public:
+
+    DatabaseThreadData();
+    ~DatabaseThreadData();
+
+    void closeDatabase();
+
+    QSqlDatabase database;
+    int          valid;
+    int          transactionCount;
+    QSqlError    lastError;
+};
+
 class DIGIKAM_EXPORT DatabaseCoreBackendPrivate : public DatabaseErrorAnswer
 {
 public:

     explicit DatabaseCoreBackendPrivate(DatabaseCoreBackend* const backend);
-    virtual ~DatabaseCoreBackendPrivate()
-    {
-    }
+    virtual ~DatabaseCoreBackendPrivate();

     void init(const QString& connectionName, DatabaseLocking* const locking);

-    QString connectionName(QThread* const thread);
+    QString connectionName();

     QSqlDatabase databaseForThread();
     QSqlError    databaseErrorForThread();
     void         setDatabaseErrorForThread(const QSqlError& lastError);

+    QSqlDatabase createDatabaseConnection();
     void closeDatabaseForThread();
-    bool open(QSqlDatabase& db);
     bool incrementTransactionCount();
     bool decrementTransactionCount();
     bool isInTransactionInOtherThread() const;
@@ -88,14 +102,10 @@ public:

 public:

-    // this is always accessed in mutex context, no need for QThreadStorage
-    QHash<QThread*, QSqlDatabase>             threadDatabases;
-    // this is not only db.isValid(), but also "parameters changed, need to reopen"
-    QHash<QThread*, int>                      databasesValid;
-    // for recursive transactions
-    QHash<QThread*, int>                      transactionCount;
+    QThreadStorage<DatabaseThreadData*>       threadDataStorage;

-    QHash<QThread*, QSqlError>                databaseErrors;
+    // This compares to DatabaseThreadData's valid. If currentValidity is increased and > valid, the db is marked as invalid
+    int                                       currentValidity;

     bool                                      isInTransaction;
Comment 102 caulier.gilles 2014-12-15 12:56:50 UTC
Git commit bab8c3f159930a3b0b31d09b25b9eb42ab1c3c62 by Marcel Wiesweg.
Committed on 15/11/2014 at 14:45.
Pushed by mwiesweg into branch 'master'.

Backport all core db changes regarding thread clean up from main digikam
Back and forward port code polish and style changes

M  +153  -150  libkface/database/databasecorebackend.cpp
M  +15   -12   libkface/database/databasecorebackend.h
M  +35   -25   libkface/database/databasecorebackend_p.h
M  +1    -0    libkface/database/databaseparameters.h

http://commits.kde.org/libkface/bab8c3f159930a3b0b31d09b25b9eb42ab1c3c62

diff --git a/libkface/database/databasecorebackend.cpp b/libkface/database/databasecorebackend.cpp
index 784e9b7..2193ce2 100644
--- a/libkface/database/databasecorebackend.cpp
+++ b/libkface/database/databasecorebackend.cpp
@@ -21,12 +21,6 @@
  *
  * ============================================================ */

-/*
-#ifndef DATABASCOREBACKEND_DEBUG
-#define DATABASCOREBACKEND_DEBUG
-#endif
-*/
-
 #include "databasecorebackend.moc"
 #include "databasecorebackend_p.h"

@@ -51,8 +45,8 @@

 // Local includes

-#include "schemaupdater.h"
 #include "dbactiontype.h"
+#include "schemaupdater.h"

 namespace KFaceIface
 {
@@ -102,22 +96,65 @@ DatabaseCoreBackendPrivate::ErrorLocker::ErrorLocker(DatabaseCoreBackendPrivate*

 // -----------------------------------------------------------------------------------------

+
+DatabaseThreadData::DatabaseThreadData()
+    : valid(0),
+      transactionCount(0)
+{
+}
+
+DatabaseThreadData::~DatabaseThreadData()
+{
+    if (transactionCount)
+    {
+        kDebug() << "WARNING !!! Transaction count is" << transactionCount << "when destroying database!!!";
+    }
+    closeDatabase();
+}
+
+void DatabaseThreadData::closeDatabase()
+{
+    QString connectionToRemove;
+    if (database.isOpen())
+    {
+        connectionToRemove = database.connectionName();
+    }
+
+    // Destroy object
+    database = QSqlDatabase();
+
+    valid            = 0;
+    transactionCount = 0;
+    lastError        = QSqlError();
+
+    // Remove connection
+    if (!connectionToRemove.isNull())
+    {
+        QSqlDatabase::removeDatabase(connectionToRemove);
+    }
+}
+
 DatabaseCoreBackendPrivate::DatabaseCoreBackendPrivate(DatabaseCoreBackend* const backend)
-    : q(backend)
+    : currentValidity(0),
+      isInTransaction(false),
+      status(DatabaseCoreBackend::Unavailable),
+      lock(0),
+      operationStatus(DatabaseCoreBackend::ExecuteNormal),
+      errorLockOperationStatus(DatabaseCoreBackend::ExecuteNormal),
+      errorHandler(0),
+      q(backend)
 {
-    status                   = DatabaseCoreBackend::Unavailable;
-    isInTransaction          = false;
-    operationStatus          = DatabaseCoreBackend::ExecuteNormal;
-    errorLockOperationStatus = DatabaseCoreBackend::ExecuteNormal;
-    errorHandler             = 0;
-    lock                     = 0;
 }

-void DatabaseCoreBackendPrivate::init(const QString& name, DatabaseLocking* const l)
+DatabaseCoreBackendPrivate::~DatabaseCoreBackendPrivate()
 {
-    QObject::connect(QCoreApplication::instance(), SIGNAL(aboutToQuit()),
-                     q, SLOT(slotMainThreadFinished()));
+    // Must be shut down from the main thread.
+    // Clean up the QThreadStorage. It deletes any stored data.
+    threadDataStorage.setLocalData(0);
+}

+void DatabaseCoreBackendPrivate::init(const QString& name, DatabaseLocking* const l)
+{
     backendName = name;
     lock        = l;

@@ -133,84 +170,44 @@ void DatabaseCoreBackendPrivate::init(const QString& name, DatabaseLocking* cons
 // finishing of the thread.
 QSqlDatabase DatabaseCoreBackendPrivate::databaseForThread()
 {
-    QThread* const thread = QThread::currentThread();
-    QSqlDatabase db       = threadDatabases[thread];
-    int isValid           = databasesValid[thread];
-
-    if (!isValid || !db.isOpen())
+    DatabaseThreadData* threadData = 0;
+    if (!threadDataStorage.hasLocalData())
     {
-        // need to open a db for thread
-        bool success = open(db);
-
-        if (!success)
-        {
-            kDebug() << "Error while opening the database. Details: [" << db.lastError() << "]";
-        }
-
-        QObject::connect(thread, SIGNAL(finished()),
-                         q, SLOT(slotThreadFinished()));
+        threadData = new DatabaseThreadData;
+        threadDataStorage.setLocalData(threadData);
     }
-
-#ifdef DATABASCOREBACKEND_DEBUG
     else
     {
-        kDebug() << "Database ["<< connectionName(thread) <<"] already open for thread ["<< thread <<"].";
+        threadData = threadDataStorage.localData();
     }

-#endif
-
-    return db;
-}
-
-void DatabaseCoreBackendPrivate::closeDatabaseForThread()
-{
-    QThread* const thread = QThread::currentThread();
+    // do we need to reopen the database because parameter changed and validity was increased?
+    if (threadData->valid && threadData->valid < currentValidity)
+    {
+        threadData->closeDatabase();
+    }

-    // scope, so that db is destructed when calling removeDatabase
+    if (!threadData->valid || !threadData->database.isOpen())
     {
-        QSqlDatabase db = threadDatabases[thread];
+        threadData->database = createDatabaseConnection();

-        if (db.isValid())
+        if (threadData->database.open())
         {
-            db.close();
+            threadData->valid = currentValidity;
+        }
+        else
+        {
+            kDebug() << "Error while opening the database. Error was" << threadData->database.lastError();
         }
     }

-    threadDatabases.remove(thread);
-    databaseErrors.remove(thread);
-    databasesValid[thread] = 0;
-    transactionCount.remove(thread);
-    QSqlDatabase::removeDatabase(connectionName(thread));
-}
-
-QSqlError DatabaseCoreBackendPrivate::databaseErrorForThread()
-{
-    QThread* const thread = QThread::currentThread();
-    return databaseErrors[thread];
-}
-
-void DatabaseCoreBackendPrivate::setDatabaseErrorForThread(const QSqlError& lastError)
-{
-    QThread* const thread = QThread::currentThread();
-    databaseErrors.insert(thread, lastError);
-}
-
-QString DatabaseCoreBackendPrivate::connectionName(QThread* const thread)
-{
-    return backendName + QString::number((quintptr)thread);
+    return threadData->database;
 }

-bool DatabaseCoreBackendPrivate::open(QSqlDatabase& db)
+QSqlDatabase DatabaseCoreBackendPrivate::createDatabaseConnection()
 {
-    if (db.isValid())
-    {
-        db.close();
-    }
-
-    QThread* const thread = QThread::currentThread();
-    db                    = QSqlDatabase::addDatabase(parameters.databaseType, connectionName(thread));
-
-    QString connectOptions;// = parameters.connectOptions;
+    QSqlDatabase db        = QSqlDatabase::addDatabase(parameters.databaseType, connectionName());
+    QString connectOptions = parameters.connectOptions;

     if (parameters.isSQLite())
     {
@@ -230,55 +227,48 @@ bool DatabaseCoreBackendPrivate::open(QSqlDatabase& db)

     db.setDatabaseName(parameters.databaseName);
     db.setConnectOptions(connectOptions);
-    /*db.setHostName(parameters.hostName);
-    db.setPort(parameters.port);
-    db.setUserName(parameters.userName);
-    db.setPassword(parameters.password);*/

-    bool success = db.open();
+    return db;
+}

-    if (success)
+void DatabaseCoreBackendPrivate::closeDatabaseForThread()
+{
+    if (threadDataStorage.hasLocalData())
     {
-        db.exec("PRAGMA synchronous=1;");
+        threadDataStorage.localData()->closeDatabase();
     }
-    else
+}
+
+QSqlError DatabaseCoreBackendPrivate::databaseErrorForThread()
+{
+    if (threadDataStorage.hasLocalData())
     {
-        kDebug() << "Error while opening the database. Error was <" << db.lastError() << ">";
+        return threadDataStorage.localData()->lastError;
     }
-
-    threadDatabases[thread]  = db;
-    databasesValid[thread]   = 1;
-    transactionCount[thread] = 0;
-
-    return success;
+    return QSqlError();
 }

-bool DatabaseCoreBackendPrivate::incrementTransactionCount()
+void DatabaseCoreBackendPrivate::setDatabaseErrorForThread(const QSqlError& lastError)
 {
-    QThread* const thread = QThread::currentThread();
-    return !transactionCount[thread]++;
+    if (threadDataStorage.hasLocalData())
+    {
+        threadDataStorage.localData()->lastError = lastError;
+    }
 }

-bool DatabaseCoreBackendPrivate::decrementTransactionCount()
+QString DatabaseCoreBackendPrivate::connectionName()
 {
-    QThread* const thread = QThread::currentThread();
-    return !--transactionCount[thread];
+    return backendName + QString::number((quintptr)QThread::currentThread());
 }

-bool DatabaseCoreBackendPrivate::isInTransactionInOtherThread() const
+bool DatabaseCoreBackendPrivate::incrementTransactionCount()
 {
-    QThread* const thread = QThread::currentThread();
-    QHash<QThread*, int>::const_iterator it;
-
-    for (it = transactionCount.constBegin(); it != transactionCount.constEnd(); ++it)
-    {
-        if (it.key() != thread && it.value())
-        {
-            return true;
-        }
-    }
+    return (!threadDataStorage.localData()->transactionCount++);
+}

-    return false;
+bool DatabaseCoreBackendPrivate::decrementTransactionCount()
+{
+    return (!--threadDataStorage.localData()->transactionCount);
 }

 bool DatabaseCoreBackendPrivate::isInMainThread() const
@@ -295,7 +285,7 @@ bool DatabaseCoreBackendPrivate::isInUIThread() const
         return false;
     }

-    return QThread::currentThread() == app->thread();
+    return (QThread::currentThread() == app->thread());
 }

 bool DatabaseCoreBackendPrivate::reconnectOnError() const
@@ -325,7 +315,8 @@ bool DatabaseCoreBackendPrivate::isConnectionError(const SqlQuery& query) const
         return false;
     }

-    return (query.lastError().type() == QSqlError::ConnectionError || query.lastError().number()==2006);
+    return query.lastError().type()   == QSqlError::ConnectionError ||
+           query.lastError().number() == 2006;
 }

 bool DatabaseCoreBackendPrivate::needToConsultUserForError(const SqlQuery&) const
@@ -336,12 +327,15 @@ bool DatabaseCoreBackendPrivate::needToConsultUserForError(const SqlQuery&) cons

 bool DatabaseCoreBackendPrivate::needToHandleWithErrorHandler(const SqlQuery& query) const
 {
-    return isConnectionError(query) || needToConsultUserForError(query);
+    return (isConnectionError(query) || needToConsultUserForError(query));
 }

 bool DatabaseCoreBackendPrivate::checkRetrySQLiteLockError(int retries)
 {
-    kDebug() << "Database is locked. Waited" << retries*10;
+    if (!(retries % 25))
+    {
+        kDebug() << "Database is locked. Waited" << retries*10;
+    }

     const int uiMaxRetries = 50;
     const int maxRetries   = 1000;
@@ -476,7 +470,7 @@ bool DatabaseCoreBackendPrivate::handleWithErrorHandler(const SqlQuery* const qu
     }
     else
     {
-        //TODO check if it's better to use an own error handler for kio slaves.
+        // TODO check if it's better to use an own error handler for kio slaves.
         // But for now, close only the database in the hope, that the next
         // access will be successful.
         closeDatabaseForThread();
@@ -755,18 +749,6 @@ void DatabaseCoreBackend::setDatabaseErrorHandler(DatabaseErrorHandler* const ha
     d->errorHandler = handler;
 }

-void DatabaseCoreBackend::slotThreadFinished()
-{
-    Q_D(DatabaseCoreBackend);
-    d->closeDatabaseForThread();
-}
-
-void DatabaseCoreBackend::slotMainThreadFinished()
-{
-    Q_D(DatabaseCoreBackend);
-    d->closeDatabaseForThread();
-}
-
 bool DatabaseCoreBackend::isCompatible(const DatabaseParameters& parameters)
 {
     return QSqlDatabase::drivers().contains(parameters.databaseType);
@@ -776,10 +758,8 @@ bool DatabaseCoreBackend::open(const DatabaseParameters& parameters)
 {
     Q_D(DatabaseCoreBackend);
     d->parameters = parameters;
-
-    // Force possibly opened thread dbs to re-open with new parameters.
-    // They are not accessible from this thread!
-    d->databasesValid.clear();
+    // This will make possibly opened thread dbs reload at next access
+    d->currentValidity++;

     int retries = 0;

@@ -913,7 +893,7 @@ DatabaseCoreBackend::QueryState DatabaseCoreBackend::handleQueryResult(SqlQuery&
     return DatabaseCoreBackend::NoErrors;
 }

-// ----------------------------------------------------------------------------------------------------
+// -------------------------------------------------------------------------------------

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql, QList<QVariant>* const values, QVariant* const lastInsertId)
 {
@@ -922,52 +902,54 @@ DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql,
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql, const QVariant& boundValue1,
-        QList<QVariant>* const values, QVariant* const lastInsertId)
+                                                             QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     SqlQuery query = execQuery(sql, boundValue1);
     return handleQueryResult(query, values, lastInsertId);
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql,
-        const QVariant& boundValue1, const QVariant& boundValue2,
-        QList<QVariant>* const values, QVariant* const lastInsertId)
+                                                             const QVariant& boundValue1, const QVariant& boundValue2,
+                                                             QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     SqlQuery query = execQuery(sql, boundValue1, boundValue2);
     return handleQueryResult(query, values, lastInsertId);
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql,
-        const QVariant& boundValue1, const QVariant& boundValue2,
-        const QVariant& boundValue3, QList<QVariant>* const values,
-        QVariant* const lastInsertId)
+                                                             const QVariant& boundValue1, const QVariant& boundValue2,
+                                                             const QVariant& boundValue3, QList<QVariant>* const values,
+                                                             QVariant* const lastInsertId)
 {
     SqlQuery query = execQuery(sql, boundValue1, boundValue2, boundValue3);
     return handleQueryResult(query, values, lastInsertId);
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql,
-        const QVariant& boundValue1, const QVariant& boundValue2,
-        const QVariant& boundValue3, const QVariant& boundValue4,
-        QList<QVariant>* const values, QVariant* const lastInsertId)
+                                                             const QVariant& boundValue1, const QVariant& boundValue2,
+                                                             const QVariant& boundValue3, const QVariant& boundValue4,
+                                                             QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     SqlQuery query = execQuery(sql, boundValue1, boundValue2, boundValue3, boundValue4);
     return handleQueryResult(query, values, lastInsertId);
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql, const QList<QVariant>& boundValues,
-        QList<QVariant>* const values, QVariant* const lastInsertId)
+                                                             QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     SqlQuery query = execQuery(sql, boundValues);
     return handleQueryResult(query, values, lastInsertId);
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql, const QMap<QString, QVariant>& bindingMap,
-        QList<QVariant>* const values, QVariant* const lastInsertId)
+                                                             QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     SqlQuery query = execQuery(sql, bindingMap);
     return handleQueryResult(query, values, lastInsertId);
 }

+// -------------------------------------------------------------------------------------
+
 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(SqlQuery& preparedQuery, QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     exec(preparedQuery);
@@ -1014,6 +996,8 @@ DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(SqlQuery& preparedQ
     return handleQueryResult(preparedQuery, values, lastInsertId);
 }

+// -------------------------------------------------------------------------------------
+
 SqlQuery DatabaseCoreBackend::execQuery(const QString& sql, const QVariant& boundValue1)
 {
     SqlQuery query = prepareQuery(sql);
@@ -1066,6 +1050,8 @@ SqlQuery DatabaseCoreBackend::execQuery(const QString& sql)
     return query;
 }

+// -------------------------------------------------------------------------------------
+
 void DatabaseCoreBackend::execQuery(SqlQuery& query, const QVariant& boundValue1)
 {
     query.bindValue(0, boundValue1);
@@ -1111,6 +1097,8 @@ void DatabaseCoreBackend::execQuery(SqlQuery& query, const QList<QVariant>& boun
     exec(query);
 }

+// -------------------------------------------------------------------------------------
+
 SqlQuery DatabaseCoreBackend::execQuery(const QString& sql, const QMap<QString, QVariant>& bindingMap)
 {
     QString preparedString = sql;
@@ -1252,7 +1240,7 @@ SqlQuery DatabaseCoreBackend::execQuery(const QString& sql, const QMap<QString,

     SqlQuery query = prepareQuery(preparedString);

-    for (int i=0; i<valuesToBind.size(); ++i)
+    for (int i=0; i < valuesToBind.size(); ++i)
     {
         query.bindValue(i, valuesToBind.at(i));
     }
@@ -1262,12 +1250,12 @@ SqlQuery DatabaseCoreBackend::execQuery(const QString& sql, const QMap<QString,
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execUpsertDBAction(const DatabaseAction& action, const QVariant& id,
-        const QStringList fieldNames, const QList<QVariant>& values)
+                                                                        const QStringList fieldNames, const QList<QVariant>& values)
 {
     QMap<QString, QVariant> parameters;
     QMap<QString, QVariant> fieldValueMap;

-    for (int i=0; i<fieldNames.size(); ++i)
+    for (int i = 0; i < fieldNames.size(); ++i)
     {
         fieldValueMap.insert(fieldNames.at(i), values.at(i));
     }
@@ -1285,7 +1273,7 @@ DatabaseCoreBackend::QueryState DatabaseCoreBackend::execUpsertDBAction(const Da
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execUpsertDBAction(const QString& action, const QVariant& id,
-        const QStringList fieldNames, const QList<QVariant>& values)
+                                                                        const QStringList fieldNames, const QList<QVariant>& values)
 {
     return execUpsertDBAction(getDBAction(action), id, fieldNames, values);
 }
@@ -1516,6 +1504,7 @@ SqlQuery DatabaseCoreBackend::copyQuery(const SqlQuery& old)
 #endif
     query.prepare(old.lastQuery());
     query.setForwardOnly(old.isForwardOnly());
+
     // only for positional binding
     QList<QVariant> boundValues = old.boundValues().values();

@@ -1635,7 +1624,7 @@ DatabaseCoreBackend::QueryState DatabaseCoreBackend::commitTransaction()
 bool DatabaseCoreBackend::isInTransaction() const
 {
     Q_D(const DatabaseCoreBackend);
-    return d->isInTransactionInOtherThread();
+    return d->isInTransaction;
 }

 void DatabaseCoreBackend::rollbackTransaction()
@@ -1663,4 +1652,18 @@ QString DatabaseCoreBackend::lastError()
     return d->databaseForThread().lastError().text();
 }

+int DatabaseCoreBackend::maximumBoundValues() const
+{
+    Q_D(const DatabaseCoreBackend);
+
+    if (d->parameters.isSQLite())
+    {
+        return 999;   // SQLITE_MAX_VARIABLE_NUMBER
+    }
+    else
+    {
+        return 65535; // MySQL
+    }
+}
+
 } // namespace KFaceIface
diff --git a/libkface/database/databasecorebackend.h b/libkface/database/databasecorebackend.h
index 9ce859a..a1beac3 100644
--- a/libkface/database/databasecorebackend.h
+++ b/libkface/database/databasecorebackend.h
@@ -44,6 +44,7 @@ namespace KFaceIface
 {

 class DatabaseCoreBackendPrivate;
+class DatabaseErrorHandler;
 class SchemaUpdater;

 class DatabaseLocking
@@ -147,17 +148,18 @@ public:
      */
     void close();

-    // -----------------------------------------------------------
+public:

     class QueryState
     {
     public:

-        QueryState() : value(DatabaseCoreBackend::NoErrors)
+        QueryState()
+            : value(DatabaseCoreBackend::NoErrors)
         {
         }

-        QueryState(QueryStateEnum value)
+        QueryState(const QueryStateEnum value)
             : value(value)
         {
         }
@@ -177,6 +179,8 @@ public:
         QueryStateEnum value;
     };

+public:
+
     /**
      * Returns the current status of the database backend
      */
@@ -275,10 +279,8 @@ public:
                        const QVariant& boundValue1, const QVariant& boundValue2,
                        const QVariant& boundValue3, const QVariant& boundValue4,
                        QList<QVariant>* const values = 0, QVariant* const lastInsertId = 0);
-    QueryState execSql(const QString& sql,
-                       const QList<QVariant>& boundValues,
-                       QList<QVariant>* const values = 0,
-                       QVariant* const lastInsertId = 0);
+    QueryState execSql(const QString& sql, const QList<QVariant>& boundValues,
+                       QList<QVariant>* const values = 0, QVariant* const lastInsertId = 0);

     QueryState execSql(SqlQuery& preparedQuery, QList<QVariant>* const values = 0, QVariant* const lastInsertId = 0);
     QueryState execSql(SqlQuery& preparedQuery, const QVariant& boundValue1,
@@ -438,6 +440,12 @@ public:
      */
     QSqlError lastSQLError();

+    /**
+     * Returns the maximum number of bound parameters allowed per query.
+     * This value depends on the database engine.
+     */
+    int maximumBoundValues() const;
+
     /*
         Qt SQL driver supported features
         SQLITE3:
@@ -462,11 +470,6 @@ public:
             LastInsertId
     */

-private Q_SLOTS:
-
-    void slotThreadFinished();
-    void slotMainThreadFinished();
-
 protected:

     DatabaseCoreBackendPrivate* const d_ptr;
diff --git a/libkface/database/databasecorebackend_p.h b/libkface/database/databasecorebackend_p.h
index c39b33f..27faaa5 100644
--- a/libkface/database/databasecorebackend_p.h
+++ b/libkface/database/databasecorebackend_p.h
@@ -29,6 +29,7 @@
 #include <QHash>
 #include <QSqlDatabase>
 #include <QThread>
+#include <QThreadStorage>
 #include <QWaitCondition>

 // Local includes
@@ -38,40 +39,54 @@
 namespace KFaceIface
 {

+class DatabaseThreadData
+{
+public:
+
+    DatabaseThreadData();
+    ~DatabaseThreadData();
+
+    void closeDatabase();
+
+    QSqlDatabase database;
+    int          valid;
+    int          transactionCount;
+    QSqlError    lastError;
+};
+
 class DatabaseCoreBackendPrivate : public DatabaseErrorAnswer
 {
 public:

-    DatabaseCoreBackendPrivate(DatabaseCoreBackend* const backend);
-    virtual ~DatabaseCoreBackendPrivate() {}
+    explicit DatabaseCoreBackendPrivate(DatabaseCoreBackend* const backend);
+    virtual ~DatabaseCoreBackendPrivate();

     void init(const QString& connectionName, DatabaseLocking* const locking);

-    QString connectionName(QThread* const thread);
+    QString connectionName();

     QSqlDatabase databaseForThread();
     QSqlError    databaseErrorForThread();
     void         setDatabaseErrorForThread(const QSqlError& lastError);

+    QSqlDatabase createDatabaseConnection();
     void closeDatabaseForThread();
-    bool open(QSqlDatabase& db);
     bool incrementTransactionCount();
     bool decrementTransactionCount();
-    bool isInTransactionInOtherThread() const;

     bool isInMainThread() const;
-    bool isInUIThread() const;
+    bool isInUIThread()   const;

-    bool reconnectOnError() const;
-    bool isSQLiteLockError(const SqlQuery& query) const;
+    bool reconnectOnError()                                       const;
+    bool isSQLiteLockError(const SqlQuery& query)                 const;
     bool isSQLiteLockTransactionError(const QSqlError& lastError) const;
-    bool checkRetrySQLiteLockError(int retries);
-    bool isConnectionError(const SqlQuery& query) const;
-    bool needToConsultUserForError(const SqlQuery& query) const;
-    bool needToHandleWithErrorHandler(const SqlQuery& query) const;
-    void debugOutputFailedQuery(const QSqlQuery& query) const;
-    void debugOutputFailedTransaction(const QSqlError& error) const;
+    bool isConnectionError(const SqlQuery& query)                 const;
+    bool needToConsultUserForError(const SqlQuery& query)         const;
+    bool needToHandleWithErrorHandler(const SqlQuery& query)      const;
+    void debugOutputFailedQuery(const QSqlQuery& query)           const;
+    void debugOutputFailedTransaction(const QSqlError& error)     const;

+    bool checkRetrySQLiteLockError(int retries);
     bool checkOperationStatus();
     bool handleWithErrorHandler(const SqlQuery* const query);
     void setQueryOperationFlag(DatabaseCoreBackend::QueryOperationStatus status);
@@ -80,19 +95,14 @@ public:
     // called by DatabaseErrorHandler, implementing DatabaseErrorAnswer
     virtual void connectionErrorContinueQueries();
     virtual void connectionErrorAbortQueries();
-
     virtual void transactionFinished();

 public:

-    // this is always accessed in mutex context, no need for QThreadStorage
-    QHash<QThread*, QSqlDatabase>             threadDatabases;
-    // this is not only db.isValid(), but also "parameters changed, need to reopen"
-    QHash<QThread*, int>                      databasesValid;
-    // for recursive transactions
-    QHash<QThread*, int>                      transactionCount;
+    QThreadStorage<DatabaseThreadData*>       threadDataStorage;

-    QHash<QThread*, QSqlError>                databaseErrors;
+    // This compares to DatabaseThreadData's valid. If currentValidity is increased and > valid, the db is marked as invalid
+    int                                       currentValidity;

     bool                                      isInTransaction;

@@ -121,7 +131,7 @@ public :
     {
     public:

-        AbstractUnlocker(DatabaseCoreBackendPrivate* const d);
+        explicit AbstractUnlocker(DatabaseCoreBackendPrivate* const d);
         ~AbstractUnlocker();

         void finishAcquire();
@@ -157,7 +167,7 @@ public :
     {
     public:

-        ErrorLocker(DatabaseCoreBackendPrivate* const d);
+        explicit ErrorLocker(DatabaseCoreBackendPrivate* const d);
         void wait();
     };

@@ -167,7 +177,7 @@ public :
     {
     public:

-        BusyWaiter(DatabaseCoreBackendPrivate* const d);
+        explicit BusyWaiter(DatabaseCoreBackendPrivate* const d);
     };

 public :
diff --git a/libkface/database/databaseparameters.h b/libkface/database/databaseparameters.h
index 655b87a..7cf84ec 100644
--- a/libkface/database/databaseparameters.h
+++ b/libkface/database/databaseparameters.h
@@ -52,6 +52,7 @@ public:

     QString databaseType;
     QString databaseName;
+    QString connectOptions;

     bool operator==(const DatabaseParameters& other) const;
     bool operator!=(const DatabaseParameters& other) const;
Comment 103 caulier.gilles 2014-12-15 12:57:46 UTC
Git commit 0fb2b35c650b20f7c208bb5b07bd701fd4d09d1f by Marcel Wiesweg.
Committed on 15/11/2014 at 14:46.
Pushed by mwiesweg into branch 'master'.

Back and forward port code polish and style changes from libkface db backend.
The files have now only the needed minor differences

M  +140  -136  libs/database/core/databasecorebackend.cpp
M  +6    -9    libs/database/core/databasecorebackend.h
M  +4    -5    libs/database/core/databasecorebackend_p.h

http://commits.kde.org/digikam/0fb2b35c650b20f7c208bb5b07bd701fd4d09d1f

diff --git a/libs/database/core/databasecorebackend.cpp b/libs/database/core/databasecorebackend.cpp
index 76a958a..480f437 100644
--- a/libs/database/core/databasecorebackend.cpp
+++ b/libs/database/core/databasecorebackend.cpp
@@ -6,7 +6,7 @@
  * Date        : 2007-04-15
  * Description : Abstract database backend
  *
- * Copyright (C) 2007-2010 by Marcel Wiesweg <marcel dot wiesweg at gmx dot de>
+ * Copyright (C) 2007-2012 by Marcel Wiesweg <marcel dot wiesweg at gmx dot de>
  *
  * This program is free software; you can redistribute it
  * and/or modify it under the terms of the GNU General
@@ -42,22 +42,23 @@
 // KDE includes

 #include <kdebug.h>
-#include <kglobal.h>

 // Local includes

-#include "thumbnailschemaupdater.h"
 #include "dbactiontype.h"
+#include "thumbnailschemaupdater.h"

-//#define DATABASCOREBACKEND_DEBUG 1
 namespace Digikam
 {

 DatabaseLocking::DatabaseLocking()
-    : mutex(QMutex::Recursive), lockCount(0) // create a recursive mutex
+    : mutex(QMutex::Recursive),
+      lockCount(0) // create a recursive mutex
 {
 }

+// -----------------------------------------------------------------------------------------
+
 // For whatever reason, these methods are "static protected"
 class sotoSleep : public QThread
 {
@@ -79,6 +80,23 @@ public:
     }
 };

+// -----------------------------------------------------------------------------------------
+
+DatabaseCoreBackendPrivate::BusyWaiter::BusyWaiter(DatabaseCoreBackendPrivate* const d)
+    : AbstractWaitingUnlocker(d, &d->busyWaitMutex, &d->busyWaitCondVar)
+{
+}
+
+// -----------------------------------------------------------------------------------------
+
+DatabaseCoreBackendPrivate::ErrorLocker::ErrorLocker(DatabaseCoreBackendPrivate* const d)
+    : AbstractWaitingUnlocker(d, &d->errorLockMutex, &d->errorLockCondVar)
+{
+}
+
+// -----------------------------------------------------------------------------------------
+
+
 DatabaseThreadData::DatabaseThreadData()
     : valid(0),
       transactionCount(0)
@@ -288,7 +306,7 @@ bool DatabaseCoreBackendPrivate::isSQLiteLockError(const SqlQuery& query) const
 bool DatabaseCoreBackendPrivate::isSQLiteLockTransactionError(const QSqlError& lastError) const
 {
     return parameters.isSQLite() &&
-           lastError.type() == QSqlError::TransactionError &&
+           lastError.type()         == QSqlError::TransactionError &&
            lastError.databaseText() == QLatin1String("database is locked");
     // wouldnt it be great if they gave us the database error number...
 }
@@ -356,74 +374,6 @@ void DatabaseCoreBackendPrivate::debugOutputFailedTransaction(const QSqlError& e
              << error.number() << error.type();
 }

-
-DatabaseCoreBackendPrivate::AbstractUnlocker::AbstractUnlocker(DatabaseCoreBackendPrivate* const d)
-    : count(0), d(d)
-{
-    // Why two mutexes? The main mutex is recursive and won't work with a condvar.
-
-    // acquire lock
-    d->lock->mutex.lock();
-    // store lock count
-    count = d->lock->lockCount;
-    // set lock count to 0
-    d->lock->lockCount = 0;
-
-    // unlock
-    for (int i=0; i<count; ++i)
-    {
-        d->lock->mutex.unlock();
-    }
-}
-
-void DatabaseCoreBackendPrivate::AbstractUnlocker::finishAcquire()
-{
-    // drop lock acquired in first line. Main mutex is now free.
-    // We maintain lock order (first main mutex, second error lock mutex)
-    // but we drop main mutex lock for waiting on the cond var.
-    d->lock->mutex.unlock();
-}
-
-DatabaseCoreBackendPrivate::AbstractUnlocker::~AbstractUnlocker()
-{
-    // lock main mutex as often as it was locked before
-    for (int i=0; i<count; ++i)
-    {
-        d->lock->mutex.lock();
-    }
-
-    // update lock count
-    d->lock->lockCount += count;
-}
-
-DatabaseCoreBackendPrivate::AbstractWaitingUnlocker::AbstractWaitingUnlocker(DatabaseCoreBackendPrivate* const d,
-                                                                             QMutex* const mutex, QWaitCondition* const condVar)
-    : AbstractUnlocker(d), mutex(mutex), condVar(condVar)
-{
-    // Why two mutexes? The main mutex is recursive and won't work with a condvar.
-    // lock condvar mutex (lock only if main mutex is locked)
-    mutex->lock();
-
-    finishAcquire();
-}
-
-DatabaseCoreBackendPrivate::AbstractWaitingUnlocker::~AbstractWaitingUnlocker()
-{
-    // unlock condvar mutex. Both mutexes are now free.
-    mutex->unlock();
-    // now base class destructor is executed, reallocating main mutex
-}
-
-bool DatabaseCoreBackendPrivate::AbstractWaitingUnlocker::wait(unsigned long time)
-{
-    return condVar->wait(mutex, time);
-}
-
-DatabaseCoreBackendPrivate::BusyWaiter::BusyWaiter(DatabaseCoreBackendPrivate* const d)
-    : AbstractWaitingUnlocker(d, &d->busyWaitMutex, &d->busyWaitCondVar)
-{
-}
-
 void DatabaseCoreBackendPrivate::transactionFinished()
 {
     // wakes up any BusyWaiter waiting on the busyWaitCondVar.
@@ -431,39 +381,18 @@ void DatabaseCoreBackendPrivate::transactionFinished()
     busyWaitCondVar.wakeOne();
 }

-DatabaseCoreBackendPrivate::ErrorLocker::ErrorLocker(DatabaseCoreBackendPrivate* const d)
-    : AbstractWaitingUnlocker(d, &d->errorLockMutex, &d->errorLockCondVar)
-{
-}
-
-/** This suspends the current thread if the query status as
- *  set by setFlag() is Wait and until the thread is woken with wakeAll().
- *  The DatabaseAccess mutex will be unlocked while waiting.
- */
-void DatabaseCoreBackendPrivate::ErrorLocker::wait()
-{
-    // we use a copy of the flag under lock of the errorLockMutex to be able to check it here
-    while (d->errorLockOperationStatus == DatabaseCoreBackend::Wait)
-    {
-        wait();
-    }
-}
-
-/** Set the wait flag to queryStatus. Typically, call this with Wait.
- */
+/** Set the wait flag to queryStatus. Typically, call this with Wait. */
 void DatabaseCoreBackendPrivate::setQueryOperationFlag(DatabaseCoreBackend::QueryOperationStatus status)
 {
     // Enforce lock order (first main mutex, second error lock mutex)
     QMutexLocker l(&errorLockMutex);
-
     // this change must be done under errorLockMutex lock
     errorLockOperationStatus = status;
     operationStatus          = status;
 }

 /** Set the wait flag to queryStatus and wake all waiting threads.
- *  Typically, call wakeAll with status ExecuteNormal or AbortQueries.
- */
+ *  Typically, call wakeAll with status ExecuteNormal or AbortQueries. */
 void DatabaseCoreBackendPrivate::queryOperationWakeAll(DatabaseCoreBackend::QueryOperationStatus status)
 {
     QMutexLocker l(&errorLockMutex);
@@ -530,7 +459,7 @@ bool DatabaseCoreBackendPrivate::handleWithErrorHandler(const SqlQuery* const qu
         }
         else
         {
-            kError() << "Failed to invoke DatabaseErrorHandler. Aborting all queries.";
+            kWarning() << "Failed to invoke DatabaseErrorHandler. Aborting all queries.";
             operationStatus = DatabaseCoreBackend::AbortQueries;
         }

@@ -568,7 +497,88 @@ void DatabaseCoreBackendPrivate::connectionErrorAbortQueries()
     queryOperationWakeAll(DatabaseCoreBackend::AbortQueries);
 }

-// -----------------------------------------------------------------------------------------------
+// -----------------------------------------------------------------------------------------
+
+DatabaseCoreBackendPrivate::AbstractUnlocker::AbstractUnlocker(DatabaseCoreBackendPrivate* const d)
+    : count(0), d(d)
+{
+    // Why two mutexes? The main mutex is recursive and won't work with a condvar.
+
+    // acquire lock
+    d->lock->mutex.lock();
+    // store lock count
+    count = d->lock->lockCount;
+    // set lock count to 0
+    d->lock->lockCount = 0;
+
+    // unlock
+    for (int i=0; i<count; ++i)
+    {
+        d->lock->mutex.unlock();
+    }
+}
+
+void DatabaseCoreBackendPrivate::AbstractUnlocker::finishAcquire()
+{
+    // drop lock acquired in first line. Main mutex is now free.
+    // We maintain lock order (first main mutex, second error lock mutex)
+    // but we drop main mutex lock for waiting on the cond var.
+    d->lock->mutex.unlock();
+}
+
+DatabaseCoreBackendPrivate::AbstractUnlocker::~AbstractUnlocker()
+{
+    // lock main mutex as often as it was locked before
+    for (int i=0; i<count; ++i)
+    {
+        d->lock->mutex.lock();
+    }
+
+    // update lock count
+    d->lock->lockCount += count;
+}
+
+// -----------------------------------------------------------------------------------------
+
+DatabaseCoreBackendPrivate::AbstractWaitingUnlocker::AbstractWaitingUnlocker(DatabaseCoreBackendPrivate* const d,
+        QMutex* const mutex, QWaitCondition* const condVar)
+    : AbstractUnlocker(d), mutex(mutex), condVar(condVar)
+{
+    // Why two mutexes? The main mutex is recursive and won't work with a condvar.
+    // lock condvar mutex (lock only if main mutex is locked)
+    mutex->lock();
+
+    finishAcquire();
+}
+
+DatabaseCoreBackendPrivate::AbstractWaitingUnlocker::~AbstractWaitingUnlocker()
+{
+    // unlock condvar mutex. Both mutexes are now free.
+    mutex->unlock();
+    // now base class destructor is executed, reallocating main mutex
+}
+
+bool DatabaseCoreBackendPrivate::AbstractWaitingUnlocker::wait(unsigned long time)
+{
+    return condVar->wait(mutex, time);
+}
+
+// -----------------------------------------------------------------------------------------
+
+/** This suspends the current thread if the query status as
+ *  set by setFlag() is Wait and until the thread is woken with wakeAll().
+ *  The DatabaseAccess mutex will be unlocked while waiting.
+ */
+void DatabaseCoreBackendPrivate::ErrorLocker::wait()
+{
+    // we use a copy of the flag under lock of the errorLockMutex to be able to check it here
+    while (d->errorLockOperationStatus == DatabaseCoreBackend::Wait)
+    {
+        wait();
+    }
+}
+
+// -----------------------------------------------------------------------------------------

 DatabaseCoreBackend::DatabaseCoreBackend(const QString& backendName, DatabaseLocking* const locking)
     : d_ptr(new DatabaseCoreBackendPrivate(this))
@@ -602,33 +612,32 @@ DatabaseAction DatabaseCoreBackend::getDBAction(const QString& actionName) const

     if (action.name.isNull())
     {
-        kError() << "No DB action defined for" << actionName << "! Implementation missing for this database type.";
+        kWarning() << "No DB action defined for" << actionName << "! Implementation missing for this database type.";
     }

     return action;
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execDBAction(const DatabaseAction& action, QList<QVariant>* const values,
-        QVariant* const lastInsertId)
+                                                                  QVariant* const lastInsertId)
 {
     return execDBAction(action, QMap<QString, QVariant>(), values, lastInsertId);
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execDBAction(const QString& action, QList<QVariant>* const values,
-        QVariant* const lastInsertId)
+                                                                  QVariant* const lastInsertId)
 {
     return execDBAction(getDBAction(action), QMap<QString, QVariant>(), values, lastInsertId);
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execDBAction(const QString& action, const QMap<QString, QVariant>& bindingMap,
-        QList<QVariant>* const values, QVariant* const lastInsertId)
+                                                                  QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     return execDBAction(getDBAction(action), bindingMap, values, lastInsertId);
 }

-DatabaseCoreBackend::QueryState DatabaseCoreBackend::execDBAction(const DatabaseAction& action, const QMap<QString,
-                                                                  QVariant>& bindingMap, QList<QVariant>* const values,
-                                                                  QVariant* const lastInsertId)
+DatabaseCoreBackend::QueryState DatabaseCoreBackend::execDBAction(const DatabaseAction& action, const QMap<QString, QVariant>& bindingMap,
+                                                                  QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     Q_D(DatabaseCoreBackend);

@@ -637,7 +646,7 @@ DatabaseCoreBackend::QueryState DatabaseCoreBackend::execDBAction(const Database

     if (action.name.isNull())
     {
-        kError() << "Attempt to execute null action";
+        kWarning() << "Attempt to execute null action";
         return DatabaseCoreBackend::SQLError;
     }

@@ -726,8 +735,8 @@ QSqlQuery DatabaseCoreBackend::execDBActionQuery(const DatabaseAction& action, c

         if (result.lastError().isValid() && result.lastError().number())
         {
-            kDebug() << "Error while executing DBAction ["<<  action.name
-                     <<"] Statement ["<<actionElement.statement<<"] Errornr. [" << result.lastError() << "]";
+            kDebug() << "Error while executing DBAction [" <<  action.name
+                     << "] Statement [" << actionElement.statement << "] Errornr. [" << result.lastError() << "]";
             break;
         }
     }
@@ -822,7 +831,7 @@ DatabaseCoreBackend::Status DatabaseCoreBackend::status() const
 }

 /*
-bool DatabaseCoreBackend::execSql(const QString& sql, QStringList* values)
+bool DatabaseCoreBackend::execSql(const QString& sql, QStringList* const values)
 {
     QSqlQuery query = execQuery(sql);

@@ -865,8 +874,7 @@ QList<QVariant> DatabaseCoreBackend::readToList(SqlQuery& query)
     return list;
 }

-DatabaseCoreBackend::QueryState DatabaseCoreBackend::handleQueryResult(SqlQuery& query, QList<QVariant>* const values,
-                                                                       QVariant* const lastInsertId)
+DatabaseCoreBackend::QueryState DatabaseCoreBackend::handleQueryResult(SqlQuery& query, QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     if (!query.isActive())
     {
@@ -891,57 +899,54 @@ DatabaseCoreBackend::QueryState DatabaseCoreBackend::handleQueryResult(SqlQuery&

 // -------------------------------------------------------------------------------------

-DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql, QList<QVariant>* const values,
-                                                             QVariant* const lastInsertId)
+DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql, QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     SqlQuery query = execQuery(sql);
     return handleQueryResult(query, values, lastInsertId);
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql, const QVariant& boundValue1,
-        QList<QVariant>* values, QVariant* lastInsertId)
+                                                             QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     SqlQuery query = execQuery(sql, boundValue1);
     return handleQueryResult(query, values, lastInsertId);
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql,
-        const QVariant& boundValue1, const QVariant& boundValue2,
-        QList<QVariant>* const values, QVariant* const lastInsertId)
+                                                             const QVariant& boundValue1, const QVariant& boundValue2,
+                                                             QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     SqlQuery query = execQuery(sql, boundValue1, boundValue2);
     return handleQueryResult(query, values, lastInsertId);
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql,
-        const QVariant& boundValue1, const QVariant& boundValue2,
-        const QVariant& boundValue3, QList<QVariant>* const values,
-        QVariant* const lastInsertId)
+                                                             const QVariant& boundValue1, const QVariant& boundValue2,
+                                                             const QVariant& boundValue3, QList<QVariant>* const values,
+                                                             QVariant* const lastInsertId)
 {
     SqlQuery query = execQuery(sql, boundValue1, boundValue2, boundValue3);
     return handleQueryResult(query, values, lastInsertId);
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql,
-        const QVariant& boundValue1, const QVariant& boundValue2,
-        const QVariant& boundValue3, const QVariant& boundValue4,
-        QList<QVariant>* const values, QVariant* const lastInsertId)
+                                                             const QVariant& boundValue1, const QVariant& boundValue2,
+                                                             const QVariant& boundValue3, const QVariant& boundValue4,
+                                                             QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     SqlQuery query = execQuery(sql, boundValue1, boundValue2, boundValue3, boundValue4);
     return handleQueryResult(query, values, lastInsertId);
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql, const QList<QVariant>& boundValues,
-        QList<QVariant>* const values, QVariant* const lastInsertId)
+                                                             QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     SqlQuery query = execQuery(sql, boundValues);
     return handleQueryResult(query, values, lastInsertId);
 }

-// -------------------------------------------------------------------------------------
-
 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql, const QMap<QString, QVariant>& bindingMap,
-        QList<QVariant>* const values, QVariant* const lastInsertId)
+                                                             QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     SqlQuery query = execQuery(sql, bindingMap);
     return handleQueryResult(query, values, lastInsertId);
@@ -949,8 +954,7 @@ DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(const QString& sql,

 // -------------------------------------------------------------------------------------

-DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(SqlQuery& preparedQuery,
-                                                             QList<QVariant>* const values, QVariant* const lastInsertId)
+DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(SqlQuery& preparedQuery, QList<QVariant>* const values, QVariant* const lastInsertId)
 {
     exec(preparedQuery);
     return handleQueryResult(preparedQuery, values, lastInsertId);
@@ -996,7 +1000,7 @@ DatabaseCoreBackend::QueryState DatabaseCoreBackend::execSql(SqlQuery& preparedQ
     return handleQueryResult(preparedQuery, values, lastInsertId);
 }

-// ----------------------------------------------------------------------------------------
+// -------------------------------------------------------------------------------------

 SqlQuery DatabaseCoreBackend::execQuery(const QString& sql, const QVariant& boundValue1)
 {
@@ -1119,9 +1123,9 @@ SqlQuery DatabaseCoreBackend::execQuery(const QString& sql, const QMap<QString,

             if (!bindingMap.contains(namedPlaceholder))
             {
-                kError() << "Missing place holder" << namedPlaceholder
-                         << "in binding map. The following values are defined for this action:"
-                         << bindingMap.keys() <<". This is a setup error!";
+                kWarning() << "Missing place holder" << namedPlaceholder
+                           << "in binding map. The following values are defined for this action:"
+                           << bindingMap.keys() <<". This is a setup error!";
                 //TODO What should we do here? How can we cancel that action?
             }

@@ -1240,7 +1244,7 @@ SqlQuery DatabaseCoreBackend::execQuery(const QString& sql, const QMap<QString,

     SqlQuery query = prepareQuery(preparedString);

-    for (int i=0; i<valuesToBind.size(); ++i)
+    for (int i=0; i < valuesToBind.size(); ++i)
     {
         query.bindValue(i, valuesToBind.at(i));
     }
@@ -1250,7 +1254,7 @@ SqlQuery DatabaseCoreBackend::execQuery(const QString& sql, const QMap<QString,
 }

 DatabaseCoreBackend::QueryState DatabaseCoreBackend::execUpsertDBAction(const DatabaseAction& action, const QVariant& id,
-        const QStringList fieldNames, const QList<QVariant>& values)
+                                                                        const QStringList fieldNames, const QList<QVariant>& values)
 {
     QMap<QString, QVariant> parameters;
     QMap<QString, QVariant> fieldValueMap;
@@ -1266,7 +1270,7 @@ DatabaseCoreBackend::QueryState DatabaseCoreBackend::execUpsertDBAction(const Da

     parameters.insert(":id",             id);
     parameters.insert(":fieldValueList", qVariantFromValue(fieldValueList));
-    parameters.insert(":fieldList",      qVariantFromValue (fieldList));
+    parameters.insert(":fieldList",      qVariantFromValue(fieldList));
     parameters.insert(":valueList",      qVariantFromValue(valueList));

     return execDBAction(action, parameters);
diff --git a/libs/database/core/databasecorebackend.h b/libs/database/core/databasecorebackend.h
index fa4dba4..ea8d558 100644
--- a/libs/database/core/databasecorebackend.h
+++ b/libs/database/core/databasecorebackend.h
@@ -42,15 +42,12 @@
 namespace Digikam
 {

-class ThumbnailSchemaUpdater;
-class DatabaseErrorHandler;
 class DatabaseCoreBackendPrivate;
-
-// ------------------------------------------------------------------------------------------------------------
+class DatabaseErrorHandler;
+class ThumbnailSchemaUpdater;

 class DIGIKAM_EXPORT DatabaseLocking
 {
-
 public:

     DatabaseLocking();
@@ -61,7 +58,7 @@ public:
     int    lockCount;
 };

-// ------------------------------------------------------------------------------------------------------------
+// -----------------------------------------------------------------

 class DIGIKAM_EXPORT DatabaseCoreBackend : public QObject
 {
@@ -202,7 +199,7 @@ public:
      * Add a DatabaseErrorHandler. This object must be created in the main thread.
      * If a database error occurs, this object can handle problem solving and user interaction.
      */
-    void setDatabaseErrorHandler(DatabaseErrorHandler* handler);
+    void setDatabaseErrorHandler(DatabaseErrorHandler* const handler);

     /**
       * Return config read from XML,
@@ -265,10 +262,10 @@ public:
      * Executes the SQL statement, and write the returned data into the values list.
      * If you are not interested in the returned data, set values to 0.
      * Methods are provided for up to four bound values (positional binding), or for a list of bound values.
-     * If you want the last inserted id (and your query is suitable), sett lastInsertId to the address of a QVariant.
+     * If you want the last inserted id (and your query is suitable), set lastInsertId to the address of a QVariant.
      * Additionally, methods are provided for prepared statements.
      */
-    QueryState execSql(const QString& sql, QList<QVariant>* values = 0, QVariant* const lastInsertId = 0);
+    QueryState execSql(const QString& sql, QList<QVariant>* const values = 0, QVariant* const lastInsertId = 0);
     QueryState execSql(const QString& sql, const QVariant& boundValue1,
                        QList<QVariant>* const values = 0, QVariant* const lastInsertId = 0);
     QueryState execSql(const QString& sql,
diff --git a/libs/database/core/databasecorebackend_p.h b/libs/database/core/databasecorebackend_p.h
index ff3a3fa..1f60dd7 100644
--- a/libs/database/core/databasecorebackend_p.h
+++ b/libs/database/core/databasecorebackend_p.h
@@ -75,7 +75,6 @@ public:
     void closeDatabaseForThread();
     bool incrementTransactionCount();
     bool decrementTransactionCount();
-    bool isInTransactionInOtherThread() const;

     bool isInMainThread() const;
     bool isInUIThread()   const;
@@ -145,10 +144,10 @@ public :
         DatabaseCoreBackendPrivate* const d;
     };

-    // ----------------------------------------------------------------------
-
     friend class AbstractUnlocker;

+    // ------------------------------------------------------------------
+
     class AbstractWaitingUnlocker : public AbstractUnlocker
     {
     public:
@@ -164,7 +163,7 @@ public :
         QWaitCondition* const condVar;
     };

-    // ----------------------------------------------------------------------
+    // ------------------------------------------------------------------

     class ErrorLocker : public AbstractWaitingUnlocker
     {
@@ -174,7 +173,7 @@ public :
         void wait();
     };

-    // ----------------------------------------------------------------------
+    // ------------------------------------------------------------------

     class BusyWaiter : public AbstractWaitingUnlocker
     {
Comment 104 caulier.gilles 2014-12-15 12:59:35 UTC
Next digiKam release 4.6.0 will include several important commits from Marcel to try to fix this memory leak with SQlite database

Gilles Caulier
Comment 105 Jean-Martial NDOUTOUME NFENGONE 2015-03-16 13:07:21 UTC
Hello guys,

Very nice libre software.  Great job.

Every time I play face tagging, my digiKam 4.8 release became extremely unresponsive and slow.

What about the work in progress of this bug?

Jean-Martial
Comment 106 caulier.gilles 2015-03-25 08:55:14 UTC
I think bug is fixed with current implementation from git/master (next 4.9.0). See bug #338176 for details.

I close this file now. Don't hesitate to re-open if necessary...

Gilles Caulier
Comment 107 Mick Sulley 2015-04-08 21:30:01 UTC
Just upgraded to DK4.9.0 on Mint and the problem is still there.  I select People > Unknown and select a name for one of the unknowns and when I select a name the memory usage starts to climb.  Closing DigiKam does not clear the problem as a DigiKam process is still running and must be killed to stop the memory runaway.
Comment 108 Maik Qualmann 2015-04-09 06:02:54 UTC
Have you updated libkface?

Maik
Comment 109 Mick Sulley 2015-04-09 08:29:11 UTC
When I looked I had
libkface2 version 1.0~digikam4.4.0-trusty~ppa2
libkface3 version 1.0~digikam4.9.0-trusty~ppa1
installed.  I uninstalled libkface2 and rebooted but the problem is still the same.
Is there anything else I can do?
Comment 110 caulier.gilles 2015-04-09 08:39:00 UTC
Not sure that you have last libkface version installed.

At least, 3.5.0 ID must be visible in list given on Help/Components Info dialog.

Gilles Caulier
Comment 111 Mick Sulley 2015-04-09 08:43:54 UTC
LibKface 3.5.0 is in the list.

The full list is -
digiKam version 4.9.0
CPU cores: 8
Demosaic GPL2 pack support: Unknown
Demosaic GPL3 pack support: Unknown
Exiv2 can write to Jp2: Yes
Exiv2 can write to Jpeg: Yes
Exiv2 can write to Pgf: Yes
Exiv2 can write to Png: Yes
Exiv2 can write to Tiff: Yes
Exiv2 supports XMP metadata: Yes
LibCImg: 130
LibEigen: 3.2.0
LibExiv2: 0.24
LibJPEG: 80
LibJasper: 1.900.1
LibKDE: 4.13.3
LibKExiv2: 2.4.0
LibKGeoMap: 3.1.0
LibKdcraw: 2.4.2
LibLCMS: 2050
LibLensFun: 0.2.8-0
LibLqr support: yes
LibPGF: 6.12.24
LibPNG: 1.2.50
LibQt: 4.8.6
LibRaw: 0.15.4
LibTIFF: LIBTIFF, Version 4.0.3 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc.
Marble Widget: 0.18.3 (stable release)
Parallelised demosaicing: Unknown
RawSpeed codec support: Unknown
Baloo support: no
Database backend: QSQLITE
Kdepimlibs support: Yes
Kipi-Plugins: 4.9.0
LibGphoto2: 2.5.3.1
LibKface: 3.5.0
LibKipi: 2.2.0
LibOpenCV: 2.4.9
Sqlite2 support: Yes
Comment 112 caulier.gilles 2015-04-09 08:47:37 UTC
It's also possible that your face database file is corrupted due to previous dysfunctions.

The idea is to rename file as .old and to re-try to scan face with a fresh DB file.

file is located in your home dir at ~/.kde4/share/apps/libkface/database/recognition.db

Gilles Caulier
Comment 113 Mick Sulley 2015-04-09 09:02:19 UTC
Just renamed the database, tried it again and it works!!!!!  Thank you so much for your help and for fixing this problem. DK is brilliant and this was the only thing I have found which did not work.

Thank you
Mick