- Large memory leak running maintenance image quality sorter. (500Mb / 500 images) - Tags are not applied. There are several of these in the valigrind log, here's the largest: ==26674== 307,183,616 bytes in 107 blocks are possibly lost in loss record 42,037 of 42,037 ==26674== at 0x4C2AFE7: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==26674== by 0x73963F0: unsigned char* Digikam::DImgLoader::new_failureTolerant<unsigned char>(unsigned long) (dimgloader.h:183) ==26674== by 0x737362B: Digikam::DImg::allocateData() (dimg.cpp:315) ==26674== by 0x737616F: Digikam::DImg::DImg(Digikam::DImg const&, int, int) (dimg.cpp:136) ==26674== by 0x738C1C8: Digikam::DImg::smoothScaleClipped(int, int, int, int, int, int) const (dimgscale.cpp:233) ==26674== by 0x738C47B: Digikam::DImg::smoothScaleClipped(QSize const&, QRect const&) const (dimgscale.cpp:194) ==26674== by 0x738C514: Digikam::DImg::smoothScale(QSize const&, Qt::AspectRatioMode) const (dimgscale.cpp:183) ==26674== by 0x738C55F: Digikam::DImg::smoothScale(int, int, Qt::AspectRatioMode) const (dimgscale.cpp:188) ==26674== by 0x74D2D07: Digikam::PreviewLoadingTask::execute() (previewtask.cpp:433) ==26674== by 0x74D1783: Digikam::PreviewLoadThread::loadSynchronously(Digikam::LoadingDescription const&) (previewloadthread.cpp:114) ==26674== by 0x50E734: Digikam::ImageQualityTask::run() (imagequalitytask.cpp:96) Also: ==26674== at 0x4C2AFE7: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==26674== by 0x73809DD: unsigned int* Digikam::DImgLoader::new_failureTolerant<unsigned int>(unsigned long) (dimgloader.h:183) ==26674== by 0x7378128: Digikam::DImg::rotate(Digikam::DImg::ANGLE) (dimg.cpp:2298) ==26674== by 0x7378576: Digikam::DImg::rotateAndFlip(int) (dimg.cpp:2649) ==26674== by 0x74C581D: Digikam::LoadSaveThread::exifRotate(Digikam::DImg&, QString const&) (loadsavethread.cpp:335) ==26674== by 0x74D2D47: Digikam::PreviewLoadingTask::execute() (previewtask.cpp:439) ==26674== by 0x74D1783: Digikam::PreviewLoadThread::loadSynchronously(Digikam::LoadingDescription const&) (previewloadthread.cpp:114) ==26674== by 0x50E734: Digikam::ImageQualityTask::run() (imagequalitytask.cpp:96) Reproducible: Always Steps to Reproduce: 1. Run image quality sorter with library > 500. 2. Mark maintenance option to clear db and re-run for all Actual Results: 2. 500Mb memory leak 3. Tags not applied Expected Results: Digikam would validate my photography skills. or at least not leak memory and apply the good, bad, ok tags. 4.0.0-beta-3 built from git gitup current as of Jan 20th 2014. Ubuntu 13.10 (gcc 2.8.1) libraw from git 0.16.0
*** Bug 330227 has been marked as a duplicate of this bug. ***
Which Image Quality Sorter settings do you use exactly ? Gilles Caulier
Gowthan, This evening from IRC channel, a thread about this entry has give more important information: [23:10] <rhenders> I'm seeing some pretty big memory leaks in Image Quality sorter in 4.0.0-beta2: https://gist.github.com/bitflippersanonymous/8530225 [23:11] <rhenders> Built from source on Ubuntu/AMD64 [23:11] <rhenders> I looked for existing bugs but couldn't find any, thought I'd check here before filing [23:12] <teprrr> uhu, that looks leaky indeed :o [23:13] <teprrr> problem is here: d->imgqsort = new ImgQSort(dimg, d->quality, &pick); [23:13] <rhenders> It grows to 100GB for my collection of 5k pictures. [23:13] <rhenders> I could build debug and get line numbers, ... what file is that in? [23:13] <teprrr> most likely. run() is being called multiple times, and that new gets called all the time then [23:14] <teprrr> digikam/utilities/maintenance/imagequalitytask.cpp [23:14] <teprrr> line 110 [23:15] <teprrr> if that isn't destroyed somewhere in the result handler that's most likely the leaker [23:16] <teprrr> DImg there is the image data itself, which may be heavy :P [23:16] <rhenders> There are several leaks in that valgrind log. The last one is the largest, but they all look similar [23:16] <teprrr> but if you can, provide a valgrind log with the debug-info enabled and report it to the bugs.kde.org :p [23:16] <teprrr> hmm, related to maintenance tool? [23:17] <rhenders> I also see leaks when rebuilding thumbnails, but this was the largest. I'll rebuild in debug and report it [23:17] <rhenders> (What's the cmake flag to enable debug) [23:18] <teprrr> -DCMAKE_BUILD_TYPE=Debug [23:18] <teprrr> or DebugFull [23:18] <rhenders> Great, thanks [23:20] <teprrr> not a problem, I gotta go now away for a while. thanks for noticing such a bug :) Gilles Caulier
Just the defaults AFAIK Enabled Speed: 1 Detect Everything Assign Everything Reject threshold: 10 Pending threshold: 40 Accepted threshold: 60 Blur Weight: 100 Noise Weight: 100 Compress Weight: 100
Thank you teppr and Ryan. You are right. Image quality sorter data was not getting deleted after completion of task. Fixed it. Please note that Image quality sorter is still under development. The goal is to tag really bad pictures, not to assess the quality of photographs subjectively. As of now, we have to manually set the threshold values according to the type of pictures processed. Presently, we're trying to detect different cases of blurring in an image. We're also trying to detect the type of pictures so that the threshold values are automatically set.
Git commit df13317e5d9310f7d07943df8b1501d4706f9e17 by Gowtham Ashok. Committed on 22/01/2014 at 01:46. Pushed by gowthamashok into branch 'master'. Fixed imgqsort huge memory leak M +6 -2 utilities/maintenance/imagequalitytask.cpp http://commits.kde.org/digikam/df13317e5d9310f7d07943df8b1501d4706f9e17 diff --git a/utilities/maintenance/imagequalitytask.cpp b/utilities/maintenance/imagequalitytask.cpp index 1df7a23..eafa8d8 100644 --- a/utilities/maintenance/imagequalitytask.cpp +++ b/utilities/maintenance/imagequalitytask.cpp @@ -78,8 +78,11 @@ void ImageQualityTask::slotCancel() { d->cancel = true; - if (d->imgqsort) + //TODO: Investigate if commented code is necessary + /* if (d->imgqsort) d->imgqsort->cancelFilter(); + + */ } void ImageQualityTask::run() @@ -111,8 +114,9 @@ void ImageQualityTask::run() ImageInfo info(d->path); info.setPickLabel(pick); + if(d->imgqsort) + delete d->imgqsort; //delete image data after setting label } - // Dispatch progress to Progress Manager QImage qimg = dimg.smoothScale(22, 22, Qt::KeepAspectRatio).copyQImage(); emit signalFinished(qimg);