Bug 195610

To track down a possible memory leak, we need to know more details.
What have you done exactly?
Do you use some filters over and over again? Which ones?
I have checked the editor, it doesn't leak when just opening images, so it must be some filter.

Jean-Marc,

please try to reproduce your workflow and tell us which filters you used (it doesn't have to be the exact order of course).

Andi

I am working on a set of 230 JPEG images from a Canon Eos 50D. I work sequentially, one image after the other, without leaving the image editor. In most cases, the actions are : crop, curves, refocus - in that order. In maybe 15% of the cases, I correct for white balance or saturation, but it is not typical because this set is mostly outdoor portraits.

The out of memory kills have if I remember well always happened during refocus - so you might want to put that one on the top of your list of suspects !

To be sure, run digiKam in valgrind as explained here :

http://www.digikam.org/contrib

Note : digiKam will be slower in this case but source code line where memory is leak, will be printed on the console. digiKam must be compiled with debug symbol.

Gilles Caulier

In the next days, I'll try using Valgrind. I'll keep you posted.

There seems to be a memory leak in the refocus plugin:

==6849== 1,936 bytes in 2 blocks are definitely lost in loss record 473 of 491
==6849==    at 0x402441D: operator new[](unsigned int) (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==6849==    by 0x410A3AF: Digikam::RefocusMatrix::init_c_mat(Digikam::CMat*, int) (matrix.cpp:102)
==6849==    by 0x410A45D: Digikam::RefocusMatrix::allocate_c_mat(int) (matrix.cpp:111)
==6849==    by 0x410B214: Digikam::RefocusMatrix::copy_cvec2mat(Digikam::Mat const*, int) (matrix.cpp:360)
==6849==    by 0x410B518: Digikam::RefocusMatrix::compute_g(Digikam::CMat const*, int, double, double, double, bool) (matrix.cpp:431)
==6849==    by 0x410B5E2: Digikam::RefocusMatrix::compute_g_matrix(Digikam::CMat const*, int, double, double, double, bool) (matrix.cpp:461)
==6849==    by 0x4109840: Digikam::DImgRefocus::refocusImage(unsigned char*, int, int, bool, int, double, double, double, double) (dimgrefocus.cpp:153)
==6849==    by 0x4109625: Digikam::DImgRefocus::filterImage() (dimgrefocus.cpp:131)
==6849==    by 0x4107EF6: Digikam::DImgThreadedFilter::startFilterDirectly() (dimgthreadedfilter.cpp:139)
==6849==    by 0x4107FFB: Digikam::DImgThreadedFilter::run() (dimgthreadedfilter.cpp:152)
==6849==    by 0x50692CD: (within /usr/lib/libQtCore.so.4.5.1)
==6849==    by 0x52786BB: start_thread (in /lib/libpthread-2.10.1.so)


It doesn't seem to waste that much memory, but when you never close the editor and work with rather big images, it might get important.

Gilles,
any idea what is wrong? For me it looks like the memory gets freed again, but I might have overlooked something.

Andi

Ah... i know this memory leak. It small.

It come from Refocus algorithm taken from gimp plugin. It's a pure math code... good luck .I never found why memory is leak here...

Perhaps a better expert than me can found it... Marcel ?

Gilles Caulier

Another (small) memory leak:
imagecurves.cpp:422

Although we seem to delete it properly in there and in the destructor, it leaks a little bit.
Any idea?

Andi

Look line 142 :     d->lut->luts      = NULL;

NULL is assigned to luts, but if this one as already values, we have memory leak...

Gilles Caulier

Jean-Marc,

I have fixed some memory leak in the imagescurves:

commit 1c8d6a6ec0ca24490b0b96a9e7e51cfe42159196
Author: aclemens <aclemens@283d02a7-25f6-0310-bc7c-ecb5cbfe19da>
Date:   Mon Jun 15 13:21:37 2009 +0000

    fix memory leak in ImageCurves

    git-svn-id: https://svn.kde.org/home/kde/trunk/extragear/graphics/digikam@982291 283d02a7-25f6-0310-bc7c-ecb5cbfe19da

libs/curves/imagecurves.cpp
libs/curves/imagecurves.h


Since your workflow also includes curves adjustment, it might be one of the sources for your memory leak problem.
As I said above, I found another one in the refocus plugin, but unfortunately can't see the actual error. The leak seems to be small though, but when you edit over 200 images, it still might be important.

Can you try with the latest SVN version?

I'm experiencing this bug too - after editing like 10-20 images, digikam uses 1GB ram for no apparent reason... No idea yet, whether the source is the image editor, or the digikam itself - however it made my computer freeze (for out-of-memory reasons, obviously) a couple of times already, so it's quite annoying...

After trying valgrind, the non-released memory was 13 MB - after doing some common editing of 3 files.. This seems to not be the problem cause (as that should be more like 50 MB or more).

Therefore, either I didn't reproduce the bug with valgrind being used (yet.. digikam is veeeery lazy when valgrinded), or the occupied memory is correctly released when quitting digikam - but is purposelessly allocated after images are closed. 

I hope I'll find the cause - it's pretty uncomfortable to close digikam after every few edits... :)

Again, what filter did you use?
I can not experience this, so there must be a problem in a specific imageplugin.

What filetypes are you editing?
What dimensions do they have?
Is your system 32bit or 64bit?

Andi

(In reply to comment #10)
> After trying valgrind, the non-released memory was 13 MB - after doing some
> common editing of 3 files.. This seems to not be the problem cause (as that
> should be more like 50 MB or more).

The 13 MB can come from an unclosed sqlite connection, this typically wastes 8-10 MB on my system...

I only use the most common ones in digikam (though things like layers, adding vignetting, clone/heal brush would be nice :)) - curves, crop, hue-value-saturation and less occassionally bw-convertor and resize... that's about that - editing jpegs, 12Mpix (around 4500x3000), 32bit..

currently I'm using the 0.10.0b3 (ubuntu), but the bug seems to be there ever since 0.10.0 (I don't remember having such problems with 0.9.x, but I was mostly using gimp to postprocess photos back then)..

maybe it's qt-related or something - it's hard to tell, when there only are two people who reported this :)

here the same.. I have e.g. an album with 300 images.. with image editor I optimized a lot of images with ctrl+shift+b; alt+o and ctrl+s After some time digikam and xorg fills my memory and swap :(

sts,

Sound like you didn't use editor plugin here, just you show image as well.

Can you run digiKam through valgrind to show memory leak, and reproduce all step in gui until to see memory dysnfuntion appear. Just report all console messages there. Command line to use is below :

valgrind --tool=memcheck --leak-check=full --error-limit=no digikam

Also, give us more info about your linux box...

Gilles Caulier

I can confirm this for a long time, and it doesn't seem to be a memory leak, at least in the classical term. Valgrind will show nothing, since at digiKam shutdown all memory is freed, but at runtime it is not. I guess this is due to the fact that IE stay open in the background once used?
Every 2 or 3 weeks I look at this bugreport here and try to find the error, but I don't know what is going wrong.
Memory usage goes up, but valgrind tells me that all is fine.
This one is a nasty bug for sure...

Is the memory listed for the digikam or the xorg process?
There is valgrind's "massif" tool to analyze heap allocation over time. If it detects e.g. a peak in memory usage it will give you backtraces from where significant portions of this memory were allocated.

Ok, I'm just trying to figure this out. Most of the memory allocation happens in DImg::allocate which isn't a big surprise.

But I think I got a suspicion what's going on here:
1. Simply viewing images without manipulations is ok, then all memory is cleaned properly.
2. Doing any operation but not saving the image also results in correctly freed memory.
3. Doing any operation and saving the changed image results in more memory use. So there must be something wrong while saving images.

Created attachment 40204 [details]
Massif log file

Massif log file while opening several images in the editor, cropping them a bit, saving them and moving on to the next image.

How large were the edited images (pixels and bit depth)?

Tiff files with 8 bit. All one side > 4000px.

I discovered different things (but unfortunately I can't remember it correctly now). It was something like:

1. Editing TIFFs : ok
2. Just viewing TIFFs: memory usage was duplicated all the time
3. As soon as I edited a TIFF again: mem was freed again

In the last snapshot from the Massif output I see 3*125 MB of memory. With the size you indicated 125MB can be two full images in memory. Two times from the histogram sidebar - once from the last cropping operation.

I just tried viewing a few TIFFs under Massif. I closed the image editor window at the end and the last heap snapshot also showed a considerable decline in memory usage. I could not identify any leaks here.

SVN commit 1080155 by mwiesweg:

Reset histogram image data when url is null.
Should probably also be done when switching to other tab.

CCBUG: 195610

 M  +2 -0      imagepropertiescolorstab.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1080155

Hm, maybe I should note that I did not use the sidebar at all during the massif snapshot. It was always closed.

Recently, while editing photos not only for testing, I suddenly had digikam filling more than 2 GB RAM, also without using any of the sidebars except the tools itself.

This entry still valid with 2.x serie ?

Gilles Caulier

Jean Marc, 

This file still valid using digiKam 2.4 ?

Gilles Caulier

New digiKam 4.11.0 is available :

https://www.digikam.org/node/740

Can you reproduce the problem with this release ?

This problem is not reproducible with last 5.0.0.
I close this file now. Don't hesitate to re-open it if necessary.
Gilles Caulier