Bug 408793 - Memory leak when .psd file in album folder
Summary: Memory leak when .psd file in album folder
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: DImg-Core (other bugs)
Version First Reported In: 6.1.0
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-06-16 17:27 UTC by Michael Born
Modified: 2019-06-18 04:28 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed/Implemented In: 6.2.0
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Born 2019-06-16 17:27:44 UTC
SUMMARY

Digikam takes (over about 5 minutes) all my 16GB memory. Then, the computer freezes.
Removing the album folder with the .psd files (from a CD of the book [1]) lets digikam work normally (no memory leak).

Having some .psd files in my album path triggers digicam to:
digikam.general: Detected change, triggering rescan of "/home/michael/Bilder/"
digikam.database: Finishing took 4 ms
digikam.dimg: "/home/michael/Bilder/Photoshop_LAB/chapter_01/artists_palette.psd"  : QIMAGE file identified
digikam.dimg.qimage: Can not load " "/home/michael/Bilder/Photoshop_LAB/chapter_01/artists_palette.psd" " using DImg::QImageLoader!
digikam.dimg.qimage: Error message from loader: "Die Bilddaten konnten nicht gelesen werden"
digikam.general: Detected change, triggering rescan of "/home/michael/Bilder/"
digikam.geoiface: ----
digikam.database: Scanning took 277 ms
digikam.database: Finishing took 17 ms
...

copying back just one .psd file gets digikam to leak memory:
...
digikam.database: Finishing took 5 ms
digikam.general: Detected change, triggering rescan of "/home/michael/Bilder/"
digikam.dimg: "/home/michael/Bilder/yellowstone_canyon.psd"  : QIMAGE file identified
digikam.dimg.qimage: Can not load " "/home/michael/Bilder/yellowstone_canyon.psd" " using DImg::QImageLoader!
digikam.dimg.qimage: Error message from loader: "Die Bilddaten konnten nicht gelesen werden"
digikam.database: Starting scan!
digikam.database: Scanning took 350 ms
digikam.general: Detected change, triggering rescan of "/home/michael/Bilder/"
digikam.database: Finishing took 5 ms
digikam.general: Detected change, triggering rescan of "/home/michael/Bilder/"
digikam.dimg: "/home/michael/Bilder/yellowstone_canyon.psd"  : QIMAGE file identified
digikam.dimg.qimage: Can not load " "/home/michael/Bilder/yellowstone_canyon.psd" " using DImg::QImageLoader!
digikam.dimg.qimage: Error message from loader: "Die Bilddaten konnten nicht gelesen werden"
digikam.database: Starting scan!
digikam.database: Scanning took 349 ms
digikam.general: Detected change, triggering rescan of "/home/michael/Bilder/"
digikam.database: Finishing took 6 ms
digikam.general: Detected change, triggering rescan of "/home/michael/Bilder/"
digikam.dimg: "/home/michael/Bilder/yellowstone_canyon.psd"  : QIMAGE file identified
digikam.dimg.qimage: Can not load " "/home/michael/Bilder/yellowstone_canyon.psd" " using DImg::QImageLoader!
digikam.dimg.qimage: Error message from loader: "Die Bilddaten konnten nicht gelesen werden"
digikam.database: Starting scan!
digikam.database: Scanning took 349 ms
digikam.general: Detected change, triggering rescan of "/home/michael/Bilder/"
digikam.database: Finishing took 7 ms
digikam.general: Detected change, triggering rescan of "/home/michael/Bilder/"
...

removing the file again lets digikam calm down.


STEPS TO REPRODUCE
1. have .psd files in the album folder
2. just start digikam
3. wait for the computer to freeze (about 5 minutes in my case)

OBSERVED RESULT
Running top in another konsole shows digikam using about 100% cpu and steadily increasing amount of memory. At about 88% memory the system freezes and has to be rebooted.

EXPECTED RESULT
Don't eat my cpu/memory.

SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: 5.16.0 Opensuse Tumbleweed current version
(available in About System)
KDE Plasma Version: 5.16.0
KDE Frameworks Version: 5.58.0
Qt Version: 5.12.3

ADDITIONAL INFORMATION
[1] Photoshop Lab Color: The Canyon Conundrum and Other Adventures in the Most Powerful Colorspace
by Dan Margulis

Additional note:
The .psd files live there for many years. Only the digikam update that came from Tumbleweed (some?) week ago started this behavior.
Comment 1 Maik Qualmann 2019-06-16 18:49:22 UTC
I rather suspect the problem in the kimageformats package. Can you uninstall it and test if the problem is still to reproduce?

Maik
Comment 2 caulier.gilles 2019-06-16 18:52:11 UTC
Note : With next 6.2.0, ImageMagick will be used instead to render PSD files.

You can test with current devel 6.2.0 pre release AppImage bundle :

https://files.kde.org/digikam/

Gilles Caulier.
Comment 3 Michael Born 2019-06-16 19:20:43 UTC
(In reply to Maik Qualmann from comment #1)
> I rather suspect the problem in the kimageformats package. Can you uninstall
> it and test if the problem is still to reproduce?
> 
> Maik

Thank you for the suggestion.
I upgraded and then downgraded again the kimageformats package from v5.58.0-1.3 (Tumbleweed OSS repo) to 5.59.0-175.1 (KDE5 repo). But, the problem staied the same.
I put the .psd file in the album folder and immediately cpu->100% and mem% increases. It stops when I delete the .psd file.

But, I found that the packages 
digikam-plugins and libdigikamcore6
were from a KDE:Extra repo instead of the TW OSS one. Changing to TW OSS did not change the problem.

Any other suggestion?

I will try the 6.2.0 when it gets shipped with my TW rolling release.
Comment 4 caulier.gilles 2019-06-16 20:22:09 UTC
You don't need to wait DK 6.2.0 for your Linux box. AppImage is a linux universal bundle that you can run as well. Nothing is installed. All run in memory in a virtual disk. It's safe and permit to check if your problem will be resolved with next release.

Just download the appimage file that we provide for you, turn on as executable file, and start it as well.

Gilles Caulier
Comment 5 Michael Born 2019-06-16 20:31:05 UTC
(In reply to Maik Qualmann from comment #1)
> I rather suspect the problem in the kimageformats package. Can you uninstall
> it and test if the problem is still to reproduce?
> 
> Maik

Fierst, I did not expect to be able to uninstall kimageformats (because of dependencies).
But, it could be done.

Then, when copying the .psd file into the album, digikam started rescanning and reported "unsupported file format" (in German "Dieser Typ von Bilddaten wird nicht unterstützt")
This repeated several 10 times. But then it stopped and digikam mem usage stopped growing at 5% and cpu load is at 2% with the following terminal output:

...
digikam.database: Starting scan!
digikam.database: Scanning took 348 ms
digikam.general: Detected change, triggering rescan of "/home/michael/Bilder/"
digikam.database: Finishing took 6 ms
digikam.general: Detected change, triggering rescan of "/home/michael/Bilder/"
digikam.dimg: "/home/michael/Bilder/yellowstone_canyon.psd"  : QIMAGE file identified
digikam.dimg.qimage: Can not load " "/home/michael/Bilder/yellowstone_canyon.psd" " using DImg::QImageLoader!
digikam.dimg.qimage: Error message from loader: "Dieser Typ von Bilddaten wird nicht unterstützt"
digikam.database: Starting scan!
digikam.database: Scanning took 352 ms
digikam.database: Finishing took 5 ms
digikam.general: Using  4  CPU core to run threads
digikam.general: Action Thread run  1  new jobs
digikam.general: One job is done
digikam.general: Cancel Main Thread
Comment 6 Maik Qualmann 2019-06-16 20:36:39 UTC
Can you provide PSD test images? I also use openSUSE Tumbleweed. If not public, also to my e-mail.

Maik
Comment 7 Maik Qualmann 2019-06-16 20:40:35 UTC
The kimageformats package can be uninstalled without any problems with dependencies.

Maik
Comment 8 Maik Qualmann 2019-06-16 21:27:39 UTC
Thanks for the test image. I have here to test the ImageMagick loader disabled to get a similar result to digiKam-6.1.0. We have different error messages from the QImage loader, you message is that the file type is not supported. My message is, an error has occurred and the memory could not be assigned. I can not reproduce a memory leak. Gimp can not open the file and the ImageMagick loader of digiKam-6.2.0 shows only an empty alpha thumbnail. I will test it tomorrow.

Maik
Comment 9 Michael Born 2019-06-17 06:48:46 UTC
(In reply to Maik Qualmann from comment #8)
> Thanks for the test image. I have here to test the ImageMagick loader
> disabled to get a similar result to digiKam-6.1.0. We have different error
> messages from the QImage loader, you message is that the file type is not
> supported. My message is, an error has occurred and the memory could not be
> assigned. I can not reproduce a memory leak. Gimp can not open the file and
> the ImageMagick loader of digiKam-6.2.0 shows only an empty alpha thumbnail.
> I will test it tomorrow.
> 
> Maik

Just to clarify.
When I uninstalled kimageformats (comment 5), the message was "unsupported file format" (in German "Dieser Typ von Bilddaten wird nicht unterstützt").

When I had kimageformats installed (Bug description), the message was "picture data could not be read" (in German "Die Bilddaten konnten nicht gelesen werden").

I will check tonight that the .psd file is identical to the book CD (and not corrupted afterwards).
Comment 10 Maik Qualmann 2019-06-18 04:28:09 UTC
Git commit 54fb97abfd11279e36f7b25219ffa48211e0cb4c by Maik Qualmann.
Committed on 18/06/2019 at 04:26.
Pushed by mqualmann into branch 'master'.

fix memory leak with QImage loader
FIXED-IN: 6.2.0

M  +2    -1    NEWS
M  +2    -0    core/libs/dimg/loaders/qimageloader.cpp

https://invent.kde.org/kde/digikam/commit/54fb97abfd11279e36f7b25219ffa48211e0cb4c