SUMMARY Digikam git beta 3 (digikam-git r43617) eats huge memory STEPS TO REPRODUCE tool -> maintenance -> update fingerprints Digikam eats about 4.5 Go memory. After 10 secunds, tmp/tmpfs is 100% full. All computer is nearly frozen, I have to kill Digikam from root tty console. When i try to restart Digikam, it crashes now Arch Aur Linux External mariadb db
I can not get out which version git r43617 is currently. Tested here with git/master are no problems. Memory usage fluctuates around 500KB. Maybe post a copy of the console, if possible. Maik
Ahh, I also see that your tmp is running full. This is due to the KImageFormat packet. it contains a broken ImageMagick Loader that generates this tmp data. As a workaround, remove this KImageFormat package and report it to Arch. OpenSuSE also added this ImageMagick loader to the KImageFormat package for a short time, but quickly removed it. It was an attempt to load XCF files. We could only prevent it by disabling our QImage Loader. Maik
Git commit b80a9bd1bfe12e24b046cb7b852a9b5a6db231d9 by Maik Qualmann. Committed on 23/08/2019 at 20:43. Pushed by mqualmann into branch 'master'. add black list to the QImage loader first mime type is XCF FIXED-IN: 6.3.0 M +2 -1 NEWS M +7 -3 core/libs/dimg/loaders/qimageloader.cpp https://invent.kde.org/kde/digikam/commit/b80a9bd1bfe12e24b046cb7b852a9b5a6db231d9
Note: compile digiKam with ImageMagick support. We have our own ImageMagick loader to support additional image formats. Although only the first layer is loaded with new Gimp XCF files, but you know that. Maik
(In reply to Maik Qualmann from comment #2) > Ahh, I also see that your tmp is running full. This is due to the > KImageFormat packet. it contains a broken ImageMagick Loader that generates > this tmp data. As a workaround, remove this KImageFormat package and report > it to Arch. OpenSuSE also added this ImageMagick loader to the KImageFormat > package for a short time, but quickly removed it. It was an attempt to load > XCF files. We could only prevent it by disabling our QImage Loader. > Imagemagick is not a dependency Arch Kimageformats package. See PKGBUILD here: https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/kimageformats - I removed Kimageformats - I recompiled Digikam from commit https://invent.kde.org/kde/digikam/commit/839921c96a98468d7a0e7a1cbaef6f161f7a1a37 It freezes computer because it eats whole memory, more than 5.5 Go. Steps to reproduce: Similarity -> duplicates -> update finger prints In console: digikam.general: Using 4 CPU core to run threads digikam.general: Creating a fingerprints task for generating fingerprints digikam.general: Creating a fingerprints task for generating fingerprints digikam.general: Creating a fingerprints task for generating fingerprints digikam.general: Creating a fingerprints task for generating fingerprints digikam.general: Action Thread run 4 new jobs digikam.general: Updating fingerprints for file: "/home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-1.xcf" digikam.general: Updating fingerprints for file: "/home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-2.xcf" digikam.general: Updating fingerprints for file: "/home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-3.xcf" digikam.general: Updating fingerprints for file: "/home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-4.xcf" digikam.metaengine: Cannot load metadata from file /home/mn/images/1perso/1Contacts/contacts-d�tails/c-d�tails-famille/1contact-PROVISOIRE/197808-3200-1.xcf (Error # 11 : /home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-1.xcf: The file contains data of an unknown image type digikam.dimg: "/home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-1.xcf" : Try to load with ImageMagick digikam.metaengine: Cannot load metadata from file /home/mn/images/1perso/1Contacts/contacts-d�tails/c-d�tails-famille/1contact-PROVISOIRE/197808-3200-3.xcf (Error # 11 : /home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-3.xcf: The file contains data of an unknown image type digikam.dimg: "/home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-3.xcf" : Try to load with ImageMagick digikam.metaengine: Cannot load metadata from file /home/mn/images/1perso/1Contacts/contacts-d�tails/c-d�tails-famille/1contact-PROVISOIRE/197808-3200-4.xcf (Error # 11 : /home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-4.xcf: The file contains data of an unknown image type digikam.dimg: "/home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-4.xcf" : Try to load with ImageMagick digikam.metaengine: Cannot load metadata from file /home/mn/images/1perso/1Contacts/contacts-d�tails/c-d�tails-famille/1contact-PROVISOIRE/197808-3200-2.xcf (Error # 11 : /home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-2.xcf: The file contains data of an unknown image type digikam.dimg: "/home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-2.xcf" : Try to load with ImageMagick digikam.metaengine: Cannot load metadata from file /home/mn/images/1perso/1Contacts/contacts-d�tails/c-d�tails-famille/1contact-PROVISOIRE/197808-3200-1.xcf (Error # 11 : /home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-1.xcf: The file contains data of an unknown image type digikam.metaengine: Cannot load metadata from file /home/mn/images/1perso/1Contacts/contacts-d�tails/c-d�tails-famille/1contact-PROVISOIRE/197808-3200-3.xcf (Error # 11 : /home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-3.xcf: The file contains data of an unknown image type digikam.metaengine: Cannot load metadata from file /home/mn/images/1perso/1Contacts/contacts-d�tails/c-d�tails-famille/1contact-PROVISOIRE/197808-3200-4.xcf (Error # 11 : /home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-4.xcf: The file contains data of an unknown image type digikam.metaengine: Cannot load metadata from file /home/mn/images/1perso/1Contacts/contacts-d�tails/c-d�tails-famille/1contact-PROVISOIRE/197808-3200-2.xcf (Error # 11 : /home/mn/images/1perso/1Contacts/contacts-détails/c-détails-famille/1contact-PROVISOIRE/197808-3200-2.xcf: The file contains data of an unknown image type digikam.dimg: Try to load image with ImageMagick codecs digikam.dimg: Try to load image with ImageMagick codecs digikam.dimg: Try to load image with ImageMagick codecs digikam.dimg: Try to load image with ImageMagick codecs "Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken."
Maybe useful: errors with "file contains data of an unknown image type" are big .xcf files, about 86 Mo
Perhaps a "run time" dependency... "file contains data of an unknown image type" => Exiv2 shared lib. Just an exception because it don't know file format to extract metadata.
Run-time Exiv dependencies? Installed on my computer: exiv2 0.27.2-1 libgexiv2 0.12.0-2 libexif 0.6.21-3
No. ImageMagick run time deps, as with convert CLI tool for ex. Gilles Caulier
What is your ImageMagick version? Maik
And didi you reproduce the dysfunction with last 6.3.0 AppImage pre-release bundle just uploaded ? https://files.kde.org/digikam/digikam-6.3.0-git-20190824T081330-x86-64.appimage.mirrorlist Gilles Caulier
imagemagick 7.0.8.59-1 is installed @gilles I could try Appimage but I don't see any md5 or sha256 in your link...
Just look on top level directory : https://files.kde.org/digikam/ Gilles Caulier
Great. Thanks!
Digikam Appimage doesnt work on my system. It needs old QT5 11. My QT5 is 5.13.0-7 In console: -- digiKam Linux AppImage Bundle -- Use 'help' as CLI argument to know all available options for digiKam application. -- Note: to integrate this bundle to your desktop, use AppImageLauncher. libudev.so.0 /tmp/.mount_digikaFQ16qT/AppRun: line 16: arch: command not found -- Preloading shared libs: :/usr/lib/libudev.so.0 digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by digikam) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libdigikamgui.so.6.3.0) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libdigikamcore.so.6.3.0) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5XmlGui.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5Notifications.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5NotifyConfig.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libQtAV.so.1) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libQtAVWidgets.so.1) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5KIOWidgets.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5WindowSystem.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5IconThemes.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5KIOCore.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Concurrent.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5JobWidgets.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libQt5X11Extras.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5Completion.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5ConfigWidgets.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5ConfigGui.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5WidgetsAddons.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5Codecs.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5Auth.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5AuthCore.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libQt5PrintSupport.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5Solid.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5Service.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5I18n.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Xml.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5ConfigCore.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5CoreAddons.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libQt5WebKitWidgets.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libQt5WebKit.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libmarblewidget-qt5.so.28) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libdigikamdatabase.so.6.3.0) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5GlobalAccel.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5ItemViews.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libQt5OpenGL.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Svg.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5Archive.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5Crash.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5DBusAddons.so.5) digikam: /tmp/.mount_digikaFQ16qT/usr/lib/libQt5Core.so.5: version `Qt_5.11' not found (required by /tmp/.mount_digikaFQ16qT/usr/lib/libKF5GuiAddons.so.5)
> What is your ImageMagick version? > Could my Imagemagick version (7.0.8.59) be too recent?
No. Qt 5.11 is included in AppImage bundle as well.
AppImage problem fixed. New 64 bits version is online. Please try again. https://files.kde.org/digikam/ Gilles Caulier
Git commit 73b6b00b5a29eb064292c292d18e8a1d0441a1c1 by Maik Qualmann. Committed on 25/08/2019 at 11:15. Pushed by mqualmann into branch 'master'. load image in real bit depth with the ImageMagick loader M +10 -4 core/libs/dimg/loaders/magickloader.cpp M +1 -0 core/libs/dimg/loaders/magickloader.h https://invent.kde.org/kde/digikam/commit/73b6b00b5a29eb064292c292d18e8a1d0441a1c1
After deeper debugging, I can not reproduce a memory hole with XCF files and our ImageMagick loader here. Is this behavior to reproduce for you with the current git/master version? Maik
Git master, sure. I compiled it on last August 24 Archlinux system Imagemagick installed version is 7.0.8.62, it may be different from yours? Here is my Arch Aur modified script to clone git and build Digikam: #################################################### pkgname=digikam-git pkgver=r42728.bf87ed54b8 pkgrel=1 pkgdesc='Digital photo management application for KDE' arch=('i686' 'x86_64') license=('GPL') url="http://www.digikam.org/" depends=(liblqr qtav lensfun knotifyconfig kfilemetadata opencv threadweaver qt5-xmlpatterns libgphoto2 imagemagick) makedepends=('git' 'extra-cmake-modules-git' 'eigen' 'doxygen' 'boost' 'mariadb' 'kdoctools') optdepends=('hugin: panorama tool' 'kimageformats') conflicts=('digikam') provides=('digikam') install=digikam-git.install source=('digikam::git+git://anongit.kde.org/digikam') md5sums=('SKIP') groups=('digikamsc-git') pkgver() { cd "${srcdir}/digikam" printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short HEAD)" } prepare() { if [[ -d "${srcdir}/build" ]]; then msg "Cleaning the previous build directory..." rm -rf "${srcdir}/build" fi mkdir "${srcdir}/build" } build() { cd "${srcdir}/build" cmake "${srcdir}/digikam" -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX=/usr \ -DLIB_INSTALL_DIR=lib \ -DKDE_INSTALL_USE_QT_SYS_PATHS=ON \ -DBUILD_TESTING=OFF \ -DENABLE_AKONADICONTACTSUPPORT=OFF \ -DENABLE_KFILEMETADATASUPPORT=ON \ -DENABLE_MYSQLSUPPORT=ON \ -DENABLE_INTERNALMYSQL=ON \ -DENABLE_MEDIAPLAYER=ON \ -DENABLE_APPSTYLES=ON \ -DENABLE_QWEBENGINE=OFF \ -DOpenGL_GL_PREFERENCE=GLVND make } package() { cd "${srcdir}/build" make DESTDIR="${pkgdir}" install }
@Maik I removed 4 xcf files with problem (see above), then, i could successfully update finger prints. I have many other xcf files but these 4 xcf files cause a problem. They are big files, about 88 Mo each. May be corrupted files? => I reopened them with Gimp, overwrite them. No change, except their size is reduced to about 75 mo. digikam.metaengine: Cannot load metadata from file /home/mn/images/1perso/1Contacts/TEST/197808-3200-3.xcf (Error # 11 : /home/mn/images/1per digikam.general: Trying to load Embedded preview with libraw^M digikam.rawengine: Failed to load embedded RAW preview^M digikam.general: Trying to load half preview with libraw^M digikam.general: Trying to load Embedded preview with Exiv2^M digikam.metaengine: Cannot load metadata using Exiv2 (Error # 11 : /home/mn/images/1perso/1Contacts/TEST/197808-3200-3.xcf: The file contain digikam.dimg: "/home/mn/images/1perso/1Contacts/TEST/197808-3200-3.xcf" : Try to load with ImageMagick^M digikam.metaengine: Cannot load metadata from file /home/mn/images/1perso/1Contacts/TEST/197808-3200-4.xcf (Error # 11 : /home/mn/images/1per digikam.general: Trying to load Embedded preview with libraw^M digikam.rawengine: Failed to load embedded RAW preview^M digikam.general: Trying to load half preview with libraw^M digikam.dimg: Try to load image with ImageMagick codecs^M digikam.general: Trying to load Embedded preview with Exiv2^M digikam.metaengine: Cannot load metadata using Exiv2 (Error # 11 : /home/mn/images/1perso/1Contacts/TEST/197808-3200-4.xcf: The file contain digikam.dimg: "/home/mn/images/1perso/1Contacts/TEST/197808-3200-4.xcf" : Try to load with ImageMagick^M digikam.dimg: Try to load image with ImageMagick codecs^M zsh: bus error (core dumped) digikam^M To get rid of this issue, i removed these 4 files from album. Digikam works now: i can update albums and finger prints "normally" @Gilles I tried your last digikam appimage (but i hate all appimages), i get error: -- digiKam Linux AppImage Bundle -- Use 'help' as CLI argument to know all available options for digiKam application. -- Note: to integrate this bundle to your desktop, use AppImageLauncher. libudev.so.0 /tmp/.mount_digikahfPb25/AppRun: line 16: arch: command not found -- Preloading shared libs: :/usr/lib/libudev.so.0 [34munknown[0m: QApplication: invalid style override passed, ignoring it. [34mDigikam::DXmlGuiWindow::setupIconTheme[0m: Breeze icons resource file found [34mDigikam::DXmlGuiWindow::setupIconTheme[0m: Breeze-dark icons resource file found [34mDigikam::AlbumWatch::AlbumWatch[0m: AlbumWatch is disabled [34mDigikam::AlbumManager::setDatabase[0m: Database Parameters: Type: "QMYSQL" DB Core Name: "dkfre" DB Thumbs Name: "dkfre" DB Face Name: "dkfre" DB Similarity Name: "dkfre" Connect Options: "" Host Name: "localhost" Host port: 3306 Internal Server: false Internal Server Path: "" Internal Server Serv Cmd: "" Internal Server Init Cmd: "" Username: "moi" Password: "XXXXXXXXXXXXXXXX" [34mDigikam::DbEngineConnectionChecker::run[0m: Error while opening the database. Error details [ QSqlError("2002", "QMYSQL: Unable to connect", "Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (13 \"Permission denied\")") ] [34mDigikam::DbEngineConnectionChecker::run[0m: Error while opening the database. Error details [ QSqlError("2002", "QMYSQL: Unable to connect", "Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (13 \"Permission denied\")") ]
The problem with connecting to a MySQL server is known. The use of "localhost" as a server address is not possible with the AppImage. You would have to use "127.0.0.1". Would it be possible to provide such a problematic XCF file to test it here? Maik
I changed localhost -> 127.0.0.1 Appimage works now > Would it be possible to provide such a problematic XCF file to test it here? I can't send present problematic XCF file but, when i'll get time, i'll create others and i'll send you. These xcf come from bw argentic film scan at high resolution.
Note: to test the AppImage the "127.0.0.1" is good. If you use your native digiKam version, you should use "localhost" again. Depending on the server configuration, "localhost" uses a socket file for communication, which is a bit faster. Maik
I use my native configuration with 127.0.0.1 now, it works. But, if you say 'localhost' is faster, I go back to it...
Digikam seems have problem with big size XCF files. Test example: Take any image about 4600x3800 Open it in Gimp-2.12 and convert it to testxcf.xcf File size for me: 49.6MB Add layer background color black Image has now 2 layers Then menu -> Image -> canvas size in the dialog box: Resize it to about width x height= 10.000 x 8175 Center it 'resize layers= image-sized layers' 'fill with= background color' Resize close dialog box Save testxcf.xcf file size is now about 50 Mo scan for new items For me, Digikam eats about 2.6 Mo memory, i get a kind of freeze, i have to wait for several minutes, I can't use computer. digikam.general: total scan value : 37490 digikam.dimg: IM to DImg : 10000 8175 digikam.dimg: IM QuantumRange : 65535 digikam.dimg: IM Format : GIMP image digikam.metaengine: Cannot load metadata from file /home/xxx/testxcf.xcf (Error # 11 : /home/xxx/testxcf.xcf: The file contains data of an unknown image type digikam.general: Trying to load Embedded preview with libraw digikam.rawengine: Failed to load embedded RAW preview digikam.general: Trying to load half preview with libraw digikam.general: Trying to load Embedded preview with Exiv2 digikam.metaengine: Cannot load metadata using Exiv2 (Error # 11 : /home/xxx/testxcf.xcf: The file contains data of an unknown image type digikam.dimg: "/home/xxx/testxcf.xcf" : Try to load with ImageMagick digikam.dimg: Try to load image with ImageMagick codecs
Rectification: Not 'Gimp 2.12' but 'Gimp 2.10.12' !
Good news. This issue doesn't happen with Appimage 6.3 I get errors but computer is not frozen. I don't understand why my Digikam git version can't do the same... ^[[34mDigikam::ActionThreadBase::cancel^[[0m: Cancel Main Thread^M ^[[34mDigikam::MetaEngine::Private::printExiv2ExceptionError^[[0m: Cannot load metadata from file /home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf (Error # 11 : /home/mn/images ^[[34mDigikam::ThumbnailCreator::createThumbnail^[[0m: Trying to load Embedded preview with libraw^M ^[[34mDigikam::DRawDecoder::loadEmbeddedPreview^[[0m: Failed to load embedded RAW preview^M ^[[34mDigikam::ThumbnailCreator::createThumbnail^[[0m: Trying to load half preview with libraw^M ^[[34mDigikam::ThumbnailCreator::createThumbnail^[[0m: Trying to load Embedded preview with Exiv2^M ^[[34mDigikam::MetaEngine::Private::printExiv2ExceptionError^[[0m: Cannot load metadata using Exiv2 (Error # 11 : /home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf: The file co ^[[34mDigikam::DImg::load^[[0m: "/home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf" : Try to load with ImageMagick^M ^[[34mDigikam::MagickLoader::load^[[0m: Try to load image with ImageMagick codecs^M ^[[34mDigikam::MagickLoader::load^[[0m: ImageMagick exception [ "/home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf" ] digikam: NoDecodeDelegateForThisImageFormat `XCF' @ error/con ^[[34mDigikam::DImg::load^[[0m: "/home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf" : Try to load with QImage^M ^[[34mDigikam::QImageLoader::load^[[0m: Blacklisted from DImg::QImageLoader: "image/x-xcf"^M ^[[34mDigikam::JPEGUtils::isJpegImage^[[0m: mimetype = "XCF" ext = "XCF"^M ^[[34mDigikam::ThumbnailCreator::createThumbnail^[[0m: Trying to load video preview with FFmpeg^M ^[[34mDigikam::VideoDecoder::initialize^[[0m: Could not open input file: "/home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf"^M ^[[34mDigikam::ThumbnailCreator::createThumbnail^[[0m: Cannot create thumbnail for "/home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf"^M ^[[34mDigikam::ThumbnailCreator::load^[[0m: Thumbnail is null for "/home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf"^M ^[[34munknown^[[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize ^[[34munknown^[[0m: Falling back to using screens root_visual.^M ^[[34mDigikam::ActionThreadBase::setMaximumNumberOfThreads^[[0m: Using 4 CPU core to run threads^M ^[[34mDigikam::MetaEngine::Private::printExiv2ExceptionError^[[0m: Cannot load metadata from file /home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf (Error # 11 : /home/mn/images ^[[34mDigikam::ThumbnailCreator::createThumbnail^[[0m: Trying to load Embedded preview with libraw^M ^[[34mDigikam::DRawDecoder::loadEmbeddedPreview^[[0m: Failed to load embedded RAW preview^M ^[[34mDigikam::ThumbnailCreator::createThumbnail^[[0m: Trying to load half preview with libraw^M ^[[34mDigikam::ThumbnailCreator::createThumbnail^[[0m: Trying to load Embedded preview with Exiv2^M ^[[34mDigikam::MetaEngine::Private::printExiv2ExceptionError^[[0m: Cannot load metadata using Exiv2 (Error # 11 : /home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf: The file co ^[[34mDigikam::DImg::load^[[0m: "/home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf" : Try to load with ImageMagick^M ^[[34mDigikam::MagickLoader::load^[[0m: Try to load image with ImageMagick codecs^M ^[[34mDigikam::MagickLoader::load^[[0m: ImageMagick exception [ "/home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf" ] digikam: NoDecodeDelegateForThisImageFormat `XCF' @ error/con ^[[34mDigikam::DImg::load^[[0m: "/home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf" : Try to load with QImage^M ^[[34mDigikam::QImageLoader::load^[[0m: Blacklisted from DImg::QImageLoader: "image/x-xcf"^M ^[[34mDigikam::JPEGUtils::isJpegImage^[[0m: mimetype = "XCF" ext = "XCF"^M ^[[34mDigikam::ThumbnailCreator::createThumbnail^[[0m: Trying to load video preview with FFmpeg^M ^[[34mDigikam::VideoDecoder::initialize^[[0m: Could not open input file: "/home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf"^M ^[[34mDigikam::ThumbnailCreator::createThumbnail^[[0m: Cannot create thumbnail for "/home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf"^M ^[[34mDigikam::ThumbnailCreator::load^[[0m: Thumbnail is null for "/home/mn/images/1perso/EXPERIENCE/Decoup/dec-fait/testxcf-2.xcf"^M ^[[34mDigikam::NewItemsFinder::slotStart^[[0m: scan mode: ScanDeferredFiles^M ^[[34mDigikam::ActionThreadBase::run^[[0m: Action Thread run 1 new jobs^M
Your ImageMagick version is the same as mine. The native output of comment 27 is not yet the current git/master version. There, the ImageMagick loader still loads everything in 16 bit, which means a huge memory requirement with such images. The output from the AppImage (comment 29) states that ImageMagick has an exception and does not support XCF. Here Gilles has to look at what's missing in the compilation. I was unable to detect a memory leak when loading such large Gimp files, but due to the huge memory requirement the computer is also very slow. Maik
Maik, I will take a look tomorrow morning. But a simple question. Can display IM CLI tool is able to run without to crash when you try to open this huge XCF file ? If there is one exception caught and is rerouted internally by the CLI tool, so it miss something in digiKam IM loader to catch the exception from IM without to crash. Gilles
My native digiKam version can load the large XCF image created by me. The current AppImage has an exception because of missing decoder. Digikam::MagickLoader::load: Try to load image with ImageMagick codecs Digikam::MagickLoader::load: ImageMagick exception [ "/daten/Bilder/Zugang/x/1/dsc_8672.xcf" ] digikam: NoDecodeDelegateForThisImageFormat `XCF' @ error/constitute.c/ReadImage/560 Maik
A missing codec in IM. Which one. I customized the IM build with this options : https://invent.kde.org/kde/digikam/blob/master/project/bundles/3rdparty/ext_imagemagick/CMakeLists.txt#L16 As you can see, i disabled "largefile" support... Gilles Caulier
Git commit 689e79e0c59aee864ed282857b629d8f9c8ae9d9 by Gilles Caulier. Committed on 27/08/2019 at 07:20. Pushed by cgilles into branch 'master'. enable IM large file support M +0 -1 project/bundles/3rdparty/ext_imagemagick/CMakeLists.txt https://invent.kde.org/kde/digikam/commit/689e79e0c59aee864ed282857b629d8f9c8ae9d9
I just rebuilt Digikam from git master commit 58c0b5f4 'sh => bash Gilles Caulier authored 6 hours ago' I got some errors and freeze after launching when Digikam try to search for new items. Digikam freezes desktop and partially computer. I see with htop in tty that Digikam eats about 3.5 Go resident memory during searching process, then, it returns to about 160 Mo memory. This issue doesn't happen at all with Digikam 6.3 beta Appimage. This one works fine. Problem still comes from 4 xcf files, about 80 Mo each. See below, for same files, differences behaviour between git version and Appimage. DIGIKAM GIT digikam.metaengine: Cannot load metadata from file /home/mn/images/imagesXCF-problem/197808-3200-1.xcf (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-1.xcf: The file contains data of an unknown image type digikam.general: Trying to load Embedded preview with libraw digikam.rawengine: Failed to load embedded RAW preview digikam.general: Trying to load half preview with libraw digikam.general: Trying to load Embedded preview with Exiv2 digikam.metaengine: Cannot load metadata using Exiv2 (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-1.xcf: The file contains data of an unknown image type digikam.dimg: "/home/mn/images/imagesXCF-problem/197808-3200-1.xcf" : Try to load with ImageMagick digikam.dimg: Try to load image with ImageMagick codecs digikam.general: Using 4 CPU core to run threads digikam.metaengine: Cannot load metadata from file /home/mn/images/imagesXCF-problem/197808-3200-4.xcf (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-4.xcf: The file contains data of an unknown image type digikam.general: Trying to load Embedded preview with libraw digikam.rawengine: Failed to load embedded RAW preview digikam.general: Trying to load half preview with libraw digikam.general: Trying to load Embedded preview with Exiv2 digikam.metaengine: Cannot load metadata using Exiv2 (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-4.xcf: The file contains data of an unknown image type digikam.dimg: "/home/mn/images/imagesXCF-problem/197808-3200-4.xcf" : Try to load with ImageMagick digikam.dimg: Try to load image with ImageMagick codecs digikam.general: scan mode: ScanDeferredFiles digikam.general: Action Thread run 1 new jobs digikam.general: Nothing to do. Databases are clean. "No carrier" digikam.metaengine: Cannot load metadata from file /home/mn/images/imagesXCF-problem/197808-3200-3.xcf (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-3.xcf: The file contains data of an unknown image type digikam.metaengine: Cannot load metadata from file /home/mn/images/imagesXCF-problem/197808-3200-2.xcf (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-2.xcf: The file contains data of an unknown image type digikam.general: Trying to load Embedded preview with libraw digikam.rawengine: Failed to load embedded RAW preview digikam.general: Trying to load half preview with libraw digikam.general: Trying to load Embedded preview with Exiv2 digikam.general: Trying to load Embedded preview with libraw digikam.rawengine: Failed to load embedded RAW preview digikam.general: Trying to load half preview with libraw digikam.general: Trying to load Embedded preview with Exiv2 digikam.metaengine: Cannot load metadata using Exiv2 (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-3.xcf: The file contains data of an unknown image type digikam.dimg: "/home/mn/images/imagesXCF-problem/197808-3200-3.xcf" : Try to load with ImageMagick digikam.metaengine: Cannot load metadata using Exiv2 (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-2.xcf: The file contains data of an unknown image type digikam.dimg: "/home/mn/images/imagesXCF-problem/197808-3200-2.xcf" : Try to load with ImageMagick digikam.dimg: Try to load image with ImageMagick codecs digikam.dimg: Try to load image with ImageMagick codecs "No carrier" ################################################## APPIMAGE [34mDigikam::MetaEngine::Private::printExiv2ExceptionError[0m: Cannot load metadata from file /home/mn/images/imagesXCF-problem/197808-3200-1.xcf (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-1.xcf: The file contains data of an unknown image type [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load Embedded preview with libraw [34mDigikam::DRawDecoder::loadEmbeddedPreview[0m: Failed to load embedded RAW preview [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load half preview with libraw [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load Embedded preview with Exiv2 [34mDigikam::MetaEngine::Private::printExiv2ExceptionError[0m: Cannot load metadata using Exiv2 (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-1.xcf: The file contains data of an unknown image type [34mDigikam::DImg::load[0m: "/home/mn/images/imagesXCF-problem/197808-3200-1.xcf" : Try to load with ImageMagick [34mDigikam::MagickLoader::load[0m: Try to load image with ImageMagick codecs [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual. [34mDigikam::MagickLoader::load[0m: ImageMagick exception [ "/home/mn/images/imagesXCF-problem/197808-3200-1.xcf" ] digikam: NoDecodeDelegateForThisImageFormat `XCF' @ error/constitute.c/ReadImage/560 [34mDigikam::DImg::load[0m: "/home/mn/images/imagesXCF-problem/197808-3200-1.xcf" : Try to load with QImage [34mDigikam::QImageLoader::load[0m: Blacklisted from DImg::QImageLoader: "image/x-xcf" [34mDigikam::JPEGUtils::isJpegImage[0m: mimetype = "XCF" ext = "XCF" [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load video preview with FFmpeg [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual. [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual. [34mDigikam::VideoDecoder::initialize[0m: Could not open input file: "/home/mn/images/imagesXCF-problem/197808-3200-1.xcf" [34mDigikam::ThumbnailCreator::createThumbnail[0m: Cannot create thumbnail for "/home/mn/images/imagesXCF-problem/197808-3200-1.xcf" [34mDigikam::ThumbnailCreator::load[0m: Thumbnail is null for "/home/mn/images/imagesXCF-problem/197808-3200-1.xcf" [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual. [34mDigikam::MetaEngine::Private::printExiv2ExceptionError[0m: Cannot load metadata from file /home/mn/images/imagesXCF-problem/197808-3200-2.xcf (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-2.xcf: The file contains data of an unknown image type [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load Embedded preview with libraw [34mDigikam::DRawDecoder::loadEmbeddedPreview[0m: Failed to load embedded RAW preview [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load half preview with libraw [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load Embedded preview with Exiv2 [34mDigikam::MetaEngine::Private::printExiv2ExceptionError[0m: Cannot load metadata using Exiv2 (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-2.xcf: The file contains data of an unknown image type [34mDigikam::DImg::load[0m: "/home/mn/images/imagesXCF-problem/197808-3200-2.xcf" : Try to load with ImageMagick [34mDigikam::MagickLoader::load[0m: Try to load image with ImageMagick codecs [34mDigikam::MagickLoader::load[0m: ImageMagick exception [ "/home/mn/images/imagesXCF-problem/197808-3200-2.xcf" ] digikam: NoDecodeDelegateForThisImageFormat `XCF' @ error/constitute.c/ReadImage/560 [34mDigikam::DImg::load[0m: "/home/mn/images/imagesXCF-problem/197808-3200-2.xcf" : Try to load with QImage [34mDigikam::QImageLoader::load[0m: Blacklisted from DImg::QImageLoader: "image/x-xcf" [34mDigikam::JPEGUtils::isJpegImage[0m: mimetype = "XCF" ext = "XCF" [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load video preview with FFmpeg [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual. [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual. [34mDigikam::VideoDecoder::initialize[0m: Could not open input file: "/home/mn/images/imagesXCF-problem/197808-3200-2.xcf" [34mDigikam::ThumbnailCreator::createThumbnail[0m: Cannot create thumbnail for "/home/mn/images/imagesXCF-problem/197808-3200-2.xcf" [34mDigikam::ThumbnailCreator::load[0m: Thumbnail is null for "/home/mn/images/imagesXCF-problem/197808-3200-2.xcf" [34mDigikam::MetaEngine::Private::printExiv2ExceptionError[0m: Cannot load metadata from file /home/mn/images/imagesXCF-problem/197808-3200-3.xcf (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-3.xcf: The file contains data of an unknown image type [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load Embedded preview with libraw [34mDigikam::DRawDecoder::loadEmbeddedPreview[0m: Failed to load embedded RAW preview [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load half preview with libraw [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load Embedded preview with Exiv2 [34mDigikam::MetaEngine::Private::printExiv2ExceptionError[0m: Cannot load metadata using Exiv2 (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-3.xcf: The file contains data of an unknown image type [34mDigikam::DImg::load[0m: "/home/mn/images/imagesXCF-problem/197808-3200-3.xcf" : Try to load with ImageMagick [34mDigikam::MagickLoader::load[0m: Try to load image with ImageMagick codecs [34mDigikam::MagickLoader::load[0m: ImageMagick exception [ "/home/mn/images/imagesXCF-problem/197808-3200-3.xcf" ] digikam: NoDecodeDelegateForThisImageFormat `XCF' @ error/constitute.c/ReadImage/560 [34mDigikam::DImg::load[0m: "/home/mn/images/imagesXCF-problem/197808-3200-3.xcf" : Try to load with QImage [34mDigikam::QImageLoader::load[0m: Blacklisted from DImg::QImageLoader: "image/x-xcf" [34mDigikam::JPEGUtils::isJpegImage[0m: mimetype = "XCF" ext = "XCF" [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load video preview with FFmpeg [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual. [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual. [34mDigikam::VideoDecoder::initialize[0m: Could not open input file: "/home/mn/images/imagesXCF-problem/197808-3200-3.xcf" [34mDigikam::ThumbnailCreator::createThumbnail[0m: Cannot create thumbnail for "/home/mn/images/imagesXCF-problem/197808-3200-3.xcf" [34mDigikam::ThumbnailCreator::load[0m: Thumbnail is null for "/home/mn/images/imagesXCF-problem/197808-3200-3.xcf" [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual. [34mDigikam::MapWidget::setBackend[0m: "setting backend marble" [34mDigikam::MetaEngine::Private::printExiv2ExceptionError[0m: Cannot load metadata from file /home/mn/images/imagesXCF-problem/197808-3200-4.xcf (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-4.xcf: The file contains data of an unknown image type [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load Embedded preview with libraw [34mDigikam::DRawDecoder::loadEmbeddedPreview[0m: Failed to load embedded RAW preview [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load half preview with libraw [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load Embedded preview with Exiv2 [34mDigikam::MetaEngine::Private::printExiv2ExceptionError[0m: Cannot load metadata using Exiv2 (Error # 11 : /home/mn/images/imagesXCF-problem/197808-3200-4.xcf: The file contains data of an unknown image type [34mDigikam::DImg::load[0m: "/home/mn/images/imagesXCF-problem/197808-3200-4.xcf" : Try to load with ImageMagick [34mDigikam::MagickLoader::load[0m: Try to load image with ImageMagick codecs [34mDigikam::MagickLoader::load[0m: ImageMagick exception [ "/home/mn/images/imagesXCF-problem/197808-3200-4.xcf" ] digikam: NoDecodeDelegateForThisImageFormat `XCF' @ error/constitute.c/ReadImage/560 [34mDigikam::DImg::load[0m: "/home/mn/images/imagesXCF-problem/197808-3200-4.xcf" : Try to load with QImage [34mDigikam::QImageLoader::load[0m: Blacklisted from DImg::QImageLoader: "image/x-xcf" [34mDigikam::JPEGUtils::isJpegImage[0m: mimetype = "XCF" ext = "XCF" [34mDigikam::ThumbnailCreator::createThumbnail[0m: Trying to load video preview with FFmpeg [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual. [34mDigikam::VideoDecoder::initialize[0m: Could not open input file: "/home/mn/images/imagesXCF-problem/197808-3200-4.xcf" [34mDigikam::ThumbnailCreator::createThumbnail[0m: Cannot create thumbnail for "/home/mn/images/imagesXCF-problem/197808-3200-4.xcf" [34mDigikam::ThumbnailCreator::load[0m: Thumbnail is null for "/home/mn/images/imagesXCF-problem/197808-3200-4.xcf" [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual. [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual. [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual. [34munknown[0m: No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SwapBehavior(SingleBuffer), swapInterval 1, colorSpace QSurfaceFormat::ColorSpace(DefaultColorSpace), profile QSurfaceFormat::OpenGLContextProfile(NoProfile)) [34munknown[0m: Falling back to using screens root_visual.
I can send you two modified original files that cause memory issue with Digikam. I'm not used to Dropbox (i just created account) I think I need email receiver.
Precisions about these two files: 28096 x 3328 pixels 3200 x 3200 ppi 8 bit 32280 x 3824 pixels 3200 x 3200 ppi 8 bit
Bear in mind that such images yield a huge amount of data as bitplanes, several layers, etc. The ImageMagick loader needs here to load my XCF image up to 4.5GB. The image buffer is about 350MB at the end. The image cache was set to 200MB in digiKam. So this image is not even cached. But I can not reproduce a memory leak. I also do not think that we can limit the memory consumption of ImageMagick. This extreme is only available with XCF images, other formats are fine. Maik
> Bear in mind that such images yield a huge amount of data as bitplanes, > several layers, etc. The ImageMagick loader needs here to load my XCF image > up to 4.5GB. The image buffer is about 350MB at the end. The image cache was > set to 200MB in digiKam. So this image is not even cached. But I can not > reproduce a memory leak. I also do not think that we can limit the memory > consumption of ImageMagick. This extreme is only available with XCF images, > other formats are fine. > It depends of image resolution, here, 3200 x 3200 xcf. Only one layer. I sent you my files.
Thanks for the test pictures. These are similar to what I already tested. My computer is older, 2 cores and 8GB RAM. Yes, the loading with ImageMagick consumes a lot of memory, up to 6GB and slow. But no memory hole. The fingerprints can also be created. You probably should not run the fingerprints with all cores. Maik
It is strange that Digikam appimage doesn't freeze my computer with these files. Appimage is compiled with which Imagemagick version?
The AppImage does not load any XCF files at the moment, no large memory block is requested, therefore no freeze. Which kernel version does your Linux system have? Maik
linux-lts 4.19.68 Archlinux system
I tried to open same file with imagemagick 7.0.8.62 (installed version on my system): 'display TestModif-resized-197808-3200-1-.xcf' It displays nothing, it freezes desktop and computer, it eats about 2 Go. I have to kill it in tty console. ==> Imagemagick 7.0.8.62 is buggy Solution: build Digikam git with better/older Imagemagick 7 version but which one? Besides, i got answer today from Imagemagick team about xcf multi-layers "bug": "Its not a bug. ImageMagick does not support Gimp 2.10 .XCF images. We may support the format in the future but currently we do not have an ETA on when support will be added." https://github.com/ImageMagick/ImageMagick/issues/1544
Rectification: i'll compile other IM git version https://aur.archlinux.org/packages/imagemagick-full/ https://aur.archlinux.org/packages/imagemagick-git/
I compiled and install this IM git version, same freeze. https://aur.archlinux.org/packages/imagemagick-full/ I think Digikam Appimage is built with old Imagemagick version.
Go to Help/components Info for details in DK... Do you suggest to compile the AppImage with a more recent version of IM ? If yes which one ? Gilles Caulier
by IM version, i want mean stable version released, not git/master Gilles Caulier
You use Imagemagick 6.9. It's different, i understand... For me, on Arch, I think I have to replace 'Imagemagick" by 'libmagick6' in dependencies https://www.archlinux.org/packages/extra/x86_64/libmagick6/ You could try with last IM, i think you'll meet same issue than mine with big xcf files. The last stable Imagemagick version is 7.0.8.62. https://github.com/ImageMagick/ImageMagick/releases It's installed on my system. https://www.archlinux.org/packages/extra/x86_64/imagemagick/ Arch script install here: https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/imagemagick
I get nearly same freeze with Digikam git master compiled with Imagemagick 6.9.10.62. Same errors than Appimage with xcf files, but Digikam Appimage 6.3 doesn't freeze computer. Problem is freeze, not errors.
> I get nearly same freeze with Digikam git master compiled with Imagemagick > 6.9.10.62. > Same errors than Appimage with xcf files, but Digikam Appimage 6.3 doesn't > freeze computer. Problem is freeze, not errors. Maybe useful Console copy of yesterday 'update finger prints' digikam.general: Updating fingerprints for file: "/home/mn/images/imagesXCF-problem/TestModif-197808-3200-1-.xcf" digikam.general: Updating fingerprints for file: "/home/mn/images/imagesXCF-problem/TestModif-resized-197808-3200-1-.xcf" digikam.general: Updating fingerprints for file: "/home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf" digikam.metaengine: Cannot load metadata from file /home/mn/images/imagesXCF-problem/TestModif-resized-197808-3200-1-.xcf (Error # 11 : /home/mn/images/imagesXCF-problem/TestModif-resized-197808-3200-1-.xcf: The file contains data of an unknown image type digikam.dimg: "/home/mn/images/imagesXCF-problem/TestModif-resized-197808-3200-1-.xcf" : Try to load with ImageMagick digikam.metaengine: Cannot load metadata from file /home/mn/images/imagesXCF-problem/TestModif-197808-3200-1-.xcf (Error # 11 : /home/mn/images/imagesXCF-problem/TestModif-197808-3200-1-.xcf: The file contains data of an unknown image type digikam.dimg: "/home/mn/images/imagesXCF-problem/TestModif-197808-3200-1-.xcf" : Try to load with ImageMagick digikam.metaengine: Cannot load metadata from file /home/mn/images/imagesXCF-problem/TestModif-resized-197808-3200-1-.xcf (Error # 11 : /home/mn/images/imagesXCF-problem/TestModif-resized-197808-3200-1-.xcf: The file contains data of an unknown image type digikam.dimg: Try to load image with ImageMagick codecs digikam.metaengine: Cannot load metadata from file /home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf (Error # 11 : /home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf: The file contains data of an unknown image type digikam.metaengine: Cannot load metadata from file /home/mn/images/imagesXCF-problem/TestModif-197808-3200-1-.xcf (Error # 11 : /home/mn/images/imagesXCF-problem/TestModif-197808-3200-1-.xcf: The file contains data of an unknown image type digikam.dimg: Try to load image with ImageMagick codecs digikam.dimg: "/home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf" : Try to load with ImageMagick digikam.metaengine: Cannot load metadata from file /home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf (Error # 11 : /home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf: The file contains data of an unknown image type digikam.dimg: Try to load image with ImageMagick codecs digikam.dimg: ImageMagick exception [ "/home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf" ] digikam: memory allocation failed `/home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf' @ error/xcf.c/ReadXCFImage/1360 digikam.dimg: "/home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf" : Try to load with QImage digikam.metaengine: Cannot load metadata from file /home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf (Error # 11 : /home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf: The file contains data of an unknown image type digikam.dimg.qimage: Can not load " "/home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf" " using DImg::QImageLoader! digikam.dimg.qimage: Error message from loader: "Unable to read image data" digikam.general: Cannot extract preview for "/home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf" "No carrier" zsh: bus error (core dumped) digikam--
ImageMagick has an exception, because it does not get the required memory. You write in your 1st comment that the tmp file system is full. Can you reproduce it and see which files are created there? --------------------------------------------------- digikam.dimg: ImageMagick exception [ "/home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf" ] digikam: memory allocation failed `/home/mn/images/test-gimp2.10/test-gimp2.10-ancien/Juglans-1.xcf' @ error/xcf.c/ReadXCFImage/1360 --------------------------------------------------- Maik
I modified big xcf file then updated folder. I get freeze during about 5 minutes. Huge files were created in /tmp with same time: │ magick-3060RirJ85e9piIh │ 2287M│ │ magick-3060Ru0G0fKDRxWo │ 2287M│ │ magick-3060fHpmWqwS22ta │ 2287M│
I forgot: freeze stops because Digikam crashes
Strange, I can not reproduce here that ImageMagick creates any files in the tmp directory. Maik
Only modified huge xcf cause problem when updating folder. I reproduced it this morning with same Digikam version (compiled with IM this way: Open TestModif-resized-197808-3200-1-.xcf in Gimp to resize it image -> canvas -> resize to 40.000 x 4739 pixels center Image info now in Gimp: resolution 3200 x 3200 40.000 x 4739 pixels grayscale 8 bit 94.2 Mo 1.4 Go in memory Then in Digikam, when i update folder, i get freeze. I go to tty (it takes time) where htop is launched. I see digikam eats about 5.5 Go. After 10 minutes, i come back to computer. Digikam has crashed. I go in /tmp. I see two huge files created by image magick: magick-1867xxxname 2287 M magick-1867xxxthername 2287 M Their permission: 100600 I remove these to magick files to get out of freeze.
Rectification : compiled with IM 6
As i wrote here https://bugs.kde.org/show_bug.cgi?id=411214#c44 issue comes from Imagemagick 6 and 7, not from Digikam itself. And yet, strangely, this bug does not appear with the Digikam Appimage 6.3 beta version... Logically, I should report the bug to IM team but they say Gimp 2.10 xcf format is not supported by IM.
I compiled digikam from last git master today (Commit 806bcd39) When I try to update finger prints, I get computer freeze. Digikam himself eats about 3700M IM7 creates 3 huge files in /tmp magick-1828xxxxxxxxx 1247M magick-1828xxxxxxxxx 1796M magick-1828xxxxxxxxx 911004K Important: 'testxcf-2.xcf' problematic file size is only 49615 K digikam.metaengine: Cannot load metadata from file /home/mn/images/testxcf-2.xcf (Error # 11 : /home/mn/images/testxcf-2.xcf: The file contains data of an unknown image type digikam.dimg: "/home/mn/images/testxcf-2.xcf" : Try to load with ImageMagick digikam.metaengine: Cannot load metadata from file /home/mn/images/testxcf-2.xcf (Error # 11 : /home/mn/images/testxcf-2.xcf: The file contains data of an unknown image type digikam.dimg: Try to load image with ImageMagick codecs
We are beginning to realize that IM is not suitable as an image loader for digiKam. See also Bug 411027. I think we will disable IM in digiKam-6.4.0 for the time being. Maik
Sad to say but, good news...
Maderios, Can you test Krita editor ? https://invent.kde.org/kde/krita This one as plenty of image loader hardcoded, including XCF ! https://invent.kde.org/kde/krita/tree/master/plugins/impex If you can open your XCF files with Krita, this want mean that i have a suitable code which can be backported in digiKam. I plan to create a new plugin interface for image loader in digiKam, as it's done with Krita. Best Gilles Caulier
I just tried to open gimp 2.10 XCF file with Krita 4.2.6 : Krita says it doesn't support 2.9/2.10 gimp xcf.
Perhaps krita team has a file in bugzilla about this topic and work on it ? Gilles Caulier
> This one as plenty of image loader hardcoded, including XCF ! > > https://invent.kde.org/kde/krita/tree/master/plugins/impex > Yes but your support is old, for gimp <= 2.8 xcf
I'm surprised that nobody report this problem yet... Nobody use Gimp 2.10 (:=))) ? Gilles Caulier
Except Gimp 2.10, at this time, no software can open/display gimp 2.10 XCF files. Gnome softs are unable to do it. Using <= 128 pixels thumbnails cached by Gimp, may be a solution to display thumbs in Digikam ?
thumb of 128x128 are not enough. For HDPI screens we support 512x512 thumbs... Gilles
If a HiDPI screen has a scaling factor, digiKam supports up to 10124x1024 thumbnails. Maik
I just recompiled digikam git without Imagemagick. It freezes same way with XCF. I see it tries to use IM 'Try to load image with ImageMagick codecs' I don't understand why...
Maik, has already investigate about this problem and found a dysfunction in a KImageFormat XCF codec : https://github.com/KDE/kimageformats/tree/master/src/imageformats KImageFormat are QImage IO plugins. Un-installing this package will not break Qt, but remove codecs used at run time. This must solve your problem. Gilles Caulier
I recompiled (thanks Ccache to get time!) last git master digikam without imagemagick and kimageformats dependencies. I uninstalled kimageformats. I get same freeze, 4 huge imagemagick files in /tmp, digikam eats more 3 G memory. Strange issue...
Issue happens only when updating finger prints.
I see in "component information' that my digikam-git is compiled with Imagemagick codecs but it's false. I think there's something inside digikam code that forces digikam build to use Imagemagick. Important: I clean build directory each time before I compile digikam with options '-sicC': man makepkg -C, --cleanbuild Remove the $srcdir before building the package. -c, --clean Clean up leftover work files and directories after a successful build. -s, --syncdeps My PKGBUILD script: pkgname=digikam-git pkgver=r43949.5352aa5f77 pkgrel=3 pkgdesc='Digital photo management application for KDE' arch=('i686' 'x86_64') license=('GPL') url="http://www.digikam.org/" depends=(liblqr lensfun opencv knotifyconfig kfilemetadata qtav threadweaver qt5-xmlpatterns) makedepends=(extra-cmake-modules doxygen eigen boost kdoctools git) optdepends=('hugin: panorama tool' 'qt5-imageformats: support for additional image formats (WEBP, TIFF)') conflicts=('digikam') provides=('digikam') install=digikam-git.install source=('digikam::git+git://anongit.kde.org/digikam') md5sums=('SKIP') pkgver() { cd "${srcdir}/digikam" printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short HEAD)" } prepare() { if [[ -d "${srcdir}/build" ]]; then msg "Cleaning the previous build directory..." rm -rf "${srcdir}/build" fi mkdir "${srcdir}/build" } build() { cd "${srcdir}/build" cmake ../digikam \ -DCMAKE_INSTALL_PREFIX=/usr \ -DCMAKE_INSTALL_LIBDIR=lib \ -DBUILD_TESTING=OFF \ -DENABLE_KFILEMETADATASUPPORT=ON \ -DENABLE_MEDIAPLAYER=ON \ -DENABLE_AKONADICONTACTSUPPORT=OFF \ -DENABLE_MYSQLSUPPORT=ON \ -DENABLE_APPSTYLES=ON \ -DENABLE_QWEBENGINE=OFF \ -DOpenGL_GL_PREFERENCE=GLVND make } package() { cd "${srcdir}/build" make DESTDIR="${pkgdir}" install }
I'm adding a whitelist to IM Loader tonight. For Webp and others it is well suited. Maik
Git commit fc62ae63611bff6bba73fa4cdb68feea919dae5a by Maik Qualmann. Committed on 16/09/2019 at 16:42. Pushed by mqualmann into branch 'master'. add blacklist to ImageMagick loader, first mime type is XCF M +8 -3 core/libs/dimg/loaders/magickloader.cpp https://invent.kde.org/kde/digikam/commit/fc62ae63611bff6bba73fa4cdb68feea919dae5a
No more freeze when updating XCF finger prints. It works :)
But also no fingerprints for gimp-2.10 files... Maik
Yes but IM creates worth problem, there is no good choice... In the past, we updated finger prints without Imagemagick. Could we get rid of IM and go back to Kimageformats?
Yes, the ImageMagick loader now ignores XCF files. Now KImageFormats tries to open the images... Maik
I imagine Digikam still needs IM because IM is still present in Digikam code. That's why it's necessary to blacklist it. Can Digikam work without IM now?
DigiKam also works without ImageMagick. Has advantages for a few special formats, e.g. Webp, because ImageMagick also supports color profiles. Maik
The problem with the memory consumption of XCF files is fixed, I close the bug now. Maik