STEPS TO REPRODUCE Sqlite db, 'Wal mode' enabled. Same issue with all settings enabled in 'perform database cleaning', , 'extended clean up and shrink database' Settings: Uses large thumbnails size for high screen resolution 'F5' in album to create thumbnails OBSERVED RESULT Creating big thumbnails works normally with 'F5', 'process is done' is displayed but strangely progress bar is stalled at 95% or 90, , 80, 85%, etc.... It stays stalled but that doesn't prevent continuing to create thumbnails in other albums/directories. It happens mainly with big files Even after closing, Digikam processes never end, i have to kill Digikam process to stop it SOFTWARE/OS VERSIONS Build rev 4cf54b71 Arch Linux Digikam git qt5 5.15.6 KF5 5.97 ffmpeg 5.1.1
Concerned files: jpeg, png, xcf, mp4, avi Messages in console: nmReply "Method \"GetDevices\" with signature \"\" on interface \"org.freedesktop.NetworkManager\" doesn't exist\n" "Object path cannot be empty" "Not supported" kf.xmlgui: Unhandled container to remove : Digikam::DigikamApp QLayout: Attempting to add QLayout "" to QWidget "", which already has a layout QLayout: Attempting to add QLayout "" to QWidget "", which already has a layout "Not supported" "Not supported" "Not supported" "Not supported" QLayout: Attempting to add QLayout "" to QWidget "", which already has a layout "Not supported" QLayout: Attempting to add QLayout "" to QWidget "", which already has a layout "Not supported" QLayout: Attempting to add QLayout "" to QWidget "", which already has a layout "Not supported" "Not supported" "Not supported" "Not supported" "Not supported" digikam.general: A tool identified as "ThumbsGenerator" is already running. QLayout: Attempting to add QLayout "" to QWidget "", which already has a layout "Not supported" qt.qpa.xcb: Unhandled client message: "_E_COMP_FLUSH" qt.qpa.xcb: Unhandled client message: "_E_COMP_DUMP" "Not supported" "Not supported" "Not supported"
Well, the reason will be clear, you use FFmpeg5. You know digiKam-8.0.0 compiles with FFmpeg5 but is not ready, the video thumbnailer will be affected too. The adjustments to FFmpeg5 will not go so quickly either. You must use FFmpeg4. We are considering using QMultimedia as an alternative, but only QMultimedia from Qt6 is interesting because it has a greatly improved API and FFmpeg support. By the way, your problem cannot be reproduced here either with video files and FFmpeg4... Maik
Maik, in ffmpegthmbnailer code, i don't see a limitation to ffmpeg 4. This code is already ported to ffmpeg 5 ? Gilles
I rebuilt Digikam with ffmpeg4.4. For those who use Arch, add in PKGBUILD 'export PKG_CONFIG_PATH='/usr/lib/ffmpeg4.4/pkgconfig' Video player works now but I get same issues with creating big thumbnails: progress bar is stalled at 99% (but big thumbnails are created like with ffmpeg5) and Digikam processe don't stop after closing.
Try to find out what image/video type the problem occurs with. Remove AVI files first, then MP4, then XCF. Maik
I copied files separately in four directories: jpeg avi xcf mp4 Same issue with big thumbnails happens with, separately: jpeg or avi or xcf or mp4 Progress bar stops at 95%, 90, 97, 96%
Please enable internal debugging in the digiKam settings under Miscellaneous-> System, restart digiKam in the terminal. Try a minimal set of images/videos where the problem occurs and give us the complete log from the terminal. Maik
Debugging file here https://www.dropbox.com/s/p3405fxo3gxya5a/digikam-dbg-thumbnails-2.txt?dl=0
Well, I see in the log that the message window has started with the message that the task is complete. This sets the progress manager to "100%" and exits. It is unknown why that probably no longer works for you, there were no program changes here. It is clear, the thumbnail creation will be fully processed. No matter how many files or types, the problem cannot be reproduced here. There were already strange effects with Qt-5.15.5 (QTreeWidget crash), let's see if I can reproduce it when openSUSE switches to Qt-5.15.6. Maik
This message is strange: digikam.general: A tool identified as "ThumbsGenerator" is already running. Do you press F5 several times while another thumbnail task is running? Maik
Git commit dc9c6ede1feb7789c3348098ce97df3a6d8d815e by Maik Qualmann. Committed on 13/09/2022 at 20:33. Pushed by mqualmann into branch 'master'. try a unique progress manager ID M +1 -1 core/utilities/maintenance/newitemsfinder.cpp M +2 -2 core/utilities/maintenance/thumbsgenerator.cpp https://invent.kde.org/graphics/digikam/commit/dc9c6ede1feb7789c3348098ce97df3a6d8d815e
Please check the latest git/master and test it. Maik
(In reply to Maik Qualmann from comment #10) > This message is strange: > > digikam.general: A tool identified as "ThumbsGenerator" is already running. > > Do you press F5 several times while another thumbnail task is running? > > Maik I remember I pressed F5 while progress bar was stalled. Progress bar stalling did not prevent generating big thumbnails
Works better but after renaming some files, progress bar is stalled at 97% https://www.dropbox.com/s/hbwetht3uga4ftp/dk-dbug-thumbnails-140922-2.txt?dl=0
Git commit 63ce584bf8a06077d04eb60ebd98e6411afed8ad by Maik Qualmann. Committed on 14/09/2022 at 18:48. Pushed by mqualmann into branch 'master'. prevent new maintenance tasks from starting while a task is still running M +9 -0 core/utilities/maintenance/dbcleaner.cpp M +9 -0 core/utilities/maintenance/duplicatesfinder.cpp M +10 -3 core/utilities/maintenance/facesdetector.cpp M +11 -3 core/utilities/maintenance/fingerprintsgenerator.cpp M +11 -3 core/utilities/maintenance/imagequalitysorter.cpp M +11 -4 core/utilities/maintenance/metadatasynchronizer.cpp M +13 -5 core/utilities/maintenance/newitemsfinder.cpp M +11 -3 core/utilities/maintenance/thumbsgenerator.cpp https://invent.kde.org/graphics/digikam/commit/63ce584bf8a06077d04eb60ebd98e6411afed8ad
After rebuilding, issue is still present: progress bar is stalled at 90%, Digikam process never ends, I have to kill it https://www.dropbox.com/s/dnl1b25saaxa01p/digikam-sqlite-150922.txt?dl=0 Other thing: I tested with mariadb in same conditions, user, folders, files: I could not reproduce issue May be sqlite problem?
> Other thing: I tested with mariadb in same conditions, user, folders, files: > I could not reproduce issue > May be sqlite problem? I'm wrong, It's not sqlite issue because i got now same problem with mariadb (internal)
*External* mariadb, not internal, of course...
I would to add more tests following comment #7 : I recommend to create a new user account and to setup a clean digiKam setup with an empty database (sqlite can be enough to start). Import one by one suspected files until to reproduce the dysfunction. Gilles
I'm wondering where the "Not supported" messages come from in your log. They usually appear before a new ActionThreadBase is created or in context. Maik
Git commit 398075e68c1e343a5eff48fd8ea6db8a61f8f5a2 by Maik Qualmann. Committed on 15/09/2022 at 21:00. Pushed by mqualmann into branch 'master'. add Debug for the thumbnails task M +1 -1 core/utilities/maintenance/maintenancethread.cpp M +1 -1 core/utilities/maintenance/thumbstask.cpp https://invent.kde.org/graphics/digikam/commit/398075e68c1e343a5eff48fd8ea6db8a61f8f5a2
Please create a log file with the additional debug messages. Try to limit it to fewer files. A lot of identical files are found in the initial scan, probably renamed. Is this a condition for the problem? Maik
New user, fresh configuration with sqlite db. I just added one .jpeg file: Digikam is entirely stalled, progression bar at 0%. I had to kill it manually. The Jpeg file i used available in dropbox Log here https://www.dropbox.com/s/gupk7vqqf3w1fz5/digikam-thumbnail-dbg-160922-2.txt?dl=0
Yes, 4 thumbnail tasks are created (you have 4 cores), 3 tasks return immediately because there is only one image. This task loads a thumbnail via DImg, but doesn't return. I suspect the problem is somewhere on your side. Please completely delete the build directory and the CCache and create everything again. Are you using any weird compiler flags? And I'll see if the "Not supported" warning exists in the Qt source code. Maik
(In reply to Maik Qualmann from comment #24) > Yes, 4 thumbnail tasks are created (you have 4 cores), 3 tasks return > immediately because there is only one image. This task loads a thumbnail via > DImg, but doesn't return. I suspect the problem is somewhere on your side. > Please completely delete the build directory and the CCache and create > everything again. Are you using any weird compiler flags? And I'll see if > the "Not supported" warning exists in the Qt source code. > > Maik I Disable ccache, remove .ccache directory, delete build directory then build Digikam again. I use 'PKGBUILD' Arch script compilation, it's very simple, see it in dropbox. The facts: I launch Digikam with empty album. Thumbnail size setting is set to big size. Then I copy to album only one jpeg file, the same jpeg than previously but with another name 'B-eyes.jpg' Big thumbnail is created then I do F5, i don't get problem. Then I delete 'B-eyes.jpg' and resize thumbnails settings to small, medium Album is empty now I copy again to album only one same jpeg file 'B-eyes.jpg' Small thumbnail is created, then, I resize to get big thumbnail and do F5 Progress bar is stalled at 0%, I can't close it. I try to close Digikam but process is still present after doing it. I have to kill process See 3 files in dropbox https://www.dropbox.com/s/xnxdkzdv2be3h3n/digikam-thumbnail-dbg-17-09-1.txt?dl=0
Digikam build log here https://www.dropbox.com/s/sg7b15bu0kwwtak/dkbuild-noccache-170922-1.txt?dl=0
We currently have a problem in showFoto (crash) with the ExifTool Metadata Parser. Under certain conditions, the parser "hangs" because it no longer exits its QEventLoop. I want to fix this problem first. It could possibly be related to your problem. Your "Not supported" message worries me more, I think it's from Qt. Do you also compile Qt yourself? Maik
I don't compile Qt, I use official Qt Arch packages 5.15.6 https://archlinux.org/packages/?q=qt5
Git commit 31b77b8e9e1f623ae8b2cf033223a1eb1bba78d7 by Maik Qualmann. Committed on 17/09/2022 at 20:32. Pushed by mqualmann into branch 'master'. remove QEventLoop from ExifToolParser fix crash with showFoto M +22 -54 core/libs/metadataengine/exiftool/exiftoolparser.cpp M +19 -17 core/libs/metadataengine/exiftool/exiftoolparser.h M +1 -6 core/libs/metadataengine/exiftool/exiftoolparser_command.cpp M +14 -14 core/libs/metadataengine/exiftool/exiftoolparser_output.cpp M +25 -29 core/libs/metadataengine/exiftool/exiftoolparser_p.cpp M +7 -4 core/libs/metadataengine/exiftool/exiftoolparser_p.h M +8 -3 core/libs/metadataengine/exiftool/exiftoolprocess.cpp M +2 -17 core/libs/metadataengine/exiftool/exiftoolprocess.h M +22 -10 core/libs/metadataengine/exiftool/exiftoolprocess_p.cpp M +6 -2 core/libs/metadataengine/exiftool/exiftoolprocess_p.h M +4 -0 core/tests/imgqsort/imgqsorttest.h M +4 -0 core/tests/metadataengine/abstractunittest.h M +4 -0 core/tests/metadataengine/exiftool/exiftoolapplychanges_cli.cpp M +4 -0 core/tests/metadataengine/exiftool/exiftoolexport_cli.cpp M +4 -0 core/tests/metadataengine/exiftool/exiftoolformats_cli.cpp M +4 -0 core/tests/metadataengine/exiftool/exiftoollanguages_cli.cpp M +4 -0 core/tests/metadataengine/exiftool/exiftoolmulticore_cli.cpp M +4 -0 core/tests/metadataengine/exiftool/exiftooloutput_cli.cpp M +4 -0 core/tests/metadataengine/exiftool/exiftoolparserout_cli.cpp M +5 -1 core/tests/metadataengine/exiftool/exiftooltagsdb_cli.cpp M +4 -0 core/tests/metadataengine/exiftool/exiftoolversion_cli.cpp M +4 -0 core/tests/metadataengine/exiftool/exiftoolwrite_cli.cpp https://invent.kde.org/graphics/digikam/commit/31b77b8e9e1f623ae8b2cf033223a1eb1bba78d7
Same issue after rebuild : progress bar is stalled at 0% as soon I do 'F5' to regenerate big thumbnail. Digikam process is still present after closing. https://www.dropbox.com/s/jrxcgfc6tarx7v0/dk-dbg-thumbnails-180922-1.txt?dl=0 (I added new jpeg file)
Are your Qt Arch Linux packages up to date? Maik
(In reply to Maik Qualmann from comment #31) > Are your Qt Arch Linux packages up to date? > > Maik Of course... !!!
Result for for 'qt5' search installed package on my system: libdbusmenu-qt5 0.9.3+16.04.20160218-6 liblastfm-qt5 1.1.0-1 libmygpo-qt5 1.1.0-3 polkit-qt5 0.114.0-1 poppler-qt5 22.09.0-1 python-pyqt5 5.15.7-1 python-pyqt5-sip 12.11.0-1 python-pyqt5-webengine 5.15.6-1 qca-qt5 2.3.4-3 qt5-base 5.15.6+kde+r167-1 qt5-charts 5.15.6+kde+r0-1 qt5-declarative 5.15.6+kde+r20-1 qt5-graphicaleffects 5.15.6+kde+r0-1 qt5-imageformats 5.15.6+kde+r4-1 qt5-location 5.15.6+kde+r3-1 qt5-multimedia 5.15.6+kde+r1-1 qt5-networkauth 5.15.6+kde+r0-1 qt5-quickcontrols 5.15.6+kde+r0-1 qt5-quickcontrols2 5.15.6+kde+r5-1 qt5-script 5.15.10-3 qt5-sensors 5.15.6+kde+r0-1 qt5-serialport 5.15.6+kde+r0-1 qt5-speech 5.15.6+kde+r1-1 qt5-svg 5.15.6+kde+r9-1 qt5-tools 5.15.6+kde+r1-1 qt5-translations 5.15.6+kde+r2-1 qt5-wayland 5.15.6+kde+r43-2 qt5-webchannel 5.15.6+kde+r3-1 qt5-webengine 5.15.10-3 qt5-webkit 5.212.0alpha4-16 qt5-websockets 5.15.6+kde+r2-1 qt5-x11extras 5.15.6+kde+r0-1 qt5-xmlpatterns 5.15.6+kde+r0-1 qtkeychain-qt5 0.13.2-1
Git commit ed2b9b763a2ca043eb39f076b9b3eea0b037b2d3 by Maik Qualmann. Committed on 19/09/2022 at 08:07. Pushed by mqualmann into branch 'master'. this should fix the thumbnail creation issue M +5 -4 core/utilities/maintenance/thumbstask.cpp https://invent.kde.org/graphics/digikam/commit/ed2b9b763a2ca043eb39f076b9b3eea0b037b2d3
Issue is not thumbnails creation (because they are created, even big thumbnails) but progress bar staying stalled at 0%, as soon as I move one jpeg file in image directory/album and doing 'F5'. Issue happens only after doing 'F5' to generate big thumbnail. After rebuild from last git, I get same issue than previously: after doing 'F5', progress bar is stalled at 0%, Digikam can't close, I have to kill process. I used same 2 jpeg files. Debug message here https://www.dropbox.com/s/wdb0xj5e5wtu5db/dk-dbg-bigthumbnails-1909-1.txt?dl=0
For a test, please deactivate the monitoring of the albums for external changes. The thumbnail is already loaded by this function, theoretically "F5" would not be necessary. Does the problem with "F5" still occur? Maik
(In reply to Maik Qualmann from comment #36) > For a test, please deactivate the monitoring of the albums for external > changes. The thumbnail is already loaded by this function, theoretically > "F5" would not be necessary. > Does the problem with "F5" still occur? > > Maik After rebuilding (commit 2af3c38d): I deactivated album monitoring, restarted, copy one jpeg image to album/folder image : no thumbnail appears (i think it's normal. Then I do 'F5', thumbnail is created but thumbs progress bar stays stalled at 0%. Same issue... https://www.dropbox.com/s/kz0jc7h4i6zeif1/dbg-bigthumbnails-1909-dk-8.0.60-1.txt?dl=0
To now, i can/could reproduce issue on "computer 1", a laptop Strange, i cannot reproduce this issue on an other computer, "computer2" in same conditions. More powerful computer than laptop. On this "computer 2", I use exactly same Arch system/packages, same user/directories/images and same sqlite db I copied from "computer 1", same settings. I can do F5 with big thumbnails on "computer 2" without any problem. For me, it means issue doesn't come from packages or images files. But from where...
Do the "Not Supported" messages also appear on the working computer? Maik
(In reply to Maik Qualmann from comment #39) > Do the "Not Supported" messages also appear on the working computer? > > Maik If I remember, yes but less than on "computer 1" , to be sure, I have to check it tomorrow, I can't do it now
Today, i get same issue on 'computer 2' : "Not Supported" messages, big thumbnails progress bar is stalled when i do F5 and i have to kill Digikam at the end. I tried for other user on this computer but I get same issue
Computer 1 After rebuilding Digikam (725ac6bd) and updating Qt5-base https://github.com/archlinux/svntogit-packages/commits/packages/qt5-base/trunk , issue has changed: Conditions: 'Monitor the album for external change' is *deactivated* F5 works normally with big thumbnails, thumbs progress bar reach 100% then it's closed Digikam closes normally But if 'Monitor the album for external change' is *activated* Issue is back: progress bar is stalled with F5 and big thumbnails
Precision: no issue when doing F5 only one/first time But issue is back if I do again F5 In two cases, I get "Not supported" message https://www.dropbox.com/s/wyhrj56tw1sxw7f/dbg-pc1-bigthumbnails-200922-6-NonMonitor.txt?dl=0
Git commit ce5c4f2d707c0b073260cc5373a1515ff56ccd84 by Maik Qualmann. Committed on 21/09/2022 at 06:03. Pushed by mqualmann into branch 'master'. add thumbnail task test debug M +3 -2 core/libs/threadimageio/thumb/thumbnailtask.cpp https://invent.kde.org/graphics/digikam/commit/ce5c4f2d707c0b073260cc5373a1515ff56ccd84
Rebuild then test with several image and video files, thumb progress bar stalled at 66% with big thumbnails https://www.dropbox.com/s/kyw9hh3bsjpj232/dk-debug-thumb-2109.txt?dl=0
Waiting for a thumbnail that is already loaded in another task is not the cause of the problem. Since the problem cannot be reproduced here, I currently have no idea why a task is not returning. And the problem can only be reproduced with "big" thumbnails? Maik
(In reply to Maik Qualmann from comment #46) > Waiting for a thumbnail that is already loaded in another task is not the > cause of the problem. Since the problem cannot be reproduced here, I > currently have no idea why a task is not returning. > > And the problem can only be reproduced with "big" thumbnails? > > Maik Yes, problem appears only with big thumbnails I just did new test: I start digikam with empty image album, with medium thumbnails size set I externally add one jpeg image, then I do F5, *medium size* thumb is created, progress bar is displayed normally, not stalled I set thumbnail size to big -> big thumbnail is created Then I do F5 with this thumbnail, I see progress bar is stalled at 0% Debug message here: https://www.dropbox.com/s/2lz1fnc7wk8xh6y/dk-debug-thumb-2109-3.txt?dl=0
Same issue after Digikam rebuild with Cmake and extra-cmake-modules git versions
I tested again today with last git version, issue *seems solved*, F5 works normally, no stalling, no freeze, but I don't understand why... Debug message here https://www.dropbox.com/s/l50us3jygw7y6sd/dk-thumb-2609.txt?dl=0
Bad news: It seems issue is still present with .avi and .mp4 files
Please try again with the latest git/master version. Maik
Issue is present again with a jpeg file as soon i do F5 with big thumbnail size
I'm now also on Qt-5.15.6 and still can't reproduce the problem. Also, there are no weird "Not Supported" messages here. Maik
(In reply to Maik Qualmann from comment #53) > I'm now also on Qt-5.15.6 and still can't reproduce the problem. Also, there > are no weird "Not Supported" messages here. > > Maik I can reproduce issue with Qt4 and Qt-5.15.6 too... (qt5-webengine package is 5.15.10-3)
Git commit 2fe9fed6ec605438baa729aa02141827f1fdb940 by Maik Qualmann. Committed on 08/10/2022 at 17:53. Pushed by mqualmann into branch 'master'. rewrite code of the Maintenance Tool thumbnail task M +47 -22 core/utilities/maintenance/thumbstask.cpp M +5 -0 core/utilities/maintenance/thumbstask.h https://invent.kde.org/graphics/digikam/commit/2fe9fed6ec605438baa729aa02141827f1fdb940
Now, this issue concerning big thumbnails seems solved. Progress bar is not freezed after doing F5 but many "Not supported" messages Debug here https://www.dropbox.com/s/l50us3jygw7y6sd/dk-thumb-2609.txt?dl=0