openSUSE x86_64 / KDE4.12.2 In an album with around 1000 images I selected 410 images (jpeg) through the search, then place them in the BQM. Selecting the rescaling tool and start the batch process. Output goes to a new album. Original size was around 5000px width and I selected to reduce size to 1920px width. The process of rescaling itself was done in approx. 2 Minutes, but after than I recognized a lot of SQL activity which took around 30 minutes. So all in all the whole process of rescaling took > 30 minutes, while throughout this time digiKam wasn't responding in any way. Attached is an image that shows the CPU-Usage during this process. Visibly it divides in 4 blocks: First high CPU-Usage - images are rescale, after that all new images are physically there. Second block of low CPU-Usage - some SQL-Activity Third block of intermittent CPU-Usage - lots of (re)reads to the newly created files and lots of SQL-Activity. Forth block of constant CPU-Usage - moderate (re)reads to the newly created files and moderate SQL-Activity to the thumbnail-DB. Reproducible: Always Steps to Reproduce: 1. Select >200 (in my case 410) images for batch processing 2. Output goes to a new album 3. Selected tool rescale with 1920px Actual Results: Process takes much too long, digiKam is not responding during batch process Expected Results: Process should be much faster (and in the past 2.x version it was) Due to not responding for over 30 minutes the user must think the process is crashed or hangs.
Created attachment 85413 [details] CPU-Usage while batch processing
digiKam 4.0.0 is out : http://www.digikam.org/node/713 Please check if this entry still valid with this new version. Thanks in advance Gilles Caulier
Still valid - I just checked with the new package (digiKam 4.0.0-8.3) from openSUSE 13.1 / KDE 4.13. But the behavior changed a bit: digiKam is not really hanging and still shows activity, but it is very slowly responding. So you still see the status bar moving and the queue is getting refreshed. And I noticed another issue with this behavior: When you output the resulting images in a new album and wait til the task is finished, the new album shows less images then it has on disk. BTW: Nevertheless digiKam is awesome and 4.0 is really excellent - great job :-)
I run another test with MySQL DB-Backend and found some error messages in the logfile may this would be helpful: 140516 18:33:58 [Warning] Aborted connection 36 to db: 'digikamdb' user: 'digikam' host: '' (Unknown error) 140516 18:36:59 [Warning] Aborted connection 37 to db: 'digikamdb' user: 'digikam' host: '' (Unknown error) 140516 18:37:03 [Warning] Aborted connection 87 to db: 'digikamdb' user: 'digikam' host: '' (Unknown error) 140516 18:41:53 [Warning] Aborted connection 91 to db: 'digikamdb' user: 'digikam' host: '' (Unknown error) 140516 19:29:21 [Warning] Aborted connection 32 to db: 'digikamdb' user: 'digikam' host: '' (Unknown error) The last warning repeated 50 times with the same timestamp (19:29:21) at this time I shutdown digiKam
Regi, This file still valid using last digiKam 4.2.0 ? Gilles Caulier
(In reply to Gilles Caulier from comment #5) > Regi, > > This file still valid using last digiKam 4.2.0 ? > > Gilles Caulier Hi Gilles, sorry to say, but yes. While I'm trying to find out what is a "save" number of images for processing, I recognized that if I start with less images (let's say 150) - in the first run everything's fine. Repeating the process at least 3 times, the initial described issue happens. I don't know if this is pointing to a memory leak? Regi
*** Bug 346012 has been marked as a duplicate of this bug. ***
In Bug 346012 there some useful addictional info that explress that this bug affect all of BQM operations(it more visible if "use all cores" turn on and if output would be in other folder, but it affect BQM even without these conditions). I think it's all because of DB locks like digikam(9707)/digikam (core) Digikam::DatabaseCoreBackendPrivate::checkRetrySQLiteLockError: Detected locked database file. There is an active transaction. Waited but giving up now. The more these locks showed - the more application is not responding.
Created attachment 92305 [details] BatchQueueManager.patch The slow down of BQM and the locked database message I could reproduce. This patch fixes it for me. Before we close this bug, he should be tested with more than 2 CPU cores. Maik
Confirmed. Patch solve the problem here too (8 cores) Gilles
Thanks, I move the patch into branch 'master'. Maik
Git commit 901788011ff6489a63ccdbb5ade0b6438f74f0e8 by Maik Qualmann. Committed on 28/04/2015 at 19:05. Pushed by mqualmann into branch 'master'. apply patch #92305 to suspend collection scan when the BQM is running FIXED-IN: 4.10.0 M +2 -1 NEWS M +2 -0 utilities/queuemanager/main/queuemgrwindow.cpp http://commits.kde.org/digikam/901788011ff6489a63ccdbb5ade0b6438f74f0e8
Git commit 93bb098c76861316a536fbf9aef582881a03b2a4 by Gilles Caulier. Committed on 29/04/2015 at 13:15. Pushed by cgilles into branch 'frameworks'. backport commit #901788011ff6489a63ccdbb5ade0b6438f74f0e8 from git/master to frameworks branch M +2 -0 utilities/queuemanager/main/queuemgrwindow.cpp http://commits.kde.org/digikam/93bb098c76861316a536fbf9aef582881a03b2a4