Bug 305359 - Advanced rename of lots of images is slow
Summary: Advanced rename of lots of images is slow
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: AdvancedRename-file (show other bugs)
Version: 2.5.0
Platform: openSUSE Linux
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-08-17 22:37 UTC by Sven Heithecker
Modified: 2022-01-31 21:57 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In: 4.3.0


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sven Heithecker 2012-08-17 22:37:40 UTC
I often have to rename a lot of images. I noticed that the renaming function gets extremly slow after the 1st ~20 images, e.g it needs several minutes to rename ~340 images.  While performing the rename operation, digiKam continuously tries to update the album/folder view.
I am unsure if this matters, but DigiKam is configured to use "SQLite" internal data base.
I admit that this is not really a bug, but I think that renaming should be faster, maybe by disabling the album update until the renaming has finished.


Reproducible: Always

Steps to Reproduce:
1. Create several hundreds of images in one folder/album.
2. Rename all images using the "rename" function including the "###" placeholder into the new file name.
Actual Results:  
Renaming takes several minutes, while digiKam continuously tries to update the album/folder view.

Expected Results:  
The renaming should be faster, maybe by disabling the album update until the renaming has finished.
Comment 1 Kenny 2012-08-25 16:38:00 UTC
I can confirm, in particular, trying to renaming a large numbers of images over a slow support (like SD) request a lot of time. Probably, like Sven said, the problem is the real-time update of the album.
Comment 2 Andi Clemens 2012-09-15 13:06:36 UTC
I guess this is a general digiKam problem, not renaming in particular. As soon as image data or metadata information is changed, digiKam updates a lot of stuff in the background.

Gilles, Marcel,

is there a way to avoid updating immediately?
Comment 3 Marcel Wiesweg 2012-09-23 14:01:12 UTC
Yes, by introducing delays...which is not good as changes are visible later.

The root of many problems is that any write operation to Sqlite triggers a call to fsync(), which is very slow and the main bottleneck in this setup. This is a principal problem. Newer development like Sqlite WAL are steps in the right direction, but WAL in my experience does not solve the problem.
You can of course switch off the "atomic" mode for you db, but I didn't dare doing this so far.

I've recently changed the ImageScanner optimizing around that problem so that there is only one write-to-disk per changed file.

Regarding the problem at hand, we should check console output during the rename operation, that for each file there's a "recognized as identical..." message and the file is not actually rescanned.
We should check if the problem is CPU or harddisk, if it's CPU, confirm it is the main digikam process. If it is, valgrind callgrind will be interesting.
Comment 4 Gregor Müllegger 2013-02-26 20:58:26 UTC
I tried a SQLite and a MySQL database. It seems even worse with MySQL. When I rename a bunch of images with a SQLite database, it takes about ~1 second per file. With MySQL it seems to rename a few files (about three) rapidly but then needs 4-5 seconds for the next few files and so on.

It definitly seems CPU bound, digikam hogs one of the cores but IO seems pretty low. If using mysql, mysql will also use a full core for during renaming. Haven't valigrind'ed anything yet... I hope to find out how to do this to provide some usefull feedback.

I wonder if none of the core developers have this issue?
Comment 5 Gregor Müllegger 2013-04-07 13:29:16 UTC
I reseted my DB (i.e. I deleted it) and all the configuration files of digiKam. This solved the issue for me, it was much faster after that and is even faster since the last update I got via kubuntu backports.

I didn't lose too much data by this since I synced the tags with the in-file meta data.
Comment 6 Frank Steinmetzger 2013-09-14 20:42:31 UTC
*** Bug 318363 has been marked as a duplicate of this bug. ***
Comment 7 caulier.gilles 2014-08-31 13:31:01 UTC
Following comment #5, problem was relevant of a broken database. 

Gilles Caulier