Bug 229888

For my 30000 pictures, the scan time is only a few secs. Only good profiling results with cachegrind or OProfile can help us here.
There were also reports about problems with sqlite on ext4, but I am also running ext4, so it cannot be a general problem.

Marcel,

I suppose you have barriers turned off, because otherwise you would have noticed a huge difference ;-) But the same is true for XFS and btrfs... all filesystems with barriers suffer from the performance loss.

But couldn't digiKam just use the system's "find" command to check whether there are any new pictures? In a most primitive implementation digiKam would not even need to use the find results, but just start doing its own search for new pictures only in case "find" finds something (which at least on my computer would not be true in most cases anyway).

I don't know what "barriers" in file systems are, but I've got my pictures on a hfsplus file system, so I can also access them from Mac OS X. My computer is a 2003 model, iMac G4 1GHz (with 1.5 GByte RAM and a relatively new 2.5" internal hard drive).

Here are again some digiKam startup times on my system:

30 seconds (1st start, not scanning for new items)
45 seconds (2nd start, scanning for new items)
20 seconds (3rd start, not scanning for new items)
40 seconds (4th start, scanning for new items)

So it is quite obvious that scanning for new items consumes about 20 seconds, which can be about 50% of the overall startup time, if digiKam had been loaded before. "find", as I wrote before, initially took just about 4 seconds scanning through my 13,000 pictures, and takes less than 1 second now that digiKam scanned through the pictures (files) already...

Any approach will involve reading (stat'ing) the directories of the collections, and not more. Like calling "find .".
Digikam gets the modification date of the files in a directory from its database and compares these.
When find . is fast for you, it's more probably a database speed issue on your system and filesystem. But we cannot be sure without profiling.

Created attachment 41573 [details]
Script to make digiKam startup faster

I did some additional tests, even copied my pictures and database to ext3 partitions, but the scan still takes about 20 seconds. I guess this is just because the G4 really is a rather slow CPU (if its "AltiVec" instructions are not used). I also checked the startup speed on my AMD64 notebook computer for comparison, and just as you, Marcel, wrote it, the scan just takes a few seconds, about two, actually (that's with digiKam beta-5 in Kubuntu).

DigiKam startup under MacOS (with digiKam 1.0-rc) was even slower than on Linux:
99 seconds (including 29 seconds scanning for new images) on the first start, and 60 seconds (including 22 seconds scanning for new images) when I shut it down and started it again.

But is digiKam checking the file dates against different dates from its database? If yes, why doesn't it just take the last startup date as a reference? It seems like digiKam always writes its config file, so that file date could simply be used as a reference.

For my purposes I wrote a shell script that makes digiKam only check for new images if "find" finds new images. I'll attach that. Maybe someone else with an old computer can compare the find command's performance to the normal digiKam scan duration. I'm sorry the attachment was committed to this bug before the comment.

Vlado,

This script still valid using digiKam 2.x Database schema ?

Gilles Caulier

Sorry, Gilles, but I'm not using that old PowerPC computer any more (at least not with digiKam). My new computer is an AMD Fusion E-350 - still quite slow (and also my 5000 RPM hard drive), but repeated starts take just about 15 seconds, no matter whether it has to scan for images or not. At most the scan takes about 2 additional seconds. Currently I'm also using the MySQL database, in case that makes a difference.