Bug 300293

Created attachment 71210 [details]
starting up phase (a)

digiKam version 2.5.0
Bilder: 
BMP: 1
GIF: 100
JP2: 16
JPEG: 28
JPG: 107777
NIKON-NEF: 27
PGF: 2
PNG: 1344
RAW-CR2: 632
RAW-CRW: 15778
RAW-DNG: 6
RAW-NEF: 61961
TIFF: 1561
XCF: 1
Gesamt: 189234
: 
: 
Videos: 
AVI: 73
MOV: 16
MPEG: 2
Gesamt: 91
: 
: 
Gesamtzahl der Einträge: 189325
Alben: 2616
Stichwörter: 79
Datenbanktreiber: QSQLITE

I can confirm this, digiKam is not usable with bigger image collections.
I created a little test program for generating small images (attached to this bugreport).
When I create the test images and start digiKam (say 100.000 images, 1000 images in each folder), digiKam seems to take more than 8 hours to scan all the images (I stopped at 20% after 2 hours).
The UI is visible, the "scan collection task" is running, but it is so damn slow.
I commented out the slow code (ImageScanner::scanFromIdenticalFile() always returning true) and digiKam was ready after 3 minutes, but I needed to kill the process because it wasted more than 2.5 GB RAM! I don't know what happened but after the scan finished, it started to increase memory usage like crazy.

We seem to have big issues with bigger image databases here... btw. I was using the SQLITE backend.

Created attachment 71217 [details]
manyImages  - create many images in multiple folders

I just deleted all these images again, now digiKam is starting and "searching for deleted albums".
The problem is I can not cancel this action. I guess now I really have to wait for 8 hours, because it wants to check all of the 100.000 images one by one...

Andi, 

Do you use 2.6.0 RC ? Because i cannot reproduce it here...

Gilles

Yes I do

2012/5/19 Gilles Caulier <caulier.gilles@gmail.com>:
> https://bugs.kde.org/show_bug.cgi?id=300293
>
> --- Comment #6 from Gilles Caulier <caulier.gilles@gmail.com> ---
> Andi,
>
> Do you use 2.6.0 RC ? Because i cannot reproduce it here...
>
> Gilles
>
> --
> You are receiving this mail because:
> You are the assignee for the bug.
> _______________________________________________
> Digikam-devel mailing list
> Digikam-devel@kde.org
> https://mail.kde.org/mailman/listinfo/digikam-devel

may be totally out of subject:
recursive search for color profile may lengthen the startup of digikam

(In reply to comment #7)
> Yes I do
> 
> 2012/5/19 Gilles Caulier <caulier.gilles@gmail.com>:
> > https://bugs.kde.org/show_bug.cgi?id=300293
> >
> > --- Comment #6 from Gilles Caulier <caulier.gilles@gmail.com> ---
> > Andi,
> >
> > Do you use 2.6.0 RC ? Because i cannot reproduce it here...
> >
> > Gilles
> >
> > --
> > You are receiving this mail because:
> > You are the assignee for the bug.
> > _______________________________________________
> > Digikam-devel mailing list
> > Digikam-devel@kde.org
> > https://mail.kde.org/mailman/listinfo/digikam-devel

Hi Gilles:

What means "here"? > "=under 2.6 RC?"

When repeating "building thumbnails", "search" (instead of building _all_ thumbnails again), it take again 10:12:06 [hh:mm:ss].

Question: is this related to your considetion on separating into severl files <http://dot.kde.org/2012/02/22/digikam-team-meets-genoa-italy>, second paragraph? 

Without knowing interal details of digiKam, I would try building a listing of pic types and _not_ check every pic file individually. instead, I would consign the spoecoific metadata of each pic file type. 

I guess, it is quicker to check listsings than prooving files 180000 times...

Hope I am not to far away... ;-) !

Axel,

What's new about this file using last digiKAm 4.2.0 ?

Gilles Caulier

*** This bug has been marked as a duplicate of bug 281959 ***

Fixed with #281959