Bug 300293 - SCAN : when starting digiKam, it takes about 5 minutes
Summary: SCAN : when starting digiKam, it takes about 5 minutes
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Database-Scan (show other bugs)
Version: 2.5.0
Platform: Ubuntu Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-05-19 06:31 UTC by Axel Krebs
Modified: 2021-12-29 14:17 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 7.5.0


Attachments
starting up phase (a) (100.71 KB, image/jpeg)
2012-05-19 06:34 UTC, Axel Krebs
Details
manyImages - create many images in multiple folders (17.50 KB, application/octet-stream)
2012-05-19 12:06 UTC, Andi Clemens
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Axel Krebs 2012-05-19 06:31:19 UTC
when starting digiKam, it takes very long times. 
checking "system monitor", it appears that only one (of four) cores are active.

Reproducible: Always

Steps to Reproduce:
1. just start digiKam
2. and wait
3. and wait
4. and wait...


Expected Results:  
didgiKam should speed up, use all available cores (if possible)
Comment 1 Axel Krebs 2012-05-19 06:34:12 UTC
Created attachment 71210 [details]
starting up phase (a)
Comment 2 Axel Krebs 2012-05-19 06:35:47 UTC
digiKam version 2.5.0
Bilder: 
BMP: 1
GIF: 100
JP2: 16
JPEG: 28
JPG: 107777
NIKON-NEF: 27
PGF: 2
PNG: 1344
RAW-CR2: 632
RAW-CRW: 15778
RAW-DNG: 6
RAW-NEF: 61961
TIFF: 1561
XCF: 1
Gesamt: 189234
: 
: 
Videos: 
AVI: 73
MOV: 16
MPEG: 2
Gesamt: 91
: 
: 
Gesamtzahl der Einträge: 189325
Alben: 2616
Stichwörter: 79
Datenbanktreiber: QSQLITE
Comment 3 Andi Clemens 2012-05-19 12:05:31 UTC
I can confirm this, digiKam is not usable with bigger image collections.
I created a little test program for generating small images (attached to this bugreport).
When I create the test images and start digiKam (say 100.000 images, 1000 images in each folder), digiKam seems to take more than 8 hours to scan all the images (I stopped at 20% after 2 hours).
The UI is visible, the "scan collection task" is running, but it is so damn slow.
I commented out the slow code (ImageScanner::scanFromIdenticalFile() always returning true) and digiKam was ready after 3 minutes, but I needed to kill the process because it wasted more than 2.5 GB RAM! I don't know what happened but after the scan finished, it started to increase memory usage like crazy.

We seem to have big issues with bigger image databases here... btw. I was using the SQLITE backend.
Comment 4 Andi Clemens 2012-05-19 12:06:20 UTC
Created attachment 71217 [details]
manyImages  - create many images in multiple folders
Comment 5 Andi Clemens 2012-05-19 12:13:13 UTC
I just deleted all these images again, now digiKam is starting and "searching for deleted albums".
The problem is I can not cancel this action. I guess now I really have to wait for 8 hours, because it wants to check all of the 100.000 images one by one...
Comment 6 caulier.gilles 2012-05-19 13:44:07 UTC
Andi, 

Do you use 2.6.0 RC ? Because i cannot reproduce it here...

Gilles
Comment 7 Andi Clemens 2012-05-19 14:03:11 UTC
Yes I do

2012/5/19 Gilles Caulier <caulier.gilles@gmail.com>:
> https://bugs.kde.org/show_bug.cgi?id=300293
>
> --- Comment #6 from Gilles Caulier <caulier.gilles@gmail.com> ---
> Andi,
>
> Do you use 2.6.0 RC ? Because i cannot reproduce it here...
>
> Gilles
>
> --
> You are receiving this mail because:
> You are the assignee for the bug.
> _______________________________________________
> Digikam-devel mailing list
> Digikam-devel@kde.org
> https://mail.kde.org/mailman/listinfo/digikam-devel
Comment 8 Pierre Hanser 2012-05-19 16:15:56 UTC
may be totally out of subject:
recursive search for color profile may lengthen the startup of digikam
Comment 9 Axel Krebs 2012-05-19 16:17:05 UTC
(In reply to comment #7)
> Yes I do
> 
> 2012/5/19 Gilles Caulier <caulier.gilles@gmail.com>:
> > https://bugs.kde.org/show_bug.cgi?id=300293
> >
> > --- Comment #6 from Gilles Caulier <caulier.gilles@gmail.com> ---
> > Andi,
> >
> > Do you use 2.6.0 RC ? Because i cannot reproduce it here...
> >
> > Gilles
> >
> > --
> > You are receiving this mail because:
> > You are the assignee for the bug.
> > _______________________________________________
> > Digikam-devel mailing list
> > Digikam-devel@kde.org
> > https://mail.kde.org/mailman/listinfo/digikam-devel

Hi Gilles:

What means "here"? > "=under 2.6 RC?"

When repeating "building thumbnails", "search" (instead of building _all_ thumbnails again), it take again 10:12:06 [hh:mm:ss].

Question: is this related to your considetion on separating into severl files <http://dot.kde.org/2012/02/22/digikam-team-meets-genoa-italy>, second paragraph? 

Without knowing interal details of digiKam, I would try building a listing of pic types and _not_ check every pic file individually. instead, I would consign the spoecoific metadata of each pic file type. 

I guess, it is quicker to check listsings than prooving files 180000 times...

Hope I am not to far away... ;-) !
Comment 10 caulier.gilles 2014-08-05 13:24:30 UTC
Axel,

What's new about this file using last digiKAm 4.2.0 ?

Gilles Caulier
Comment 11 caulier.gilles 2014-08-07 12:20:03 UTC

*** This bug has been marked as a duplicate of bug 281959 ***
Comment 12 caulier.gilles 2021-12-29 14:17:26 UTC
Fixed with #281959