Bug 289204

Summary:	General performance problems because of lacking multithreading / multicore support
Product:	[Applications] digikam	Reporter:	wuselwu <einmaladresse_2>
Component:	Thumbs-Engine	Assignee:	Digikam Developers <digikam-bugs-null>
Status:	RESOLVED FIXED
Severity:	wishlist	CC:	caulier.gilles, domlyons, marcel.wiesweg, oliver.koch
Priority:	NOR
Version:	2.3.0
Target Milestone:	---
Platform:	openSUSE
OS:	Linux
Latest Commit:		Version Fixed In:	4.3.0
Sentry Crash Report:

Description wuselwu 2011-12-17 14:56:55 UTC

Version:           2.3.0 (using KDE 4.7.3) 
OS:                Linux

The release announcement of Digikam 1.2.0 claims the implementation of multithreading support. However, as of 2.3.0 very basic components of Digikam still are seemingly lacking real multicore support for modern CPUs. 

This leads to (very...) poor performance especially when handling larger albums with hi-res photos.

Examples (on a Phenom quadcore with 2.4 Ghz each core):

Thumbnail generation definitely only uses one processor core. If it is running in the background, Digikam can get unusable for long periods of time. This is especially annoying as simple operations like "changing time/date of photos" lead to a void rebuild of the thumbnails. (Should I file this as a seperate bug?)
- Time consuming effects (like local contrast) are obviously only single-core.
- In this I'm not quite sure, but sometimes viewing pictures takes so much time that I wonder if the picture viewer is also only single threaded.

Generally I have the feeling that during Digikam operations only one processor core is used most of the time (according to sysinfo plasmoid and top).

Reproducible: Didn't try

Steps to Reproduce:
Do anything that requires e.g. rebuilding thumbnails in a bigger album.

Actual Results:  
Digikam gets sluggish and hangs for minutes. Only one processor cores is working, albeit at 100%.

Expected Results:  
Smoothly background workload done by all processor cores.

Comment 1 caulier.gilles 2011-12-17 16:07:10 UTC

*** Bug 160873 has been marked as a duplicate of this bug. ***

Comment 2 caulier.gilles 2013-08-09 12:54:31 UTC

Git commit 9ba77fdbce49247df206d9f26251d82e1558348a by Gilles Caulier.
Committed on 09/08/2013 at 12:50.
Pushed by cgilles into branch 'master'.

Maintenance Tool : Add Multi-core support for Metadata Synchronizer tool.
Now, metadata synchronization between database and files (and vis-versa) is multi-threaded and use multi-core CPU.
This do not freeze GUI as previous in previous versions and speed to process files is drasticaly increased.

M  +2    -0    CMakeLists.txt
M  +23   -35   utilities/maintenance/metadatasynchronizer.cpp
M  +2    -0    utilities/maintenance/metadatasynchronizer.h
A  +106  -0    utilities/maintenance/metadatatask.cpp     [License: GPL (v2+)]
A  +76   -0    utilities/maintenance/metadatatask.h     [License: GPL (v2+)]
A  +119  -0    utilities/maintenance/metadatathread.cpp     [License: GPL (v2+)]
A  +82   -0    utilities/maintenance/metadatathread.h     [License: GPL (v2+)]

http://commits.kde.org/digikam/9ba77fdbce49247df206d9f26251d82e1558348a

Comment 3 caulier.gilles 2013-08-09 13:08:39 UTC

This is a list of features supported by In Maintenance tools :

* Scan for new items : 
- multi-threaded : yes
- multi-core CPU : no (*)

* Rebuild Thumbnails :
- multi-threaded : yes
- multi-core CPU : no (TODO using ThreadWeaver)

* Rebuild Finger-prints :
- multi-threaded : yes
- multi-core CPU : no (TODO using ThreadWeaver)

* Find Duplicates Items :
- multi-threaded : no (KIO-SLAVE)
- multi-core CPU : no (KIO-SLAVE)

* Detect and Recognize Faces :
- multi-threaded : yes
- multi-core CPU : yes

* Sync Metadata and DB :
- multi-threaded : yes
- multi-core CPU : yes

Marcel, can you comment (*) about multi-core support plan in ScanController & co. Thanks in advance...

Gilles Caulier

Comment 4 caulier.gilles 2013-08-14 15:58:35 UTC

Git commit 48b47b5402366ba05c400ac2996fc26535601639 by Gilles Caulier.
Committed on 14/08/2013 at 15:57.
Pushed by cgilles into branch 'master'.

Maintenance tool : Thumbs Generator now support multi-core CPU

M  +1    -0    CMakeLists.txt
M  +19   -2    utilities/maintenance/maintenancethread.cpp
M  +3    -0    utilities/maintenance/maintenancethread.h
M  +2    -3    utilities/maintenance/metadatatask.cpp
M  +25   -57   utilities/maintenance/thumbsgenerator.cpp
M  +2    -4    utilities/maintenance/thumbsgenerator.h
C  +28   -33   utilities/maintenance/thumbstask.cpp [from: utilities/maintenance/metadatatask.cpp - 050% similarity]
C  +25   -21   utilities/maintenance/thumbstask.h [from: utilities/maintenance/thumbsgenerator.h - 052% similarity]     [License: UNKNOWN]  *

The files marked with a * at the end have a non valid license. Please read: http://techbase.kde.org/Policies/Licensing_Policy and use the headers which are listed at that page.


http://commits.kde.org/digikam/48b47b5402366ba05c400ac2996fc26535601639

Comment 5 caulier.gilles 2013-08-16 05:59:44 UTC

Git commit af5d9098daf2fff82652e0f27013bec8ffe8062b by Gilles Caulier.
Committed on 16/08/2013 at 05:58.
Pushed by cgilles into branch 'master'.

MantenanceTool : FingerPrints generator is now able to use Multi-core CPU.

M  +1    -0    CMakeLists.txt
M  +23   -64   utilities/maintenance/fingerprintsgenerator.cpp
M  +4    -4    utilities/maintenance/fingerprintsgenerator.h
C  +32   -18   utilities/maintenance/fingerprintstask.cpp [from: utilities/maintenance/thumbstask.cpp - 050% similarity]
C  +31   -19   utilities/maintenance/fingerprintstask.h [from: utilities/maintenance/fingerprintsgenerator.h - 057% similarity]
M  +19   -2    utilities/maintenance/maintenancethread.cpp
M  +5    -3    utilities/maintenance/thumbsgenerator.cpp
M  +3    -1    utilities/maintenance/thumbstask.cpp

http://commits.kde.org/digikam/af5d9098daf2fff82652e0f27013bec8ffe8062b

Comment 6 caulier.gilles 2014-05-01 20:38:30 UTC

Git commit c7a52e8fe575e2e4b527416f245ca7e056296855 by Gilles Caulier.
Committed on 01/05/2014 at 20:25.
Pushed by cgilles into branch 'master'.

Blur Tool : add support of multicore CPU. Use new fast algorithm of simple blur based on original implemntation from
libkimageeffect2 code (never released) written by Daniel M Duley.
Code have been ported to DImg and to support 1§ bits color depth image of course.
CImg dependency to Blur tool have been dropped.
Old and slow Guaussian blur algorithm have been removed
CCMAIL: veaceslav.munteanu90@gmail.com

M  +1    -0    NEWS
M  +1    -1    imageplugins/TODO
M  +124  -357  libs/dimg/filters/fx/blurfilter.cpp
M  +8    -41   libs/dimg/filters/fx/blurfilter.h

http://commits.kde.org/digikam/c7a52e8fe575e2e4b527416f245ca7e056296855

Comment 7 caulier.gilles 2014-05-03 16:34:53 UTC

Git commit cb3bec4e49d903288335d1f1eeccc5141cb86878 by Gilles Caulier.
Committed on 03/05/2014 at 16:34.
Pushed by cgilles into branch 'master'.

Charcoal Tool : add support of multicore CPU based on QtConcurrent API.

M  +1    -0    NEWS
M  +1    -1    imageplugins/TODO
M  +1    -0    libs/dimg/filters/fx/blurfilter.cpp
M  +0    -2    libs/dimg/filters/fx/blurfilter.h
M  +113  -66   libs/dimg/filters/fx/charcoalfilter.cpp
M  +4    -3    libs/dimg/filters/fx/charcoalfilter.h

http://commits.kde.org/digikam/cb3bec4e49d903288335d1f1eeccc5141cb86878

Comment 8 caulier.gilles 2014-05-04 07:34:45 UTC

Git commit b451f671f393ebac6ea09957fa8e9635a1ceeb3f by Gilles Caulier.
Committed on 04/05/2014 at 07:34.
Pushed by cgilles into branch 'master'.

OilPaint Tool : add support of multicore CPU based on QtConcurrent API.

M  +1    -0    NEWS
M  +1    -1    imageplugins/TODO
M  +70   -51   libs/dimg/filters/fx/oilpaintfilter.cpp
M  +4    -3    libs/dimg/filters/fx/oilpaintfilter.h

http://commits.kde.org/digikam/b451f671f393ebac6ea09957fa8e9635a1ceeb3f

Comment 9 caulier.gilles 2014-05-05 14:11:45 UTC

Git commit 5776c84b0ab1dacdd99cbac606e659c252484a11 by Gilles Caulier.
Committed on 05/05/2014 at 14:11.
Pushed by cgilles into branch 'master'.

Emboss Tool : add support of multicore CPU based on QtConcurrent API.

M  +1    -0    NEWS
M  +1    -1    imageplugins/TODO
M  +73   -38   libs/dimg/filters/fx/embossfilter.cpp
M  +2    -1    libs/dimg/filters/fx/embossfilter.h

http://commits.kde.org/digikam/5776c84b0ab1dacdd99cbac606e659c252484a11

Comment 10 caulier.gilles 2014-05-05 14:46:14 UTC

Git commit ed81651f82024885f0cd3d206587b3c691a45b10 by Gilles Caulier.
Committed on 05/05/2014 at 14:45.
Pushed by cgilles into branch 'master'.

Raindrop Tool : add support of multicore CPU based on QtConcurrent API.

M  +1    -0    NEWS
M  +1    -1    imageplugins/TODO
M  +63   -29   libs/dimg/filters/fx/raindropfilter.cpp
M  +18   -1    libs/dimg/filters/fx/raindropfilter.h

http://commits.kde.org/digikam/ed81651f82024885f0cd3d206587b3c691a45b10

Comment 11 caulier.gilles 2014-05-06 12:29:48 UTC

Git commit cee47d42b0b6313e3f37a45b383c437a24929458 by Gilles Caulier.
Committed on 06/05/2014 at 12:29.
Pushed by cgilles into branch 'master'.

FilmGrain Tool : add support of multicore CPU based on QtConcurrent API.

M  +82   -35   libs/dimg/filters/fx/filmgrainfilter.cpp
M  +2    -0    libs/dimg/filters/fx/filmgrainfilter.h

http://commits.kde.org/digikam/cee47d42b0b6313e3f37a45b383c437a24929458

Comment 12 caulier.gilles 2014-05-06 13:52:51 UTC

Git commit c63d1ffd627958d2dff8b89a704ca132740daa70 by Gilles Caulier.
Committed on 06/05/2014 at 13:52.
Pushed by cgilles into branch 'master'.

NoiseReduction Tool : add support of multicore CPU based on QtConcurrent API.

M  +1    -0    NEWS
M  +1    -1    imageplugins/TODO
M  +137  -248  libs/dimg/filters/nr/nrfilter.cpp
M  +21   -9    libs/dimg/filters/nr/nrfilter.h

http://commits.kde.org/digikam/c63d1ffd627958d2dff8b89a704ca132740daa70

Comment 13 caulier.gilles 2014-05-06 14:31:57 UTC

Git commit 62e99542a3edf1a6b383b116ffaeee15403b6987 by Gilles Caulier.
Committed on 06/05/2014 at 14:30.
Pushed by cgilles into branch 'master'.

add info about CPU cores available in Information Dialog

M  +5    -1    libs/dialogs/libsinfodlg.cpp
M  +1    -1    libs/dialogs/libsinfodlg.h

http://commits.kde.org/digikam/62e99542a3edf1a6b383b116ffaeee15403b6987

Comment 14 caulier.gilles 2014-05-07 09:24:27 UTC

Example of digiKam running in multicore with Refocus tool. 

https://www.flickr.com/photos/digikam/14099672266/sizes/o

Gilles Caulier

Comment 15 caulier.gilles 2014-05-07 12:35:54 UTC

Git commit 520094d5daa86b5dee1d02dd482d762a116ebeee by Gilles Caulier.
Committed on 07/05/2014 at 12:34.
Pushed by cgilles into branch 'master'.

ZoomBlur effect : add support of multicore CPU based on QtConcurrent API. Computation time is divied by 2 un i7 (8 core CPU)

M  +95   -70   libs/dimg/filters/fx/blurfxfilter.cpp
M  +16   -0    libs/dimg/filters/fx/blurfxfilter.h

http://commits.kde.org/digikam/520094d5daa86b5dee1d02dd482d762a116ebeee

Comment 16 caulier.gilles 2014-05-07 15:20:13 UTC

Git commit e5d465a5bfe4a94dcd2d1f05b195e7ff97601d37 by Gilles Caulier.
Committed on 07/05/2014 at 15:18.
Pushed by cgilles into branch 'master'.

RadialBlur effect : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 5 with i7 (8 core CPU)

M  +102  -75   libs/dimg/filters/fx/blurfxfilter.cpp
M  +3    -2    libs/dimg/filters/fx/blurfxfilter.h

http://commits.kde.org/digikam/e5d465a5bfe4a94dcd2d1f05b195e7ff97601d37

Comment 17 caulier.gilles 2014-05-07 17:20:26 UTC

Git commit c94c9f4aa3c670f82c930c00a0fb873d0a3a2d55 by Gilles Caulier.
Committed on 07/05/2014 at 17:19.
Pushed by cgilles into branch 'master'.

Mosaic effect : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 2 with i7 (8 core CPU)

M  +63   -37   libs/dimg/filters/fx/blurfxfilter.cpp
M  +8    -3    libs/dimg/filters/fx/blurfxfilter.h

http://commits.kde.org/digikam/c94c9f4aa3c670f82c930c00a0fb873d0a3a2d55

Comment 18 caulier.gilles 2014-05-07 21:34:09 UTC

Git commit f1498ce57db4652719b906c89ab960563908c680 by Gilles Caulier.
Committed on 07/05/2014 at 21:33.
Pushed by cgilles into branch 'master'.

FocusBlur effect : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 2 with i7 (8 core CPU)

M  +106  -77   libs/dimg/filters/fx/blurfxfilter.cpp
M  +4    -0    libs/dimg/filters/fx/blurfxfilter.h

http://commits.kde.org/digikam/f1498ce57db4652719b906c89ab960563908c680

Comment 19 caulier.gilles 2014-05-08 10:31:59 UTC

Git commit 5909360bdf68b20b75689e17a4990e2e45fd0f82 by Gilles Caulier.
Committed on 08/05/2014 at 10:30.
Pushed by cgilles into branch 'master'.

ShakeBlur effect : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 2 with i7 (8 core CPU)

M  +111  -57   libs/dimg/filters/fx/blurfxfilter.cpp
M  +9    -0    libs/dimg/filters/fx/blurfxfilter.h

http://commits.kde.org/digikam/5909360bdf68b20b75689e17a4990e2e45fd0f82

Comment 20 caulier.gilles 2014-05-08 12:28:48 UTC

Git commit 27ec1240dac3cae3bb4bb8f0d5cef1f6f32bbf7f by Gilles Caulier.
Committed on 08/05/2014 at 12:28.
Pushed by cgilles into branch 'master'.

MotionBlur effect : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 4 with i7 (8 core CPU)

M  +81   -55   libs/dimg/filters/fx/blurfxfilter.cpp
M  +7    -0    libs/dimg/filters/fx/blurfxfilter.h

http://commits.kde.org/digikam/27ec1240dac3cae3bb4bb8f0d5cef1f6f32bbf7f

Comment 21 caulier.gilles 2014-05-08 13:08:59 UTC

Git commit dda58640422eba71e3e57308402193f241a17dcc by Gilles Caulier.
Committed on 08/05/2014 at 13:08.
Pushed by cgilles into branch 'master'.

SoftenerBlur effect : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 2 with i7 (8 core CPU)

M  +103  -77   libs/dimg/filters/fx/blurfxfilter.cpp
M  +1    -0    libs/dimg/filters/fx/blurfxfilter.h

http://commits.kde.org/digikam/dda58640422eba71e3e57308402193f241a17dcc

Comment 22 caulier.gilles 2014-05-08 13:36:29 UTC

Git commit d017ee3a62257934fed9fd14c161d87241f289a7 by Gilles Caulier.
Committed on 08/05/2014 at 13:35.
Pushed by cgilles into branch 'master'.

FarBlur effect : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 3 with i7 (8 core CPU)

M  +182  -120  libs/dimg/filters/fx/blurfxfilter.cpp
M  +8    -0    libs/dimg/filters/fx/blurfxfilter.h

http://commits.kde.org/digikam/d017ee3a62257934fed9fd14c161d87241f289a7

Comment 23 caulier.gilles 2014-05-08 17:41:11 UTC

Git commit 32b43547027f5bca080f40779ae2f4e8b544958a by Gilles Caulier.
Committed on 08/05/2014 at 17:40.
Pushed by cgilles into branch 'master'.

SmartBlur effect : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 3 with i7 (8 core CPU)

M  +181  -125  libs/dimg/filters/fx/blurfxfilter.cpp
M  +4    -1    libs/dimg/filters/fx/blurfxfilter.h

http://commits.kde.org/digikam/32b43547027f5bca080f40779ae2f4e8b544958a

Comment 24 caulier.gilles 2014-05-09 14:31:52 UTC

Git commit 066ba35da5c88b5c8a1b250e8876c0a2beea414d by Gilles Caulier.
Committed on 09/05/2014 at 14:30.
Pushed by cgilles into branch 'master'.

FishEyes and Caricature effects : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 2 with i7 (8 core CPU)

M  +85   -59   libs/dimg/filters/fx/distortionfxfilter.cpp
M  +2    -2    libs/dimg/filters/fx/distortionfxfilter.h

http://commits.kde.org/digikam/066ba35da5c88b5c8a1b250e8876c0a2beea414d

Comment 25 caulier.gilles 2014-05-09 14:42:17 UTC

Git commit cb3746289f4a34ea9defb2032d046e1e9098c400 by Gilles Caulier.
Committed on 09/05/2014 at 14:41.
Pushed by cgilles into branch 'master'.

Twirl effect : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 2 with i7 (8 core CPU)

M  +84   -58   libs/dimg/filters/fx/distortionfxfilter.cpp
M  +3    -0    libs/dimg/filters/fx/distortionfxfilter.h

http://commits.kde.org/digikam/cb3746289f4a34ea9defb2032d046e1e9098c400

Comment 26 caulier.gilles 2014-05-09 14:54:57 UTC

Git commit c646df77f34e1f015a1f8c366a5669f8f0ce6130 by Gilles Caulier.
Committed on 09/05/2014 at 14:53.
Pushed by cgilles into branch 'master'.

Cylindricals effects : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 2 with i7 (8 core CPU)

M  +88   -58   libs/dimg/filters/fx/distortionfxfilter.cpp
M  +4    -0    libs/dimg/filters/fx/distortionfxfilter.h

http://commits.kde.org/digikam/c646df77f34e1f015a1f8c366a5669f8f0ce6130

Comment 27 caulier.gilles 2014-05-09 15:02:51 UTC

Git commit c974655d4c36ba482e7f8de8892453b419c74a29 by Gilles Caulier.
Committed on 09/05/2014 at 15:02.
Pushed by cgilles into branch 'master'.

Multiple Corners effect : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 2 with i7 (8 core CPU)

M  +58   -32   libs/dimg/filters/fx/distortionfxfilter.cpp
M  +3    -0    libs/dimg/filters/fx/distortionfxfilter.h

http://commits.kde.org/digikam/c974655d4c36ba482e7f8de8892453b419c74a29

Comment 28 caulier.gilles 2014-05-09 21:35:12 UTC

Git commit 90decd50e2983406c3a71132972ef867099b6c64 by Gilles Caulier.
Committed on 09/05/2014 at 21:28.
Pushed by cgilles into branch 'master'.

BlockWaves effects : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 2 with i7 (8 core CPU)

M  +65   -38   libs/dimg/filters/fx/distortionfxfilter.cpp
M  +7    -0    libs/dimg/filters/fx/distortionfxfilter.h

http://commits.kde.org/digikam/90decd50e2983406c3a71132972ef867099b6c64

Comment 29 caulier.gilles 2014-05-09 21:54:52 UTC

Git commit 4cd66508a93a9ebe5e3be895c526da4021fd02e7 by Gilles Caulier.
Committed on 09/05/2014 at 21:54.
Pushed by cgilles into branch 'master'.

CircularWaves effects : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 2 with i7 (8 core CPU)

M  +61   -31   libs/dimg/filters/fx/distortionfxfilter.cpp
M  +6    -0    libs/dimg/filters/fx/distortionfxfilter.h

http://commits.kde.org/digikam/4cd66508a93a9ebe5e3be895c526da4021fd02e7

Comment 30 caulier.gilles 2014-05-10 09:06:47 UTC

Git commit c1bec9e2f94881fd01c3700df184868f018fcac0 by Gilles Caulier.
Committed on 10/05/2014 at 09:06.
Pushed by cgilles into branch 'master'.

Waves effects : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 3 with i7 (8 core CPU)

M  +98   -44   libs/dimg/filters/fx/distortionfxfilter.cpp
M  +3    -0    libs/dimg/filters/fx/distortionfxfilter.h

http://commits.kde.org/digikam/c1bec9e2f94881fd01c3700df184868f018fcac0

Comment 31 caulier.gilles 2014-05-10 09:47:58 UTC

Git commit 70ed071c0a7173e76c1d946ab3fc0770fc60a43b by Gilles Caulier.
Committed on 10/05/2014 at 09:47.
Pushed by cgilles into branch 'master'.

Polar effects : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 2 with i7 (8 core CPU)

M  +76   -49   libs/dimg/filters/fx/distortionfxfilter.cpp
M  +2    -0    libs/dimg/filters/fx/distortionfxfilter.h

http://commits.kde.org/digikam/70ed071c0a7173e76c1d946ab3fc0770fc60a43b

Comment 32 caulier.gilles 2014-05-10 12:36:55 UTC

Git commit 57b8f644eae0d86894fb2ba1f249296ab60e108a by Gilles Caulier.
Committed on 10/05/2014 at 12:35.
Pushed by cgilles into branch 'master'.

Tile effect : add support of multicore CPU based on QtConcurrents API. Computation time is divided by 4 with i7 (8 core CPU)

M  +1    -0    NEWS
M  +1    -1    imageplugins/TODO
M  +60   -29   libs/dimg/filters/fx/distortionfxfilter.cpp
M  +4    -0    libs/dimg/filters/fx/distortionfxfilter.h

http://commits.kde.org/digikam/57b8f644eae0d86894fb2ba1f249296ab60e108a

Comment 33 caulier.gilles 2014-08-21 22:03:26 UTC

multicore support have been implemented in severals features of digiKam. I close this file now...

Gilles Caulier