Bug 303546

Summary: MYSQL : re-read metadata from images is very slow
Product: [Applications] digikam Reporter: Michael Holtermann <linux>
Component: Database-MysqlAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: caulier.gilles, swatilodha27
Priority: NOR    
Version: 2.7.0   
Target Milestone: ---   
Platform: Unlisted Binaries   
OS: Linux   
Latest Commit: Version Fixed In: 5.1.0
Attachments: MySQL Global Status and System Variables

Description Michael Holtermann 2012-07-15 10:41:55 UTC 
Hello,

because of a crash of my digikam database I have to re-import the metadata from round about 22.000 images back into the MySQL database. I'm using the Ubuntu PPAs built by Philip on an AMD 64x2 4200+ with 6GB of RAM.

Unfortunately, the process is very slow, shifting round about 1000 pictures per hour, only.

top shows 84% CPU at mysqld and 35% at digikam, and 3% of MEM, both. iotop shows nothing.

I guess the meta data reading is done by single kio_digikamtags? I see these kind of process starting, living for ~6 seconds, and dying.

Any help or tuning hints would be appreciated....

Kind regards,
Michael

Reproducible: Always

Steps to Reproduce:
1. Select 1000 pictures
2. Select "re-read metadata from images"
3. waiting...
Comment 1 Marcel Wiesweg 2012-07-15 17:33:57 UTC 

> top shows 84% CPU at mysqld and 35% at digikam, and 3% of MEM, both. iotop
> shows nothing.

This would indicate the db side as bottleneck, but can be misleading. It will be quite difficult to benchmark, there are two processes involved and at both ends CPU or harddisk I/O can be limiting.

> 
> I guess the meta data reading is done by single kio_digikamtags? I see these
> kind of process starting, living for ~6 seconds, and dying.

No, this is just updating the tags counter. Scanning is done in a thread of the main process.
Comment 2 Michael Holtermann 2012-07-15 17:47:07 UTC 
Hello Marcel,

thank you for your reply. I'm running mysql-workbench in parallel, but I'm affraid I'm far away from able to understand the values :-/

I'll attach the status and system variables, assuming that there is something hidden inside...

Kind regards,
Michael
Comment 3 Michael Holtermann 2012-07-15 17:47:40 UTC 
Created attachment 72538 [details]
MySQL Global Status and System Variables
Comment 4 caulier.gilles 2016-07-06 20:35:32 UTC 
This file still valid using last digiKam 5.0.0 ?

Gilles Caulier
Comment 5 Michael Holtermann 2016-07-12 18:21:28 UTC 
Sorry, don't know, I'm on Fedora 23 with Digikam 4.14, currently. I guess it's a rare use case that did not gain much attraction, so I think it's fair to close this ticket.