Bug 303546 - MYSQL : re-read metadata from images is very slow
Summary: MYSQL : re-read metadata from images is very slow
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Database-Mysql (show other bugs)
Version: 2.7.0
Platform: Unlisted Binaries Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-07-15 10:41 UTC by Michael Holtermann
Modified: 2016-07-13 18:57 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 5.1.0


Attachments
MySQL Global Status and System Variables (36.42 KB, text/plain)
2012-07-15 17:47 UTC, Michael Holtermann
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Holtermann 2012-07-15 10:41:55 UTC
Hello,

because of a crash of my digikam database I have to re-import the metadata from round about 22.000 images back into the MySQL database. I'm using the Ubuntu PPAs built by Philip on an AMD 64x2 4200+ with 6GB of RAM.

Unfortunately, the process is very slow, shifting round about 1000 pictures per hour, only.

top shows 84% CPU at mysqld and 35% at digikam, and 3% of MEM, both. iotop shows nothing.

I guess the meta data reading is done by single kio_digikamtags? I see these kind of process starting, living for ~6 seconds, and dying.

Any help or tuning hints would be appreciated....

Kind regards,
Michael

Reproducible: Always

Steps to Reproduce:
1. Select 1000 pictures
2. Select "re-read metadata from images"
3. waiting...
Comment 1 Marcel Wiesweg 2012-07-15 17:33:57 UTC

> top shows 84% CPU at mysqld and 35% at digikam, and 3% of MEM, both. iotop
> shows nothing.

This would indicate the db side as bottleneck, but can be misleading. It will be quite difficult to benchmark, there are two processes involved and at both ends CPU or harddisk I/O can be limiting.

> 
> I guess the meta data reading is done by single kio_digikamtags? I see these
> kind of process starting, living for ~6 seconds, and dying.

No, this is just updating the tags counter. Scanning is done in a thread of the main process.
Comment 2 Michael Holtermann 2012-07-15 17:47:07 UTC
Hello Marcel,

thank you for your reply. I'm running mysql-workbench in parallel, but I'm affraid I'm far away from able to understand the values :-/

I'll attach the status and system variables, assuming that there is something hidden inside...

Kind regards,
Michael
Comment 3 Michael Holtermann 2012-07-15 17:47:40 UTC
Created attachment 72538 [details]
MySQL Global Status and System Variables
Comment 4 caulier.gilles 2016-07-06 20:35:32 UTC
This file still valid using last digiKam 5.0.0 ?

Gilles Caulier
Comment 5 Michael Holtermann 2016-07-12 18:21:28 UTC
Sorry, don't know, I'm on Fedora 23 with Digikam 4.14, currently. I guess it's a rare use case that did not gain much attraction, so I think it's fair to close this ticket.