| Summary: | Digikam becomes unusable with many Metatags | ||
|---|---|---|---|
| Product: | [Applications] digikam | Reporter: | bzrudie |
| Component: | Searches-Advanced | Assignee: | Digikam Developers <digikam-bugs-null> |
| Status: | RESOLVED FIXED | ||
| Severity: | major | CC: | caulier.gilles, metzpinguin |
| Priority: | NOR | ||
| Version: | 5.7.0 | ||
| Target Milestone: | --- | ||
| Platform: | Neon | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed In: | 7.2.0 | |
| Sentry Crash Report: | |||
| Attachments: | Valgrind log | ||
|
Description
bzrudie
2017-02-24 15:32:11 UTC
Where digiKam waste time at startup ? Can you investigate using kcallgrind tool ? This will indicate which code will be called a lots of time in this situation... Gilles Caulier Created attachment 104212 [details]
Valgrind log
(In reply to caulier.gilles from comment #1) > Where digiKam waste time at startup ? Can you investigate using kcallgrind > tool ? This will indicate which code will be called a lots of time in this > situation... > > Gilles Caulier Hi Gilles, Digikam starts within a minute or two, so that is no problem for me as I usually will start it once a day. The problem comes up when I try to add/edit/delete a tag, Digikam will freeze for a whole minute until done. I just installed valgrind and started digikam like this: valgrind --tool=callgrind digikam I get lots of segment overflows like : ==5290== brk segment overflow in thread #1: can't grow to 0x4a38000 and I am waiting now 20 minutes for Digikam to start up ;) I found some info on stackoverflow: http://stackoverflow.com/questions/35129135/valgrind-reporting-a-segment-overflow still not sure if that is a problem at all. However, seems I can't even get Digikam to start up running valgrind. I killed the process and attaching the output till here, maybe it's already usefull for debugging. Thanks! new 5.6.0 pre-release as bundle is available here : https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM Please check if this problem still reproducible with these versions. Thanks in advance Gilles Caulier Thanks Gilles for looking into that. Unfortunately I think it didn't change anything. I'm now using Digikam on my more powerful MacBook Pro with SSD but still have the problem with tags management. Just to make sure I stopped some times to have something to compare if you find something other to try :) === - Startup time digikam: 3min 20 seconds (mostly "Reading Database...") - Startup time "Open Tag Manager" 23 seconds (scrolling through the list of tags is fast BTW) - Click on "+" to add a new tag, 5 seconds till window open. - Entering a new tag in that window and click save makes digikam become unresponsive for 1min and 33 seconds, doh! (e.g. digikam completely hangs) - Deleting the newly created tag takes nearly the same time, 1min and 36s === Here are some stats from my database: digikam version 5.6.0 Images: JPG: 32227 total: 32227 : Videos: AVI: 1 total: 1 : Total Items: 32228 Albums: 422 Tags: 5124 : Database backend: QSQLITE Thanks again! cheers bzrudi ok. Just to be sure : 1/ backup your sqlite database files. 2/ rename it as .old or delete files. 3/ restart digiKam and repopulate the collections from scratch. 4/ Check if problem remain. Notes : - here i use more than 150000 items with sqlite and a lots of tags without any special time latency. - 5.6.0 has a database garbage collector tool in maintenance. Thi scan be a solution to cleanup your original DB. Gilles Caulier Okay Gilles, did some more testing. Started from scratch and sadly no change, problem exists. So I checked database and started to simply delete some tags from the Tags relation. E.g. as i have more than 5000 tags I started with: sqlite> delete from Tags where id > 4500; no real change... sqlite> delete from Tags where id > 4000; no real change... sqlite> delete from Tags where id > 3000; better, tag manager now starts within 7 seconds... sqlite> delete from Tags where id > 1000; nice, tag manager starts within 1-2 seconds. I would say things work within an acceptable time. going even further and just leave 500 tags within database things rock! === - Startup time digikam: 5000+ tags 3min 20 seconds vs. 500 tags 13 seconds! - Startup time "Open Tag Manager": 5000+ 23 seconds vs. 500 tags 1 second! - Click on "+" to add a new tag: 5000+ 5 seconds vs. 500 tags realtime - Entering a new tag in that window and click save makes digikam become unresponsive for 1min and 33 seconds, doh! vs. 500 tags 2 seconds - Deleting the newly created tag takes nearly the same time, 1min and 36s vs. 500 tags 2 seconds === So things get massiv slower when reaching 2000 or 3000 tags. Feels some indexes are missing!? Maybe I have the time tomorrow for testing out. cheers bzrudi I think the problem can be the auto-completer for tag names in search tree-view. This one is populated with database contents and is updated at each changes. This will explain why with a fresh DB, the problem remain... Gilles digiKam 5.6.0 is now released and available as bundle for Linux, MacOS and Windows. https://www.digikam.org/news/2017-06-21-5.6.0-release-announcement/ Can you check if problem still exists with this version ? Thanks in advance Gilles Caulier New digiKam 5.7.0 are built with current implementation as pre-release bundles: https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM Problem still reproducible ? Hey Gilles, sadly problem still exists in latest 5.7.0 beta. Can you reproduce the dysfunction using digiKam 6.0.0 pre-release bundle available here : https://files.kde.org/digikam/ Can you reproduce the dysfunction using the last digiKam 6.0.0-beta3 just released ? https://www.digikam.org/news/2018-12-30-6.0.0-beta3_release_announcement/ Good news, After 2 weeks of works, the pre-release 6.1.0 bundles are now reconstructed from scratch with: - All OpenCV options for CUDA, OPenMP, and OPenCL disabled to prevent crashes in face management. - A large upgrade of Qt5 from 5.9.7 to 5.11.3. - An upgrade to KF5 5.55. - An upgrade to Ffmpeg 3.3.9 - The fontconfig/freetype integration in the bundle to reduce system dependencies Files can be downloaded here : https://files.kde.org/digikam/ Please test and report. Gilles Caulier Hi, Can you check if this problem still exist with last weekly bundle build of digiKam 7.0.0 available here: https://files.kde.org/digikam/ Thanks in advance Gilles Caulier digiKam 7.0.0 stable release is now published: https://www.digikam.org/news/2020-07-19-7.0.0_release_announcement/ We need a fresh feedback on this file using this version. Thanks in advance Gilles Caulier The main problem was sorting the tags. We had created a QCollator object for every sorting process of a tag name. Since the QCollator loads a database file every time, this is very slow. We now have a static QCollator. A test with 30,000 tags shows a good speed here. I'm closing the bug now. If there are still problems, please open again. Maik |