| Summary: | Work-around for maximum number of images in an album | ||
|---|---|---|---|
| Product: | [Applications] digikam | Reporter: | Pete Hall <pete> |
| Component: | Albums-TableView | Assignee: | Digikam Developers <digikam-bugs-null> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | caulier.gilles, jay_marm, metzpinguin, pete |
| Priority: | NOR | ||
| Version: | 6.0.0 | ||
| Target Milestone: | --- | ||
| Platform: | Appimage | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed In: | 7.5.0 | |
| Sentry Crash Report: | |||
|
Description
Pete Hall
2018-08-26 20:49:35 UTC
I do not really understand their report, it may also be because of my English, but think to know what it is about. I think it's not the right way, the tree view and the icon view should handle such data sets. We have also made some progress in the last few days. The start of digiKam could be shortened from 1:30 minutes to 20 seconds with a test collection of 11000 albums. Maik I agree with Maik, this report is not enough clear. Pete, please can you write more details about the problem and solution proposed. Gilles Caulier After storing a certain number (unknown but less then 32767, so not a data type issue) of images, digikam cannot display the number of images in a directory (it always shows zero) for most recently added directories (ONLY MOST RECENT DIRECTORIES). Images in directories that show zero items in digikam, do not appear as thumbnails, and cannot be retrieved for viewing or editing. This can be worked around by splitting up images into two groups so you never reach that magic number of images. Splitting up images means that you must store part of your total image catalog on a different file system, and must umount it. If both parts are mounted at the same time, you exceed that magic number of images across all file systems, and the problem re-appears. I can do this umounting and mounting externally to digikam, and this works, but is not very user friendly. Also cannot move images from the mounted group to the umounted group, which is even more user un-friendly. By the way, I have tried moving to mariadb. maybe that will solve the problem eventually, but cannot get mariadb to work on ubuntu 18.04 due to dependency version mismatches in the ubuntu repository. I do not want to violate this due to unknown side effects with other packages. The problem may not be the database anyway. Maybe it is caused by a maximum number of objects allowed in a java collection or something like that. This I cannot determine. Do you use a 32 bit operating system? Maik No. I run a 64 bit ubuntu virtualbox client on a 64 bit slackware host. Ubuntu is version 18.04 with no guest additions. Virtualbox is 5.2.18. with QT5 6.1. Slackware is 14.2. All are at current patch levels. Thanks Pete Git commit f27ab9c1051bd0a0bba6e79bc77899c74a7e6bf8 by Maik Qualmann. Committed on 07/10/2018 at 13:47. Pushed by mqualmann into branch 'master'. add a global cache for grouped images When we load the images into the Icon view, we ask each time, whether there are grouped images, with 30000 images in the view are that also 30000 SQL query. With this patch, the time to load a view with many images is faster with MySQL 3x and with SQLite 2x. Related: bug 391840, bug 398921, bug 396086 M +24 -0 core/libs/database/coredb/coredb.cpp M +5 -0 core/libs/database/coredb/coredb.h M +1 -10 core/libs/database/item/imageinfo.cpp M +19 -2 core/libs/database/item/imageinfocache.cpp M +7 -0 core/libs/database/item/imageinfocache.h M +0 -3 core/libs/database/item/imageinfodata.h https://commits.kde.org/digikam/f27ab9c1051bd0a0bba6e79bc77899c74a7e6bf8 After 3 weeks of work, i finally completed the compilation of AppImage using Qt 5.11.3 + QWebkit 5.212. New 6.1.0 pre-release AppImage bundle can be found here (64 bits only for the moment) : https://files.kde.org/digikam/ Please check if this bugzilla entry still valid. Thanks in advance Gilles Caulier Git commit 1f8edbda8bff2a3e201b8870a686f0066e20d590 by Maik Qualmann. Committed on 22/04/2019 at 18:35. Pushed by mqualmann into branch 'master'. use QReadWriteLock for the CollectionManager Loading a large item view is now about 4x faster here with MySQL Related: bug 398921, bug 391115, bug 391840 M +12 -13 core/libs/database/collection/collectionmanager.cpp M +1 -1 core/libs/database/collection/collectionmanager.h M +11 -9 core/libs/database/collection/collectionmanager_album.cpp M +118 -107 core/libs/database/collection/collectionmanager_location.cpp M +1 -1 core/libs/database/collection/collectionmanager_p.cpp M +2 -0 core/libs/database/collection/collectionmanager_p.h M +1 -1 core/libs/database/coredb/coredbaccess.cpp https://invent.kde.org/kde/digikam/commit/1f8edbda8bff2a3e201b8870a686f0066e20d590 Not sure if this is the same issue, but I just imported a directory (my main photo repository) that has several dozen subdirectories with nearly 25K files. The resulting album has only 15 subdirectories and a small fraction files. Pete, Your dysfunction is not reproducible here with plenty of albums including a lots of items inside (some ahve more than 1000 photos) Can you reproduce your problem using digiKAm Linux AppImage 7.0.0-beta3 ? https://files.kde.org/digikam/ Gilles Caulier digiKam 7.0.0 stable release is now published and now available as FlatPak: https://www.digikam.org/news/2020-07-19-7.0.0_release_announcement/ We need a fresh feedback on this file using this version. Thanks in advance Gilles Caulier I close this file now. 2 main commits have been applied to fix the latency problem with huge collection of files. Don't hesitate to reopen if necessary. |