Bug 514767 - [Performance] Ignored directories should not be scanned at all during "Find new items"
Summary: [Performance] Ignored directories should not be scanned at all during "Find n...
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Maintenance-Database (other bugs)
Version First Reported In: 8.8.0
Platform: Microsoft Windows Microsoft Windows
: NOR minor
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2026-01-17 21:21 UTC by kde
Modified: 2026-01-18 17:11 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed/Implemented In: 9.0.0
Sentry Crash Report:


Attachments
Sample of the behaviour (535.06 KB, image/png)
2026-01-17 21:32 UTC, kde
Details

Note You need to log in before you can comment on or make changes to this bug.
Description kde 2026-01-17 21:21:55 UTC
SUMMARY
When DigiKam crawls for new items, even though directorieds are listed as ignored, they are still scanned (as can be monitored using ProcMon). Though in the end this is probably not a big problem since I suppose they are filtered later, not scanning them at all could improve greatly the scan process in some situation (for instance, my #snapshot folder is actually visible and there are like 50x my whole library in there).

STEPS TO REPRODUCE
1. In configuration, enter some ingored directories (in my case, #snapshot;#recycle)
2. Start the scan
3. Monitor with ProcMon (or any other mean, I read about a DigiKam console but I don'T know where to find it)

OBSERVED RESULT
The ignored folders and their content are scanned

EXPECTED RESULT
The ignored folders and their content should be ignored

SOFTWARE/OS VERSIONS
digiKam: 8.8.0
KDE Frameworks: 6.17.0
Qt: Using 6.9.1 and built against 6.9.1
Windows 11 Version 25H2
Build ABI: x86_64-little_endian-llp64
Kernel: winnt 10.0.26200
Comment 1 kde 2026-01-17 21:32:50 UTC
Created attachment 188627 [details]
Sample of the behaviour
Comment 2 Maik Qualmann 2026-01-18 06:49:11 UTC
We have to live with this; we use a QDirIterator, or in Qt6, the new QDirListing function. We can't simply "skip" a directory. In digiKam-9.0.0, there are some changes; we now use a directory cache, so a second scan of the collection isn't necessary to detect deleted albums.

Maik
Comment 3 Maik Qualmann 2026-01-18 10:41:57 UTC
Git commit 1514f26e4df3c2b915e53d881591c56d3326af8b by Maik Qualmann.
Committed on 18/01/2026 at 10:40.
Pushed by mqualmann into branch 'master'.

apply ignored directories when creating the cache
FIXED-IN: 9.0.0

M  +1    -0    core/libs/database/collection/collectionscanner.h
M  +2    -3    core/libs/database/collection/collectionscanner_scan.cpp
M  +29   -41   core/libs/database/collection/collectionscanner_utils.cpp

https://invent.kde.org/graphics/digikam/-/commit/1514f26e4df3c2b915e53d881591c56d3326af8b
Comment 4 kde 2026-01-18 16:48:48 UTC
Though I understand the simplicity (and possibly the safety) of using an OOB enumerator function like QDirListing, the drawbacks of it can be quite important for large (networked) libraries or large ignored folders.

Maybe I could open a new one as Wishlist to keep in mind that implementing a custom iterator that would iterate with some parallelism (maybe a pool of workers with a synchronized directories queue?) and be able to prune ignored folders could be a useful addition?
Comment 5 Maik Qualmann 2026-01-18 17:11:42 UTC
Even multiple workers can't easily solve this problem. I've conducted several tests. The following test setup: external USB hard drive, FAT32 file system; the scan takes minutes. Everything is the same now with NTFS, and the scan now takes only a few seconds. We need the modification date of the directory/file. With FAT32, this is stored in the file entry; with NTFS, it's in the directory – immediately available.

You're using a UNC path; have you tried using a mount point?

Maik