Every now and then, regardless of the version, digikam seems to be stuck on finding new items and the process takes long time. Is there general fix to this issue to bypass it? It usually takes to "fix" it to wait that abnormally long process to eventually finish, and then next time it works normally again. Occasionally searching thumbs slows down which can be cancelled but that's different process. I have "fast scan" setting enabled.
It appears that during the process files thumbnails-digikam.db thumbnails-digikam.db-journal digikam4.db digikam4.db-journal are getting updated so also the thumbnail process seems to be going on after all even though it's not visible in the digikam process display.
Related processes in task manager seem to be /usr/bin/perl -w /usr/bin/exiftool -stay_open true -@ - -common_args -charset filename=UTF8 -charset iptc=UTF8 digikam digikam seems to be occupying cpu constantly, exiftool hitting faster/shorter.
The problem is not known, the search for new items is always the same here. When we had bug reports, it was usually a hard disk error. So check your hard disk for errors. Otherwise, create a log with the Qt debug variable active in the terminal, as described here (https://www.digikam.org/contribute/) and post the process of a slow scan. Maik
I think I've had this occasionally with previous hardware too without disk failures. I thought checking "fast scan" earlier totally fixed this. Process just finished. If the problem persists I will try it again with debugging.
Where is your collection located (local drive, external USB drive or network)? ow many albums and pictures does the collection contain? How long does the scan take? Maik
I have both local drive and external USB. This time it actually started when I had USB connected which I don't always have. I'm unsure if the original slow search process started after connecting the drive (and not when I actually imported new pictures), but it continued slow after disconnecting USB. Next set I just imported was normal again as expected. I suspect I'm not experiencing this soon again, but next time it happens, I will start the process in debug mode. I could also try experimenting doing the import while external drive is connected but it's unlikely the reason as I think I'm (less often) doing import that way too.
I had it again. I'm running it now in debug mode but there's so much file paths and such I possibly don't want to share and cleaning them would need work. Anyway, what I did: i) Imported 100+ images from SD disk and renamed them. ii) I moved ~15 Gb data from internal HD to USB drive within digikam and afterwards deleted the folder at filemanager (Thunar). ~iii) At some point I had opened another instance of digikam, possibly at very start. All processes were finished, then I possibly closed first the first digikam instance where I did the import and file transfer (this instance had been open from previous session wich I had ended suspending the workstation), and then closed the second instance I had opened possibly around time I started the import. Then I reopened digikam and slow "Find new items" process started. I closed the program and restarted in debug mode. What I see this problem could possibly arise from keeping two digikam instances open while doing some file related processes. I have to check the drives from errors but there has been no other issues and I suspect that's not the cause.
Created attachment 175683 [details] debug output all directory paths and such replaced from output to /*DIR*/ etc.
Debug output is just a sample from beginning and interrupted the process. It seems that digikam is really scanning only this USB drive content slowly. When I closed the program, then disconnected the drive, it didn't start the process. Closing again, connecting drive, slow process started again. Drive is rather new and I haven't experienced any other problems.
Via Disks, SMART data is not available, but by checking filesystem is intact.
I understand correctly that you added 15GB of media? These need to be scanned until their metadata is in the database, that will take a long time... There is nothing unusual in the log, a normal file scan. Maik
No, 15 gb media was on local hdd which I moved to usb hdd within digikam. Such time used is not usual.
And those files in log were not transferred. It appears digikam started to scan all files.
I was about to do similar plain transfer test, but it seems that digikam is again stuck in scanning files at 19% while I have USB connected. There's nothing new I have made between last connection.
As such I have to keep the USB drive disconnected to get digikam properly scan new files at local drive only. I haven't found reason why digikam does that scan. All files at USB seem to be normally accessible.
Was now at 25% which I interrupted. So currently I have to keep USB disconnected when importing new files to make them appear. Process insists on scanning all files when USB is connected and I don't know why.
Still hanging after upgrading to 8.5.0 appimage. So what is digikam trying to do? It is constantly starting thorough scanning of files that has long been where they are (at USB drive). Scanning them isn't directly related to prementioned file transfer. I regularly do such file transfers from local drive to usb drive, and they're all working perfectly fine: files are accessible at new location without any separate scan. File transfer shouldn't cause such scan if it even is the cause.
Could this possibly be related to comparing data at sidecars and sql-database? Is there any information what is exactly happening?
Could it simply be that your drive has not been fully scanned yet because you keep interrupting it? A full scan of a USB drive with many GB of data can take hours. If you move or copy data, no scan is performed because digiKam copies it internally in the DB. However, if you have activated monitoring of albums for external changes, a scan will still be performed. Therefore, we recommend not activating monitoring of albums for external changes unless you absolutely need the update while digiKam is running. Maik
I have let it run completely through earlier, but now I started interrupting because it appeared so soon again. It went through last night though, taking several hours. I'll check the settings, but I've made all transfers within digikam and otherwise files have been intact for longer so I don't what's the reason. Waiting if it reoccurs again. The another suspection was if accidentally keeping two digikam instances open in certain situations causes this.
So, what this section from debug tells? Digikam::DMetadata::load: Loading metadata with "Exiv2" backend from "/FILEPATHNAMEREADSHERE" Digikam::DImg::load: "/FILEPATHNAMEREADSHERE" : "RAW" file identified Digikam::MetaEngine::getDigitizationDateTime: DateTime (Exif digitalized): QDateTime(DATETIMEHERE Qt::UTC) Digikam::MetaEngine::getDigitizationDateTime: DateTime (digitization date): QDateTime(DATETIMEHERE Qt::UTC) Digikam::MetaEngine::getXmpTagStringSeq: XMP String Seq ( Xmp.digiKam.TagsList ): QList("TAGS","LISTED","HERE") Digikam::ItemScanner::commit: Scanning took 44 ms Digikam::ItemScanner::~ItemScanner: Finishing took 6 ms ...repeating for all files. It scans files, but what it does to data it reads? And what actually starts this operation as just connecting USB drive seems to trigger it, and it doesn't get cancelled closing instances?
There was actually a bug in connection with digiKam < 8.5.0 that only occurred with the MySQL database, where images were scanned again. With digiKam-8.5.0 there will therefore only be one larger scan, after which there should no longer be a scan when the drive is connected. Maik
In my case this is happening after renaming an album (from within DK). The subsequent launches take forever to "find new items". It turns out that the paths in thumbnails-digikam.db still use the old album path and this is causing problems. Manually fixing the db solves the issue.