Bug 505713

Summary: When adding a local collection, some subfolders are missing in the tree view
Product: [Applications] digikam Reporter: tb.digikam
Component: Database-ScanAssignee: Digikam Developers <digikam-bugs-null>
Status: REPORTED ---    
Severity: normal CC: caulier.gilles, metzpinguin
Priority: NOR    
Version First Reported In: 8.6.0   
Target Milestone: ---   
Platform: Microsoft Windows   
OS: Microsoft Windows   
Latest Commit: Version Fixed/Implemented In:
Sentry Crash Report:
Attachments: Windows folder vs Digikam treeview
Log file once import seems finished
Logs after re-installationn
digikamrc with strange

Description tb.digikam 2025-06-17 21:18:31 UTC
Created attachment 182338 [details]
Windows folder vs Digikam treeview

SUMMARY
When I create a collection from a local directory containing subfolders on three levels and pictures inside the third level, only a part of the subfolders of the first level is imported.

Here is the context and the structure of my directory:
- I have a local brand new  NVMe disk of 4TB (in addition of another system disk C:)
- It is formated as NTFS and mounted on the letter D:
- It contains a folder named 'Photos_videos_Synchro' (level 1)
- Inside it, one subfolders for each year from 2000 to 2025  (+ 4 folders  for 1983, 1986, 1996, 1999) (Level 2)
- Inside these folders, pictures and videos are organised in a set of subfolders named as "aaaa-mm-dd - Description" (Level 3) with 'Description' about 35 characters
- Pictures formats are : jpg (99%), bmp and png (less than 1%)
- Video format are : mov, avi (up to 2004) and mov, avi, mpg, mpeg after 2004
- Digikam is set to 'French language'
- I run the test both with SQLite and with MariaDb (db digkam_core, digikam_faces, digikam_similarity) with the same results.

STEPS TO REPRODUCE
1. Open Digikam 8.6.0
2. Open the collection configuration menu (Menu Configuration/Configure Digikam/collection)
3. Add a local collection on D:\Photos_videos_Synchro

OBSERVED RESULT
Only folders 1983, 1986, 1996, 1999, 2000, 2001, 2002, 2003, 2004 are listed in the treeview representing a total about 3,03 GB of 3724 files in 179 subfolders (all other folders, one per year from 2004 to 2025, are missing - 795Go for 116 168 files in 2455 folders).

Moreover, In the folder named 'D:\Photos_videos_Synchro\2024', Only 25 subfolders over 34 are listed.

Note: The result is the same with or without checking the box 'Monitore albums to detect external changes'
Note : The result is the same after closing and relaunching Digikam
Note : The result is the same if I rename Photos_videos_Synchro by "Photos"
Note : The result is the same if I copy/past all subfolders from D:\Photos_videos_Synchro\... in a newly created folder named D:\TEST\....
Note : The result is the same either with SQLite and Maria DB
Note : I Let it run for 8 hours without seeing more subfolders inside the collection

EXPECTED RESULT
I was expecting to have a collection named "Photos_videos_Synchro", with all subfolders as they are on D:\Photos_videos_Synchro\...

or a message alerting me if I went over the limits in terms of numbers of subfolders, files, etc...

SOFTWARE/OS VERSIONS
Windows 11 Famille - 24H2

ADDITIONAL INFORMATION
RAM 32Go
I did not find logs :-/ sorry
Comment 1 caulier.gilles 2025-06-18 02:47:55 UTC
To see the log you need debugview tool from Microsoft. Look instruction here :

https://www.digikam.org/contribute/#windows-host
Comment 2 Maik Qualmann 2025-06-18 04:05:54 UTC
I think you're going too fast, and the collection isn't fully scanned yet. This could take hours or days. As long as there's a progress bar in the status bar, the collection is still being scanned. You should wait for the first scan and not interrupt it.

Maik
Comment 3 tb.digikam 2025-06-18 06:25:19 UTC
I don't think so because the progress bar is shown only for 10 seconds. Then, after a refresh of the collection, the progress bar indicates "Elément analysé à la recherche de vignettes : 0 / Le processus a terminé" => "Element analysed to find thumbnails : 0 / The processus has finished".

During the 8 hours I've wait, there was no more progress bar, It was indicating that the import was finish. Moreovers, in 8 hours, no more directory was added to the directory 2024.

I uninstalled Digikam and re-installed it and I have the same behaviour.

=> I'll install the component enabling logs tomorow and see.
Comment 4 tb.digikam 2025-06-18 06:52:45 UTC
Created attachment 182347 [details]
Log file once import seems finished

Here are the logs. 
- I removed the collection.
- I activated internal logs in digikam
- I close and restart digikam
- I chose D:\Photos_videos_synchro as my first collection once digikam asked me
- after few minutes, I had no more messages from process [32944] (digikam)
- folders are still not visible in th treeview
Comment 5 caulier.gilles 2025-06-18 06:59:48 UTC
Under Linux i use 4Tb NVME full of video + photo. I don't have any disfunction like you, as after a long scan (few hours), all is here and work properly.

I suspect something in your Windows which lock digiKam at run-time, eg. an anti-virus for ex...

Best regards

Gilles Caulier
Comment 6 Maik Qualmann 2025-06-19 05:48:58 UTC
The log is very strange. It starts with a database error dialog, but you don't seem to have made any changes. The items are being found again in NULL albums, presumably because you removed the collection (theoretically, this isn't necessary). All of your JPG or BMP images are an unknown image format for digiKam—presumably because no DImg plugins were loaded. Generally, your log doesn't show any plugin loading. Sorry, there's nothing we can do here for now. Reinstall your digiKam and database, and try using an SQLite database first.

Maik
Comment 7 tb.digikam 2025-06-19 06:19:05 UTC
Hello, 
Thanks guys for your time and these advises. I'll try it to re-install all of this, deactivate the antivirus and I'll keep you in touch.
Kind regards, Thierry.
Comment 8 tb.digikam 2025-06-21 07:38:23 UTC
Created attachment 182472 [details]
Logs after re-installationn

Logs after re-installation and re-import only the problematic folder.
Comment 9 tb.digikam 2025-06-21 09:27:39 UTC
So, I Uninstalled DigiKam following this procedure:
Removing :
C:\Users\<xxx>\AppData\Local\digiKam\
C:\Users\<xxx>\AppData\Local\digikamrc
C:\Users\<xxx>\AppData\Roaming\digiKam\
C:\Users\<xxx>\AppData\Roaming\digikamrc
C:\Users\<xxx>\AppData\Local\kde.org\
C:\Users\<xxx>\AppData\Local\kde*

removing : C:\digiKam\Thumbnails (I previously choosed to use this directory instead of C:\Users\<TonNom>\Pictures\digiKam\Thumbnails
)

Doping tables:
DROP DATABASE digikamcore;
DROP DATABASE digikam_faces;
DROP DATABASE digikam_similarity;

And then, I re-installed Digikam with a SQLite DB and... The problem was the  same, But I found from where it comes.

During the import, everything works fine until it get to this folder ; "2004-10-jj- Diplome GBM" At this point, the import process stops, the two videos in the folders are never imported in Digikam.

After this, I removed and re-install Digikam again REMOVING this problematic folder from my disk => All my other folders have been imported in less than 1H, everything is fine.

I then put back "D:\Photos_videos_Synchro\2004\2004-10-jj- Diplome GBM" in place an restarted Digikam.

At this point, we can read this into the logs:

[24216] digikam.metaengine: ExifTool "Load Chunks" "-TagsFromFile D:\\Photos_videos_Synchro\\2004\\2004-10-jj- Diplome GBM\\PA030028.MOV -all -icc_profile -api QuickTimeUTC -xmp-dc:Subject<Microsoft:Category -xmp-microsoft:RatingPercent<Microsoft:SharedUserRating -o -.exv"
[24216] digikam.metaengine: ExifTool process finished with code: 255 and status QProcess::NormalExit

I Gues that old .MOV files are not supported.

End of the story for me, I'm fine if I can't read them. nevermind.

Note: I also have below messages in logs, but I don't think that is it is related:
[24216] kf.config.core: KConfigIni: In file C:/Users/tbxbl/AppData/Roaming/digikam/digikamstaterc, line 5: Invalid escape sequence: «\.»
[24216] kf.config.core: KConfigIni: In file C:/Users/tbxbl/AppData/Roaming/digikam/digikamstaterc, line 5: Invalid escape sequence: «\I»
[24216] kf.config.core: KConfigIni: In file C:/Users/tbxbl/AppData/Roaming/digikam/digikamstaterc, line 12: Invalid escape sequence: «\A»
[24216] kf.config.core: KConfigIni: In file C:/Users/tbxbl/AppData/Roaming/digikam/digikamstaterc, line 13: Invalid escape sequence: «\L»
[24216] kf.config.core: KConfigIni: In file C:/Users/tbxbl/AppData/Roaming/digikam/digikamstaterc, line 14: Invalid escape sequence: «\P»
[24216] kf.config.core: KConfigIni: In file C:/Users/tbxbl/AppData/Roaming/digikam/digikamstaterc, line 15: Invalid escape sequence: «\S»
[24216] kf.config.core: KConfigIni: In file C:/Users/tbxbl/AppData/Roaming/digikam/digikamstaterc, line 25: Invalid escape sequence: «\.»
[24216] kf.config.core: KConfigIni: In file C:/Users/tbxbl/AppData/Roaming/digikam/digikamstaterc, line 25: Invalid escape sequence: «\I»
[24216] kf.config.core: KConfigIni: In file C:/Users/tbxbl/AppData/Roaming/digikam/digikamstaterc, line 31: Invalid escape sequence: «\.»
[24216] kf.config.core: KConfigIni: In file C:/Users/tbxbl/AppData/Roaming/digikam/digikamstaterc, line 31: Invalid escape sequence: «\I»

Kind regarde, Thierry.
Comment 10 caulier.gilles 2025-06-23 14:44:17 UTC
The C:/Users/tbxbl/AppData/Roaming/digikam/digikamstaterc error are strange as it's about the encoding of characters inside.

Can you share this file for investigation ?
Comment 11 Maik Qualmann 2025-06-23 14:49:59 UTC
I'm aware of this issue, but we can ignore it for now. It's caused by KF6. The State ByteArray values ​​for the KXML window are probably not stored in Base64 encoding. We should report it as a bug to the KF6 team for Windows.

Maik
Comment 12 tb.digikam 2025-06-23 20:54:00 UTC
Created attachment 182577 [details]
digikamrc with strange

(In reply to caulier.gilles from comment #10)
> The C:/Users/tbxbl/AppData/Roaming/digikam/digikamstaterc error are strange
> as it's about the encoding of characters inside.
> Can you share this file for investigation ?

Sure, here it is.
Comment 13 Maik Qualmann 2025-06-24 07:53:11 UTC
Here it is the monitor display name that contains the invalid escape sequences.

Maik
Comment 14 Maik Qualmann 2025-06-24 12:25:58 UTC
Let's get back to the actual problem. It looks like ExifTool crashes when scanning the MOV video file. That would be bad, and we'd like to reproduce it, perhaps even report it as a bug to ExifTool. Is it possible to make the file available?

Maik
Comment 15 tb.digikam 2025-06-24 21:40:20 UTC
Hello Maik,
As there is people on the videos, I have to ask them prior to share the files. I'll keep you in touch. 
Kind regards, Thierry.
Comment 16 tb.digikam 2025-06-28 08:19:42 UTC
Hi Maik,
People on these 2 videos do not want me to share them. If you guide me, maybe I can use some tools to explore the format of these videos and obtain a report about their structure.
Kind regardes, Thierry.
Comment 17 caulier.gilles 2025-12-09 15:47:00 UTC
to tb.digikam@domewe.com

You can install ExifTool GUI for Windows. Read the instruction here :
https://exiftool.org/gui/

...and load a video file in this application to see the parse works as expected..

Alternative : https://www.xnview.com/en/

This one use ExifTool in the background. I'm not sure if it support all video formats.