Bug 489279

Summary: Albums Not Discovering all Pictures in a Folder
Product: [Applications] digikam Reporter: Gerry Helfrich <ghelfrich919>
Component: Albums-EngineAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: caulier.gilles, metzpinguin
Priority: NOR    
Version First Reported In: 8.3.0   
Target Milestone: ---   
Platform: Microsoft Windows   
OS: Microsoft Windows   
Latest Commit: Version Fixed/Implemented In: 8.7.0
Sentry Crash Report:
Attachments: Link to 2 pictures not discovered/found by digiKam
attachment-1534854-0.html

Description Gerry Helfrich 2024-06-27 04:51:29 UTC
Created attachment 171058 [details]
Link to 2 pictures not discovered/found by digiKam

SUMMARY
I have discovered 2 pictures in 2 separate folders within an album which was added as a collection on a network share that are not being discovered by digiKam.  Both of these folders have .jpg pictures in them and digiKam discovered all of the pictures in them except for one.  

I have thousands of pictures and hundreds of folders.  I only discovered these pictures were not displaying in digiKam when I was tagging pictures, and I couldn't find all the pictures in these 2 folders displaying in digiKam.  I've tried refreshing the folders in digiKam (RMC on folder and selecting 'refresh'), but these pictures are still not being found

There may be more pictures not displaying, but these are the only ones I've found so far.  I have no easy way of determining if there are others without a lot of manual work.  I've only started recently using digiKam as my primary photo management application, so I can't say this issue is new to this version of digiKam or not.  

STEPS TO REPRODUCE
I am not able to reproduce this issue for other pictures.  I can say these 2 pictures are not being discovered/found by digiKam either on a network share or a local drive.

OBSERVED RESULT
These 2 pictures are not discovered/found by digiKam.

EXPECTED RESULT
These 2 pictures are discovered/found by digiKam.

SOFTWARE/OS VERSIONS
Windows: MS Windows 10

ADDITIONAL INFORMATION
I have attached the 2 pictures that are not being discovered by digiKam.
Comment 1 Gerry Helfrich 2024-06-27 04:56:20 UTC
*** Bug 489278 has been marked as a duplicate of this bug. ***
Comment 2 Maik Qualmann 2024-06-27 05:43:43 UTC
Your two sample images are displayed here without any problems.
Did you read my comment?

https://bugs.kde.org/show_bug.cgi?id=489278#c1

Maik
Comment 3 caulier.gilles 2024-06-27 05:58:42 UTC
No problem too with these files while importing in my collection...
Comment 4 Gerry Helfrich 2024-06-27 19:19:59 UTC
Thanks for the info and suggestions.  

First, I navigated to "settings->configure digiKam->versioning->In Main view" and made sure the options "Always Show Original" and "Always show intermediate snapshots" were selected.

Next, in the album context menu, I selected "Repair hidden items" for the appropriate folders as well as the album for the copied local pictures.

Unfortunately, this didn't solve my issue.

I was afraid you'd come back and say it's working for you.  But obviously something is amiss for me.  I could see one could think there is something amiss with the network share.  But when I copy these same 2 files to a local SSD drive, they still aren't discovered.  If I copy other files from the same folder on the network share to the local SSD drive, digiKam discovers them.  

Also, there isn't any problem opening these pictures using a photoviewer so the files aren't corrupted.   A common denominator between these 2 scenarios seems to me to be digiKam/configuration.

Is there something else I can look at or try?   My concern is I found these 2 missing pictures after looking closely at only the first 5% of my pictures and there is likely to be more.  

Any help is appreciated.
Comment 5 Maik Qualmann 2024-06-27 20:09:49 UTC
Hmm, OK, the images aren't visible on a local SSD either, that's weird. I can only think of one reason at the moment, and that is that there's something wrong with the extension. Could it be that there's a space appended to the .jpg, for example, or a similar invisible character? Try renaming the file in Explorer.

But note that digiKam only looks for changed files when you restart (if the option to search for new files is activated - this is standard) or you run "Search for new entries" under Tools, or with F5, Ctrl+F5 for the current album.

Maik
Comment 6 Gerry Helfrich 2024-06-28 15:46:30 UTC
Let me give you additional background on my pictures, and what I have been able to uncover since my last note.  

I have been using Picasa for many years for my photo management and face tagging.  But Picasa's ability to write face tag info in the picture metadata region never advanced beyond the experimental stage.  So, I have not used it to write this data to the picture metadata.  Thus, my face tag info has been contained in the proprietary structure of Picasa. 

So recently I decided to migrate over to digiKam for my photo management and face tagging.  To not lose or have to recreate my face tags in digiKam, I decided to use the exiftool along with the "picasa_faces.config" config_file module available for the exiftool to wrote Picasa proprietary face tag into both the MWGRegion and MPRegion of my pictures.  

Over the last year I have slowly been using the exiftool/picasa_faces.config tool to update this face tag info into these metadata regions for my 10k+ pictures located in 400 folders+.  And now I've been setting up collections for my pictures in digiKam which is when I saw these 2 pictures were missing from digiKam.

That gives you some info on my pictures.  Now let me give you some info on what I was able determine through my additional testing last evening.  Now that you know about my updating face tag regions of my pictures, I started to wonder if the metadata face tag updates weren't somehow related to the issue I was seeing.  

So, I decided to use the exiftool tool to delete the face tag regions for these 2 pictures to see if digiKam would now discover them.  I deleted the face tag region for both these regions and digiKam automatically discovered them within a few seconds.  

I then used the exiftool/picasa_faces.config tool to write the face tag info back into the metadata regions to see what digiKam would do.  I expected digiKam to not discover these 2 pictures again when I did so.  To my surprise, digiKam did actually discover these 2 pictures again.  So, I was really perplexed because I was seeing different results than what I was expecting.

I'm not sure why I did this, but I decided to rename one of these pictures.  And as soon as I did, digiKam did not rediscover the picture.  I repeated the same exercise for the 2nd picture, except this time I decided to move the picture to a different folder rather than rename it.  Once again, after moving the picture, digiKam didn't rediscover the picture.  And to be specific, I was using exiftool v12.5 and the latest (2016) version of the picasa_faces.config add-in tool.

So, here is the bottom-line of what I saw.  Certain pictures, when their face tag info has been updated with the exiftool v12.5 and the (2016) version of the picasa_faces.config add-in tool led to digiKam not discovering the picture.  It appears these pictures are actually initially discovered by digiKam, but then not rediscovered by digiKam when something about the picture has changes (ie. rename/move)

Then I thought I should retry writing the Picasa face tag info using the latest exiftool v12.8.7.  So, I repeated the same steps mentioned above using this latest version.  Jumping to the conclusion, when I rewrote the Picasa face tag info with this version of the exiftool, digiKam quickly rediscovered these pictures.  And when I moved/renamed these pictures, digiKam rediscovered them as well.  

So, it appears something about how/what the 12.5 version did to write the Picasa face tag info to the picture metadata face tag regions interferes with certain pictures not being routinely discovered by digiKam.  But whatever the exiftool was doing in v12.5 was changed in a later version and no longer interferes with them being routinely discovered by digiKam.

I now understand how I can recover when I discover any further pictures digiKam is not rediscovering.  What greatly concerns me is I am likely to see additional pictures initially discovered not be rediscovered by digiKam when they are renamed, or more likely moved sometime in the future, probably years down the road.  I used the exiftool v12.5 tool to add the Picasa tag into the face tag regions for all 10+ of my pictures.  I don't plan to redo writing the Picasa face tag info again using exiftool v12.8.7.

Bottom line is this.  There is still something in how/what the exiftool v12.5 wrote to the 2 picture face tag metadata regions (and likely more) that trips up the digiKam picture discovery engine.  

Please let me know how I can help figure out what is going on here.  :)
Comment 7 Maik Qualmann 2024-06-28 19:15:26 UTC
I'm sure it has nothing to do with the face metadata if digiKam doesn't show the image. It can only have something to do with the extension if digiKam ignores the image because it doesn't recognize it as the desired image type.
Please add an image that digiKam doesn't recognize to a ZIP file and send it by email or upload service.

Maik
Comment 8 caulier.gilles 2025-04-11 18:13:52 UTC
Hi,

The 8.7.0 pre-release Windows installer from today have been rebuilt from
scratch with Qt 6.8.3, KDE 6.12, OpenCV 4.11 + CUDA support, Exiv2 0.28.5, ExifTool 13.27, ffmpeg 7, all image codecs updated to last version (jxl, avif, heif, aom, etc.).

Please try with this version to see if your problem still reproducible...

https://files.kde.org/digikam/

Thanks in advance
Best regards

Gilles Caulier
Comment 9 Gerry Helfrich 2025-04-15 05:22:33 UTC
Created attachment 180276 [details]
attachment-1534854-0.html

 Gilles, 
I just downloaded and installed the 8.7.0 pre-release windows version as you suggested.  I opened digiKam to view the 2 albums with the 2 pictures on a network drive not previously discovered by digiKam.  
From what I see, both pictures have been discovered and are showing in digiKam.  
Thanks for the follow-up. 
Gerry
    On Friday, April 11, 2025 at 03:08:47 PM EDT, bugzilla_noreply@kde.org <bugzilla_noreply@kde.org> wrote:  
 
 https://bugs.kde.org/show_bug.cgi?id=489279

--- Comment #8 from caulier.gilles@gmail.com ---
Hi,

The 8.7.0 pre-release Windows installer from today have been rebuilt from
scratch with Qt 6.8.3, KDE 6.12, OpenCV 4.11 + CUDA support, Exiv2 0.28.5,
ExifTool 13.27, ffmpeg 7, all image codecs updated to last version (jxl, avif,
heif, aom, etc.).

Please try with this version to see if your problem still reproducible...

https://files.kde.org/digikam/

Thanks in advance
Best regards

Gilles Caulier
Comment 10 caulier.gilles 2025-04-15 08:05:15 UTC
Great, thanks for the feedback