Bug 354819

Created attachment 95303 [details]
Sample picture which does not work

One of the files in question. Permission is granted by the photographer to use this file for bug testing. Please do not redistribute.

Turn on debug trace and run digiKam in a console as explained here :

https://www.digikam.org/contrib

Try to reproduce the problem and report all messages printed on the console.

This file still valid using last digiKam 5.0.0 ?

Gilles Caulier

What's about this file using digiKam AppImage bundle 5.4.0 pre release given at
this url :

https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM

Gilles Caulier

I have a similar issue since several digikam releases including 5.7. I'm running it on OpenSUSE 42.3. 

It seems only pictures taken with my smartphone SM-G900FD are affected.

Sometimes, it helps to rename the problematic photos, but with the image attached here as "testcase.jpg", the problem persists, independent of filename. 

When I copy it in a new directory, this just looks like an empty album in Digikam. When I copy it in an existing album, digikam will show all other photos, but not "testcase.jpg".

Created attachment 109454 [details]
testcase for picture missing from preview list

Test picture, can be displayed with gthumb, dolphin, GIMP, etc. without problems, but digikam 5.7 on OpenSUSE 42.3 refuses to show it in the preview list.

Ok, after checking the digikam debug output on stdout, I found that digikam says this when the problematic image is added: 

----
digikam.database: Recognized "/home/gernot/sync/digibilder/to_sort/aa/testcase.jpg" as identical to item 94617. 
----

So, I'm likely hit by #375483. My smartphone producing those images is a Galaxy S5 and all images have the same ImageUniqueID.

As the initial sample picture from Andreas in this report has no ImageUniqueID, my problem is likely distince and I probably shouldn't have added my issue here, sorry.

I recommend to test with 5.8.0 pre-release Linux AppImage bundle where some code have been fixed around the non single image ID provided by Samsung camera.

File is here :

https://files.kde.org/digikam/

Gilles Caulier

Unfortunately, 5.8.0 pre-release as of today does NOT fix the issue for me. I still have photos which are not shown by digikam. Also, I don't think this is directly connected to bug 375483 any more. Even removing the uuid (exiftool -ImageUniqId= <file>) doesn't change anything.

So far I found no rule when those files are visible in digikam and when not. With 5.7, I even had the same file (identical md5sum) twice under two different names in test directory "aa", one was visible, the other not. When moving them to another folder they both disappeared.

So far, it seems this does only affect photos taken by my Samsung smartphone. So there's probably another weirdness in the way Samsung creates JPGs.

When touching a visible and an invisible photo while digikam is running, stdout with 5.8 looks nearly identical:

invisible file:
digikam.general: Detected change, triggering rescan of "/home/gernot/sync/digibilder/to_sort/2017-08-xx Familie Landshut/"
digikam.database: Starting scan!
digikam.dimg: "/home/gernot/sync/digibilder/to_sort/2017-08-xx Familie Landshut/20170824_094516.jpg"  : JPEG file identified
digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal =>  QDateTime(2017-08-24 09:45:16.000 CEST Qt::TimeSpec(LocalTime))
digikam.metaengine: DateTime (Exif digitalized):  Do. Aug. 24 09:45:16 2017
digikam.metaengine: Orientation => Exif.Image.Orientation =>  6
digikam.metaengine: "/home/gernot/sync/digibilder/to_sort/2017-08-xx Familie Landshut/20170824_094516.jpg"  ==> Read Iptc Keywords:  ()
digikam.metaengine: Loading image history  ""
digikam.database: Scanning took 3 ms
digikam.database: Finishing took 36 ms
digikam.dimg: "/home/gernot/sync/digibilder/to_sort/2017-08-xx Familie Landshut/20170824_094516.jpg"  : JPEG file identified
digikam.metaengine: Orientation => Exif.Image.Orientation =>  6

visible file:
digikam.general: Detected change, triggering rescan of "/home/gernot/sync/digibilder/to_sort/2017-08-xx Familie Landshut/"
digikam.database: Starting scan!
digikam.dimg: "/home/gernot/sync/digibilder/to_sort/2017-08-xx Familie Landshut/20170824_091250.JPG"  : JPEG file identified
digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal =>  QDateTime(2017-08-24 09:12:50.000 CEST Qt::TimeSpec(LocalTime))
digikam.metaengine: DateTime (Exif digitalized):  Do. Aug. 24 09:12:50 2017
digikam.metaengine: Orientation => Exif.Image.Orientation =>  1
digikam.metaengine: "/home/gernot/sync/digibilder/to_sort/2017-08-xx Familie Landshut/20170824_091250.JPG"  ==> Read Iptc Keywords:  ()
digikam.metaengine: Loading image history  ""
digikam.database: Scanning took 6 ms
digikam.database: Finishing took 38 ms
digikam.dimg: "/home/gernot/sync/digibilder/to_sort/2017-08-xx Familie Landshut/20170824_091250.JPG"  : JPEG file identified
digikam.dimg: "/home/gernot/sync/digibilder/to_sort/2017-08-xx Familie Landshut/20170824_091250.JPG"  : JPEG file identified
digikam.metaengine: Orientation => Exif.Image.Orientation =>  1
digikam.metaengine: Orientation => Exif.Image.Orientation =>  1

The only difference is that "JEPG file identified" and the "Orientation" are printed *three* times for a working file while they are only printed *two* times for a file which is not shown in the UI. Does that somehow ring a bell for you?

Any further idea how I could narrow down this issue?

Just to make this clear: touching a problematic file while digikam is running leads to a rescan and the file is found according to debug output (see my last comment), but it still does NOT appear in the UI.

I enabled "Rescan file when files are modified" in the metadata settings as suggested in bug 375483.

Can you provide a test image from this smartphone?

Maik

Git commit 9a59b9d72b4d7606cf2bdb52be12a91fd22dbf21 by Maik Qualmann.
Committed on 21/12/2017 at 07:19.
Pushed by mqualmann into branch 'master'.

fix search or the Uuid if no XMP available

M  +29   -27   libs/dmetadata/dmetadata.cpp

https://commits.kde.org/digikam/9a59b9d72b4d7606cf2bdb52be12a91fd22dbf21

(In reply to Maik Qualmann from comment #11)
> Can you provide a test image from this smartphone?
> 
> Maik

I already attached it, see attachment 109454 [details]. Do you need further images? Unfortunately, this Heisen-bug isn't 100% reproducible, but if it helps I can add more examples to this bug or put them on my website.

Thanks for CC'ing https://commits.kde.org/digikam/9a59b9d72b4d7606cf2bdb52be12a91fd22dbf21, but you don't expect this to fix my bug, do you? I always add XMP:Creator to my pictures, so this commit shouldn't change anything for me, if I'm not mistaken.

Ok, I'm one large step ahead: all problematic images are internally tagged as "Original Version" despite I surely never edited those files:

sqlite> select * from ImageTags where imageid in (93042);
93042|112
sqlite> select * from Tags where id=112;
112|110|Original Version|0|

I can either delete the wrong internal tag:
delete from ImageTags where imageid in (select id from Images where name="20170907_102007.jpg");
or enable the "always show original version" setting in digikam.

My theory: During initial import from the Samsung smartphone, the files had the non-unique ImageUniqueID in EXIF data which Samsung deliberately uses to confuse people. Due to importing several files with the same unique ID, digikam somehow internally thought they are related and tagged some of them internally as "original version". 

Now, we could do what we want: manually change/remove EXIF ImageUniqueID field, internally create own uniqueIDs for it, moving the file away and back to digikam folders - this would never remove the internal originalVersion tag again.

Is it possible that those pictures received this flag because they

Right, the first import was the problem. We read though no XMP data exists an XMP key. Exiv2 returns the Exif key for this, but we will not check it there. My last commit fixes this issue. We always create a new Uuid now when the metadata is completely read. The problem should not occur now. I myself have a Samsung smartphone with this problem.

Maik

We should probably continue discussion in bug 375483, so that others can find this information, too, right?

The cause of this bug is now known and the bug with the Samsung UniqueID is fixed (Bug 375483). I close this bug now.

Maik