Bug 416755

Summary: Tag with "." not processed correctly, likely to result in data loss
Product: [Applications] digikam Reporter: harald.aust
Component: Tags-KeywordsAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: metzpinguin
Priority: NOR    
Version: 6.4.0   
Target Milestone: ---   
Platform: Microsoft Windows   
OS: Microsoft Windows   
Latest Commit: Version Fixed In: 7.0.0
Sentry Crash Report:
Attachments: Structered tag as well as correct tag
Sample picture with "." tag that does not show up in digiKam
tags.png
After 'Add Collection'
Metadata after 'Add Collection'
After 'Reread Metadata From File'

Description harald.aust 2020-01-25 16:24:46 UTC
SUMMARY
A tag assigned outside of digiKam that contains a "." (just the dot, without the quotation marks) is not processed correctly. Instead of the tag, an hierarchy of tags is created. If subsequently the picture is updated from digiKam, the original tag is lost, wherefore I consider this bug critical (data loss).


STEPS TO REPRODUCE
1. Pick any <picture> that you have in digiKam.
2. Run   exiftool -overwrite_original -m -IPTC:keywords+=a.b.c -XMP:subject+=a.b.c <picture>
3. Re-read metadata in digiKam, if not done automatically.
4. Look at the tags of <picture> in digiKam.

OBSERVED RESULT
Instead of a single tag "a.b.c", you will see a tag hierarchy "a / b / c".
Interestingly, when you look into "Edit Metadata", the tag is correct there.


SOFTWARE/OS VERSIONS
Win 10 1809 64bit
digiKam 6.4

ADDITIONAL INFORMATION
Result is the same whether tag is created with exiftool, Lightroom, or Picasa. Also, it doesn't seem to matter whether the tag is added after the picture was imported into digiKam, or whether the picture is imported into digiKam with the tag already there.
Comment 1 Maik Qualmann 2020-01-25 16:53:10 UTC
It works fine here. The tag is recognized as "a.b.c". IPTC: keywords uses the "." as a separator. My guess, you pushed IPTC: keywords in the advanced metadata settings before XMP, which means that IPTC: keywords is processed and found first and then creates the tag tree.

Maik
Comment 2 harald.aust 2020-01-29 12:24:49 UTC
Maik,

So it’s a feature, not a bug. But this is unique to digiKam, isn’t it? I have not observed this behavior with any of the other software I use (e.g. Lightroom, Picasa, IrfanView, exiftool).

Your guess is right. I had changed the order for tags, putting iptc.application2.keywords first. Reason for that was that many of my photos contain *only* iptc:keywords, and I wanted to make sure that those are used, and not some potentially arbitrary contents of a different field that some software might have written without me intending so.

Now, I did „reset to default“; read in a photo with „a.b.c“ tag both in iptc:keywords and xmp:subject, and it worked :-)

Are there any other pitfalls to be avoided? 

Best regards,
Harald
Comment 3 Maik Qualmann 2020-01-29 14:52:47 UTC
Ok, I just took another look at the IPTC Guide, which speaks of commas or other separators. Not of points. Then it would be a mistake in digiKam and easy to change.

Maik
Comment 4 Maik Qualmann 2020-01-29 22:03:00 UTC
Git commit 56082288b29c19eaacfeb9dab6fb16e6d1b8bff3 by Maik Qualmann.
Committed on 29/01/2020 at 22:01.
Pushed by mqualmann into branch 'master'.

change separator for Iptc.Application2.Keywords to comma
FIXED-IN: 7.0.0

M  +2    -1    NEWS
M  +1    -1    core/libs/metadataengine/dmetadata/dmetadatasettingscontainer.cpp

https://invent.kde.org/kde/digikam/commit/56082288b29c19eaacfeb9dab6fb16e6d1b8bff3
Comment 5 Maik Qualmann 2020-01-29 22:07:30 UTC
Note: For the change to take effect, a reset to the default settings in the advanced metadata settings is required.

Maik
Comment 6 harald.aust 2020-04-28 08:25:27 UTC
Hi,

just to let you know that this seems to be only partially fixed in 7.0.0-rc. Or rather, it is weirder than before.

Observation: When doing a "Reread metadata" on pictures that contain a "." in a Tag name, all of them get the correct tag assigned. However, an additional (incorrect) structured Tag is still created, and *some* of the pictures (but not all of them)get the structured tag assigned, too. See attachment.

If I don't reread metadata on an existing collection, but add a new collection, the structured tag is created, but none of the newly imported pictures gets assigned either the structured nor the correct tag...
Comment 7 harald.aust 2020-04-28 08:26:02 UTC
Created attachment 127941 [details]
Structered tag as well as correct tag
Comment 8 Maik Qualmann 2020-04-28 11:22:43 UTC
Did you reset to default settings in the advanced metadata settings?

Otherwise, upload a test image that you think is having a problem.

Note: if you re-read the metadata, the "incorrect" tags will not be removed from the tag tree.

Maik
Comment 9 harald.aust 2020-04-28 15:20:58 UTC
I had not, because I had already set it up, but have done now. Results arenow different but still not ok.

Steps to reproduce:
- Remove collection that contains images with "." tag
- Delete tag with "." in it from "caption" list
- Add collection that contains images with "." tag

Result: The correct tag is created, but is not assigned to the pictures that have it in their "keywords" and "subject" metadata (i.e. it is neither checked in the "Captions" tab, nor does it show up in the "Properties" tab)
Comment 10 harald.aust 2020-04-28 15:22:07 UTC
Created attachment 127946 [details]
Sample picture with "." tag that does not show up in digiKam
Comment 11 Maik Qualmann 2020-04-28 16:35:40 UTC
Created attachment 127948 [details]
tags.png

I can not reproduce a problem with the image. Look at the screenshot. According to the metadata contained in the image, the result is correct.
You may have to temporarily activate the option to clean up the database in the digiKam metadata setup and re-read the metadata.

Maik
Comment 12 harald.aust 2020-04-29 09:37:47 UTC
Created attachment 127979 [details]
After 'Add Collection'

Maik,

Interesting.

'Clean up metadata' is always checked. (Is this not a good idea?)

Observation: After adding the collection, the tag is created but not assigned. See screenshot.
Comment 13 harald.aust 2020-04-29 09:39:18 UTC
Created attachment 127980 [details]
Metadata after 'Add Collection'

Image contains keyword alright (and also XMP:Subject).
Comment 14 harald.aust 2020-04-29 09:41:53 UTC
Created attachment 127981 [details]
After 'Reread Metadata From File'

Only after "Reread Metadata From File", the tag is assigned and the situation is as it should be.

Is *there* something wrong, or am *I* doing something wrong?

Best regards,
Harald
Comment 15 Maik Qualmann 2020-04-29 11:06:11 UTC
I understand your problem in such a way that the tags do not appear in the digiKam GUI if you have added the image to the collection? Only after re-reading the metadata?

Then the question is, how do you add images? At runtime from digiKam? With the import tool from digiKam? With the Explorer?

Maik
Comment 16 harald.aust 2020-04-29 13:22:01 UTC
Maik,

Not exactly. What I have done so far is not adding a picture to an existing collection but adding an entire collection.

I have done more tests and found that the problem probably is not just related to tags with a "." but to tags in general.

Steps to reproduce:

1. Add a collection. Make sure that some tags in this collection are already known to digiKam ("type A"), some are not ("type B").
=> Tags in pictures of this collection that digiKam already knows ("type A") are assigned correctly.
=> Tags in pictures of this collection that digiKam does not yet know ("type B") are created in digiKam (i.e. are listed in the "Captions" pane) but are *not* assigned to the pictures.

2. Add a picture to the new collection, outside of digiKam, e.g. by moving an additional picture to the collection's folder.
=> New picture shows up in digiKam.
=> Tags in this new picture of "type A" are assigned correctly, tags of "type B" are not.

3. Do "Reread metadata from files"
=> Now all tags ("type A" and "type B") are assigned correctly.

4. Add a picture to the collection, outside of digiKam, e.g. by moving an additional picture to the collection's folder.
=> New picture shows up in digiKam.
=> All tags ("type A" and "type B") are assigned correctly.


So it seems that after creating a new collection and before doing a "Reread Metadata", tags that have been introduced to digiKam with this collection are listed but still somehow "unknown" and not assigned.

Hope this helps.

Best regards,
Harald
Comment 17 Maik Qualmann 2020-04-29 19:12:41 UTC
Could it be that the images are "duplicate" or images that have already been included in the digiKam database? When digiKam recognizes images by the unique hash, the properties of image A are transferred from the database to image B. Digikam assumes that you want the same tags, color labels, etc. for the same image. In any case, you should also activate the option in the metadata settings for re-reading the metadata when files have been changed.

Maik
Comment 18 harald.aust 2020-05-01 08:25:04 UTC
Maik,

That is well possible because I'm still in the "experimental phase", with just a few of my 40'000+ pictures. I'll try with new pictures and let you know.

"Rescan files when files are modified" is checked.

Best regards,
Harald
Comment 19 harald.aust 2020-05-01 08:47:34 UTC
Maik,

That was it. Tried with pictures digiKam had not seen before, and it worked as I expected it to -- old tags, new tags, with or without ".", were assigned correctly.

It is, of course, an implementor's choice, and probably not too relevant in non-evaluation scenarios, but to me, it seems rather counter-intuitive that when importing new pictures, it may happen that digiKam doesn't actually use the keywords that are in the pictures *now* but those that had been there *before*. But I'm sure you have your reasons for doing it this way.

Thanks for your help, and best regards,
Harald