Bug 413938

Summary: Metadata is not written to all pictures in a tag hierarchy
Product: [Applications] digikam Reporter: MarcP <iwannaberich>
Component: Tags-KeywordsAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: iwannaberich, metzpinguin
Priority: NOR    
Version First Reported In: 6.4.0   
Target Milestone: ---   
Platform: Appimage   
OS: Linux   
Latest Commit: https://invent.kde.org/kde/digikam/commit/4736b12a1baa18a8435716f6334dc515f31c1160 Version Fixed/Implemented In: 7.0.0
Sentry Crash Report:
Attachments: Screen capture showing the issue
Case1.jpg
Case2.jpg

Description MarcP 2019-11-08 10:01:20 UTC 
SUMMARY
This bug is derived from this tread in the mailing list: http://digikam.1695700.n4.nabble.com/digiKam-users-Hierarchical-tags-how-are-they-stored-td4709630.html

In short, when creating a Tag hierarchy using existing tags by drag an dropping tags in the tree view, the metadata change is not saved to all pictures contained in the hierarchy (implying you are saving metadata to pictures, of course). The particular situation can be a bit hard to explain, so please fasten your seatbelts and I'll do my best.

As you know, a picture can be part of a tag hierarchy, but not necessarily contain all tags in that hierarchy. For instance:

[x] Canada
    [ ] Ontario
        [x] Toronto

The picture in this case has the tags Toronto and Canada, but not Ontario, although it's part of the hierarchy and it's located there in the tag tree, and can be browsed as part of Ontario (if "Include Tag Sub-Tree" option is checked, which is by default).

Usually, doing changes to a lower level tag (closer to the root) will imply that all pictures in the hierarchy will be modified as well. e.g., If I rename the tag "Ontario" to something else (e.g. "ONT"), all pictures containing Toronto will be written to reflect those changes. In the XMP metadata browser, which contains many different tag systems, changes will be reflected this way (for simplicity sake, just lightroom):

  lr:     hierarchicalSubject = Canada|Ontario|Toronto

to 

  lr:     hierarchicalSubject = Canada|ONT|Toronto



This is when things work normally, but there is one situation when this is not working properly, which is why I am reporting this bug.

Imagine you have pictures tagged either one of the following ways:

Case 1)                            Case 2)

    [X] Ontario        but also       [] Ontario
        [x] Toronto                       [x] Toronto

and you create a new root-level tag called  "Canada". Next, you drag and drop "Ontario" over "Canada". All pictures in Case 1 will be written and updated correctly:

from  lr:  hierarchicalSubject = Ontario|Toronto 
to    lr:  hierarchicalSubject = Canada|Ontario|Toronto

But pictures in Case 2 will NOT be updated. Their metadata will remain intact. Although changes have been correctly applied in the database. Until you manually select any of these pictures and use "Item, Write metadata to file" option.

Basically, if there is a "gap" in the hierarchy where a parent tag is not "checked", the metadata update will skip these files, which will not be updated unless they are updated manually. So this is a situation where the database and the metadata have become out of sync.

A symptom of this behavior can also be found in the warning when a large number of files are going to be modified (since these kind of reorganizations also imply hundreds or thousands of pictures). The number of affected pictures by the warning will be fewer than the total number of pictures included in the hierarchy. But this behavior can be easily reproduced in using just two pictures.

As I said, manually selecting the affected (non-updated) files and forcing writing meta-data to files updates them with the correct hierarchy once again.


SOFTWARE/OS VERSIONS

Linux: Digikam 6.4.0 (stable release) appimage in Ubuntu 18.04 LTS
Comment 1 MarcP 2019-11-08 10:05:21 UTC 
Created attachment 123791 [details]
Screen capture showing the issue

This is a screen capture showing the problem with just two files. Observe how the Case2.jpg picture did not had its metadata updated when "Ontario" was moved inside "Canada", unless picture Case1.jpg, which did.

Manually selecting Case2.jpg and doing "Item, Write metadata to file" finally updated the metadata. This should have been done automatically in the previous step.
Comment 2 Maik Qualmann 2019-11-08 11:01:03 UTC 
Marc,

how it handles digikam is the metadata standard and completely correct. How should a tag list indicate that the tag is part of the path but not marked? The only tag list that allows this is the proprietary format of ACDsee. DigiKam can handle this format, you could push it as default all the way up in the advanced metadata settings. But I think except digiKam and ACDSee itself, nobody will use this tag list.

Maik
Comment 3 Maik Qualmann 2019-11-08 11:34:18 UTC 
Can you provide 2 test files that are already tagged with just one more step to reproduce the problem?

Maik
Comment 4 Maik Qualmann 2019-11-08 11:41:23 UTC 
Updating a "tag"move" is an interesting task, we update the tags that are directly affected. Otherwise, we will have to recrusively update all other tags. That could be a long task. Actually, we have the maintenance tool to do such things.

Maik
Comment 5 MarcP 2019-11-08 12:01:11 UTC 
Sure, no problem. I have attached two pictures. This is their current and only metadata:

  Case1.jpg                         Case2.jpg

  [X] Ontario                       [ ] Ontario
      [x] Toronto                       [x] Toronto

Now create an empty tag called "Canada" at the root of your tag tree, and then drag and drop the tag "Ontario" on "Canada". Observe how metadata is updated in Case1.jpg, but not in Case2.jpg. Now, if you click on "Item, Write metadata to file", the metadata is updated. But this last step should have been done automatically.

This issue deals with how digikam tries to propagate a change across a tag tree. Usually, the whole tree is saved in each individual picture (so each image contains a "template" of its parent tags). When changing a parent tag, this change is also saved in each individual picture. However, if some intermediate tag is not "checked", changes are not propagated to that branch.

Btw, I am not complaining about how digikam handles metadata. I think it does a great job considering how different software uses different standards, and it tries to integrate all of them, which is perfect.

I honestly wouldn't have a problem if a picture in a tag tree would be tagged with each one of its subtags, just like lightroom does. The only consequence would be a little bit of clutter because of the amount of tags under each miniature. But in any case, the current implementation is perfectly fine and gives some more flexibility while maintaining the compatibility with other picture managers.

PS: Yes, generally recursively updating a tag tree is something that can affect hundreds or thousands of pictures, and take a long time, but the user is already being warned about it when more than 100 pictures are going to be modified. The user has also the option of using sidecars, which would greatly improve the speed. I think it should be their responsibility if they want to make these big changes in their library. Actually, the recursive update is already the default behavior, except in the very particular case described in this bug report.
Comment 6 MarcP 2019-11-08 12:01:47 UTC 
Created attachment 123793 [details]
Case1.jpg
Comment 7 MarcP 2019-11-08 12:02:05 UTC 
Created attachment 123794 [details]
Case2.jpg
Comment 8 Maik Qualmann 2019-11-08 12:24:06 UTC 
Ok, with the sample files I can reproduce it. I even think that the fix is very simple.

Maik
Comment 9 Maik Qualmann 2019-11-08 17:33:48 UTC 
Git commit 4736b12a1baa18a8435716f6334dc515f31c1160 by Maik Qualmann.
Committed on 08/11/2019 at 17:32.
Pushed by mqualmann into branch 'master'.

get all recursive tags from the moved tag
FIXED-IN: 7.0.0

M  +2    -1    NEWS
M  +1    -1    core/libs/album/manager/albummanager_talbum.cpp

https://invent.kde.org/kde/digikam/commit/4736b12a1baa18a8435716f6334dc515f31c1160
Comment 10 MarcP 2019-11-08 17:54:56 UTC 
Wow, you are the best!