Bug 380876 - Tags in Digikam DB maintained after being removed from file and file re-scanned
Summary: Tags in Digikam DB maintained after being removed from file and file re-scanned
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Tags-Engine (show other bugs)
Version: 5.6.0
Platform: Microsoft Windows Microsoft Windows
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-06-06 07:49 UTC by Sebas
Modified: 2022-09-03 10:35 UTC (History)
7 users (show)

See Also:
Latest Commit:
Version Fixed In: 6.0.0


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sebas 2017-06-06 07:49:42 UTC
Alright another tag issue here. Looks like bug 352711, but that one is outdated and I got some details here.

1. Assign EXIF XPKeywords tag to image using Windows Explorer (or possibly other software).
2. Add image to Digikam.
3. Remove EXIF XPKeywords from image using Windows Explorer. (Note: Digikam can't edit/remove EXIF XPKeywords tags anyways.)
4. Use any re(read) metadata functionality in Digikam to re(read) tags.
5. The removed EXIF XPKeywords tag is still being displayed by Digikam.
Comment 1 Maik Qualmann 2017-06-06 16:34:43 UTC
The problem is that Exif.Image.XPKeywords in Exiv2 is a read-only tag. We can not change or clear this tag at the moment.

Maik
Comment 2 Sebas 2017-06-06 18:06:37 UTC
Yes that's not the problem in this bug. The problem is that rescanning a tagless file still keeps Digikam displaying the removed tags, so Digikam must be loading them from database.

Please read description again.
Comment 3 Maik Qualmann 2017-06-06 18:21:25 UTC
I understand the problem. Yes, if a tag is no longer present, it is not deleted from the database. There are users who only store their metadata in the database. The database would be empty if the metadata was re-read from the images. We need a new option in the settings. Let's see if I can still make it into the 5.6.0 release.

Maik
Comment 4 Maik Qualmann 2017-08-23 18:33:47 UTC
*** Bug 383924 has been marked as a duplicate of this bug. ***
Comment 5 fabio 2017-08-23 19:30:48 UTC
(In reply to Maik Qualmann from comment #4)
> *** Bug 383924 has been marked as a duplicate of this bug. ***

Maybe you can enable the DB updating after a re-reading only for selected images.
And maybe add an extra warning message before wiping out the DB info, leaving the responsibility and the freedom in the hands of the user. Don't try to be too smart and let me think for myself...
Thanks.
Comment 6 Steve Braun 2017-12-28 14:28:02 UTC
I have the same bug, different situation:
My wife and I both use digikam with one shared album on our fileserver, so all tags are stored in the JPEG files.
But with the current behaviour it is impossible to have a consistent tag tree on both digikam instances, because when she removes a tag from some photos and I then "re-read the metadata from the images", the tags still remain in my digikam database.

In Comment 3 it sounded like there would be an option in one of the next releases, but as of today (I tested the current 5.8.0 prerelease on Windows) I couldn't find any.

Do you already have a time frame when this option would hit a release (if ever)?
Comment 7 caulier.gilles 2017-12-28 16:44:24 UTC
I'm not sure if Maik have patched the source code as explained in comment #3. He can confirm or not this point.

Happy new year.

Gilles Caulier
Comment 8 Maik Qualmann 2017-12-28 19:06:15 UTC
No not yet. I will now make it for digiKam-5.9.0. Meanwhile, my idea is that if we do a full scan of the image, optionally delete all tags and comments from the image in the database beforehand. I think that's the easiest way.

Maik
Comment 9 Nicofo 2018-01-15 17:02:14 UTC
Another idea, which seems me easy: duplicate the reread metadata function. In the menu, There will be:
- Update metada from picture (= present behaviour)
- Reread metadata from picture (= clean all metadata and reread)

Just an idea...
Comment 10 Maik Qualmann 2018-03-11 19:17:28 UTC
*** Bug 391720 has been marked as a duplicate of this bug. ***
Comment 11 MarcP 2018-03-11 19:56:45 UTC
I understand there might be people who prefer to have a read-only picture library, and only using digikam's database to organize it.

In the configuration menu, there are already a few checkboxes asking if the metadata should be or should not be written to the picture files. If these boxes are checked, one could assume that metadata from pictures should prevail over already-existing metadata in the database. Maybe another option like "Always sync picture metadata with digikam's database" could be placed there.

If I am not mistaken, that is the default behavior in most photography softwares, from microsoft picture viewer to Google's Picasa, ACDsee, or Adobe Lighthroom. They all write metadata on files by default and without confirmation.
Comment 12 MarcP 2018-03-20 23:15:15 UTC
I've been doing some tests to determine the exact nature of this behavior. 

Let's say that some tag was removed from a group of pictures, so it no longer exists. Digikam does not remove that tag from the Tag list (and the count number indicating how many pictures have it remains untouched). Only removing the picture collection from the database will bring that count to 0 (but the tag will still appear in the list). Upon adding the picture collection once more, and letting digikam scan for pictures, tags will be populated again. But even the deleted tag, that no picture has anymore, and it was empty in the database, will be populated once more and recover its previous count.

Not even cleaning the database, even when there isn't a single collection, will get rid of these inexistent tags.

Only manually deleting the empty tag will get rid of it, but that would require a previous knowledge of which tags have been modified (possibly by another user). So the only option that currently remains, in order to properly update the library, is deleting the whole database and start anew.

I hope it helps...
Comment 13 Maik Qualmann 2018-04-12 18:21:02 UTC
Git commit 6ed95813516deeaebfb0275625a892fedbb601c4 by Maik Qualmann.
Committed on 12/04/2018 at 18:19.
Pushed by mqualmann into branch 'master'.

add function to remove all image attributes before rescan,
not yet configurable via the setup

M  +55   -3    core/libs/database/coredb/coredb.cpp
M  +12   -3    core/libs/database/coredb/coredb.h
M  +5    -0    core/libs/database/item/imagescanner.cpp

https://commits.kde.org/digikam/6ed95813516deeaebfb0275625a892fedbb601c4
Comment 14 Maik Qualmann 2018-04-14 16:14:46 UTC
Git commit cfa88b7f0f8fae3e2aa44f152a93123ae9328a4e by Maik Qualmann.
Committed on 14/04/2018 at 16:13.
Pushed by mqualmann into branch 'master'.

remove in the DB all old image informations when rereading metadata from file
when all 8 metadata write options are enabled
Related: bug 392017, bug 352711
FIXED-IN: 6.0.0

M  +4    -1    NEWS
M  +35   -12   core/libs/database/coredb/coredb.cpp

https://commits.kde.org/digikam/cfa88b7f0f8fae3e2aa44f152a93123ae9328a4e
Comment 15 Sebas 2019-04-27 17:28:58 UTC
Still happened in 6.0 for MOV-files. No results for other filetypes.
Comment 16 Maik Qualmann 2019-04-27 23:27:53 UTC
If you remove metadata externally and digiKam should also remove it, you must enable the option in the metadata setting to clean up the database.
This is a option and no longer depends on whether you have enabled write options.

Maik
Comment 17 Sebas 2019-04-28 07:30:36 UTC
I removed tags from MOV-files in Digikam, then removed the tags from database. Then, after starting with a new database due to the PNG-bug, the MOV-files came back with the tags attached ánd enabled again.
Comment 18 Maik Qualmann 2019-04-28 10:36:07 UTC
You can not remove tags in MOV files. DigiKam can not write to video files. You would have to activate sidecar and if this sidecar is then available, digiKam would not add the tags again because they will no longer exist in the sidecar.

Maik
Comment 19 Sebas 2019-04-28 14:07:13 UTC
Understood. So if I want to only write to sidecar for unsupported files I should enable:
[X] Write to sidecar files: Write to XMP sidecar for read-only item only

And to later read from it, I guess also:
[X] Read from sidecar files
Comment 20 Maik Qualmann 2019-04-30 07:51:26 UTC
Yes, this is the correct settings.

Maik
Comment 21 Ralf S 2022-08-31 12:15:04 UTC
>> If you remove metadata externally and digiKam should also remove it, you must enable the option in the metadata setting to clean up the database. 
>> This is a option and no longer depends on whether you have enabled write options.

May I ask which option you mean in particular? On 7.7.0 / MacOS I can’t find anything like it, and still struggle with this old tag issue. But maybe I’m doing something wrong.
Comment 22 Maik Qualmann 2022-09-01 06:18:41 UTC
The setting for cleaning up the database is in the digiKam settings under Metadata as a check box. If external tags are removed, they are otherwise not removed from the images in the database. This is intentional, since there are users who only write their information to the database, otherwise all information would be lost when the metadata was read in again.
However, cleaning up the database does not mean that the tags are also deleted from the tags tree. If a tag is actually no longer assigned to an image, you can completely remove the tag in the Tags Manager.
If the tag reappears when you rescan images, you will still have images that have the tag still present in their metadata / sidecars.

Maik
Comment 23 Ralf S 2022-09-01 18:42:32 UTC
Thanks a lot for your explanation, Maik – what you describe is essentially what  I had expected. Unfortunately reality doesn’t exactly match my expectations:

I have checked 'Clean up the metadata from the database when rescan files' in Settings > Metadata > Behaviour. My test image is a DNG that I want ot keep in sync with Adobe Lightroom (I actually plan to get rid of the latter in the near future).

The original Lightroom DNG had the (hierarchic) keywords 'food', 'vergetables' and 'potatoes' saved inside the DNG (afaik as an XMP). When I scan the image in Digikam, everything is great, including the hierarchic structure. Changing and saving the keywords in Digikam and rescan the image in Lightroom is working flawless as well, but not the other way.

If I’m changing the keywords in Lightroom, let’s say to 'agriculture' and nothing else, save the metadata and do the Album > Reread metadata from files command in Digikam, all metadata will be updated, except the keywords, 'food', 'vergetables' and 'potatoes' – they remain as if they’re carved in stone, obviously no chance to get rid of them. And I don’t mean the overall tag list – where they are welcome – but in the assigned file’s metadata.
Comment 24 Maik Qualmann 2022-09-01 21:15:50 UTC
Lightroom will probably only change a metadata type e.g. Xmp.lr.HierarchicalSubject or Xmp.dc.Subject. digiKam writes its own XMP tags list in the metadata and reads them out again by default. However, since Lightroom does not change these, digiKam will not recognize anything that has changed. Go to the advanced digiKam metadata settings, here you can specify which tag metadata digiKam reads first, the list corresponds to the order from top to bottom in which digiKam queries the metadata in the file. You'll probably need to move Xmp.lr.HierarchicalSubject to the top.

Maik
Comment 25 Ralf S 2022-09-03 10:35:49 UTC
Thanks Maik, you led me on the right path. I fiddled around with different orders and with [1] Xmp.lr.HierarchicalSubject, [2] Xmp.dc.Subject [3] Xmp.digiKam.TagsList everything seems to work like I want it, in both directions. Great!