| Summary: | Moving location of RAW file causes loss of metadata | ||
|---|---|---|---|
| Product: | [Applications] digikam | Reporter: | meku <kde.bugs> |
| Component: | Database-Mysql | Assignee: | Digikam Developers <digikam-bugs-null> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | metzpinguin |
| Priority: | NOR | ||
| Version: | 6.0.0 | ||
| Target Milestone: | --- | ||
| Platform: | Appimage | ||
| OS: | Linux | ||
| Latest Commit: | https://commits.kde.org/digikam/245740b1f390c9998a32a3e81f31c7be6cfb26a5 | Version Fixed In: | 6.0.0 |
| Sentry Crash Report: | |||
|
Description
meku
2018-12-20 13:26:23 UTC
It has nothing to do with a DATETIME(3) field. I'm watching tonight if I can reproduce the problem. When reading sidecar files, the modification date is now also queried to determine if there was an external change. In your configuration, if you do not write in sidecars, the metadata option "Cleanup Database if reread metadata" should NOT be enabled, otherwise the metadata in the database will be replaced by the sidecar/image metadata. If the images are moved within the same collection? Or between different drives/partitions? Maik I can not confirm or reproduce the loss of metadata. Therefore, I suspect that you have enabled the metadata option "Cleanup Database ...". The creation of a new image ID is done only when moving between different collections because the album root ID changes. This behavior is exactly the same in digiKam 5.9.0. I will see if I can optimize this behavior. Maik I have not enabled "Cleanup Database". I am testing with a single collection, on a single partition. I believe there is an additional condition that triggers this bug and it is related to the filesystem timestamps of the photos. Using 'stat' on my system shows nanosecond accuracy. Some files have not been modified for some years and show all zeros for the trailing digits, eg. Modify: 2013-01-29 14:31:22.000000000 other files have been modified recently and have use all the trailing digits, eg. Modify: 2018-12-23 01:06:52.322555833 This bug appears to be related to the files timestamp has non-zeros in the trailing digits. Furthermore this bug only triggers when a FOLDER is moved, it does not appear to trigger when an individual file is moved. STEPS TO REPRODUCE 1. Create at least 2 test folders with RAW files (include xmp sidecars to test with sidecars enabled) At least one RAW/xmp file should have full resolution timestamp. 'touch' file if necessary to update timestamp and use 'stat' to confirm. 2. Start DK5.9 and add collection with these folders. At this point you can choose whether to test the RAW or xmp timestamps. Select Metadata option 'Read from sidecar files' to test the xmp timestamps. 3. Add some metadata to the file that you 'touched', such as caption or tag. 3. Close DK5.9 4. Open DK6.0 5. Using Digikam album list, move the folder - such as dragging and dropping onto a subfolder. OBSERVED RESULT The metadata added in step 3 will have disappeared. And inspecting the Images database table will show duplicate records with same filename and uniqueHash. ADDITIONAL INFORMATION I have tested this on Ubuntu 18.04.1 with external database MariaDB v10.2.19 Altering the Images table to use DATETIME(3) for 'modificationDate' and then running 'Reread metadata from image' appears to mitigate this bug. It definitely does not matter, because SQLite has always saved with 3 additional digits. Such a date is scanned at most once and now the more accurate date is stored. But no metadata are lost. Are there any LR sidecar files in the directory? Digikam-6b3 will first look for sidecars in LR format and use these if they exist. I will go over their instructions step by step again. I see it right that their database is in the state of digiKam-5.9.0, since a back from 6.0.0 would not be possible because of the higher version number. Then the 3 additional digits will not exist after the start of digiKam-6.0.0 with the migration to V10. To get the extra 3 digits you need to create a new database and use the migration tool to convert the database. I'll check... Maik Git commit 245740b1f390c9998a32a3e81f31c7be6cfb26a5 by Maik Qualmann. Committed on 22/12/2018 at 21:22. Pushed by mqualmann into branch 'master'. various fixes to find a moved item in the database and use it FIXED-IN: 6.0.0 M +2 -1 NEWS M +14 -15 core/libs/database/coredb/coredb.cpp M +6 -8 core/libs/database/coredb/coredb.h M +1 -1 core/libs/database/item/scanner/itemscanner.h M +11 -6 core/libs/database/item/scanner/itemscanner_database.cpp https://commits.kde.org/digikam/245740b1f390c9998a32a3e81f31c7be6cfb26a5 I could reproduce the problem. The fact that it did not occur in digiKam-5.9.0 is also a bug when writing metadata to the database. Now that we use the modification date from either the image or the sidecar, it can not be part of the recognition of an image when searching for images with album id == NULL. Maik Git commit a24ab41458ec5a73a6aeb470cccefdd7ffbe449f by Maik Qualmann. Committed on 23/12/2018 at 08:08. Pushed by mqualmann into branch 'master'. reset grouped items cache M +4 -0 core/libs/database/item/containers/iteminfocache.cpp https://commits.kde.org/digikam/a24ab41458ec5a73a6aeb470cccefdd7ffbe449f Git commit db18a967f684e7c35e27f9c4c06b72b7fc7d170d by Maik Qualmann. Committed on 23/12/2018 at 13:15. Pushed by mqualmann into branch 'master'. disable external album scans when processing internal files M +6 -0 core/libs/album/engine/albumwatch.cpp M +24 -11 core/libs/database/utils/ifaces/dio.cpp M +6 -2 core/libs/database/utils/ifaces/dio.h https://commits.kde.org/digikam/db18a967f684e7c35e27f9c4c06b72b7fc7d170d |