Version: 7.1.0 OS: Mac OS X High Sierra I import photos and videos into Digikam by copying the files directly into a scanned directory and Digikam automatically picks up the new file and processes it. Most of the time, the "Creation date" is scanned and annotated correctly but in any bunch of new files, at least one or two videos will invariably have "right now" as the "Creation date", and often more. Example: I copied over 14 GoPro videos with sequentially named GOPR00[15|28].MP4, and with the actual file creation date ranging from 2019-06-02T09:27 to 2019-06-02T11:05. Exactly half (7) were detected by Digikam as having a creation date of 2020-09-24T08:51. The incorrectly scanned files are not in any sequential order that I can tell. File ending in 19,20,22,23,24,25,26 have the wrong date while files ending in 15,16,17,18,21,27,28 are all correct. I will look more closely at two representative examples. GOPR0026.MP4 Digikam Creation Date: Sep 24, 2020 8:51AM $ GetFileInfo -d GOPR0026.MP4 06/02/2019 10:59:26 $ exiftool GOPR0026.MP4 | grep "Create Date" Create Date : 2019:06:02 10:59:26 GOPR0027.MP4 Digikam Creation Date: June 02, 2019 11:04AM $ GetFileInfo -d GOPR0027.MP4 06/02/2019 11:04:45 $ exiftool GOPR0027.MP4 | grep "Create Date" Create Date : 2019:06:02 11:04:45 The files were physically created five seconds apart, back in 2019. All utilities I can find either command-line or GUI show the "proper" Creation date. Digikam, for some reason, ignores that and chooses the scan time for one of those two but not the other. Let me know if this is a known issue with scanning (maybe on Macs) or if you need me to track down any other debugging info. This is a very minor issue but annoying, since my video feeds tend to "tell a story" in order and having them sorted by date in the wrong order is frustrating.
It figures that after months of seeing this with no known cause or workaround, I have potentially found both literally minutes to hours after posting this bug. Maybe a bit of rubber duck debugging. Anyway, the light bulb went off initially when doing an Import->Add Images rather than a copy-and-auto-scan and seeing that not only were all of the dates correct, but it also took a decent amount of time to import. Why so much time? Well, these files are quite large -- some are multi-gigabyte and it simply takes notable time to copy that much data on a mechanical HDD. Oh... could that be it? I did a test were I did a manual copy of some video files into another directory and noticed that MacOS reports the 'creation time' of the files as 'right now' for the duration of the copy and only sets the actual 'creation time' to match the original AFTER the copy is completed. So could this be a race condition? Is Digikam maybe monitoring the albums/directories on a set polling basis and could it simply be encountering the "wrong date" video files BEFORE the copying is completed? Seems plausible. With that in mind, there is a potential workaround (not counting the Import) and that is to run Tools -> Maintenance -> Sync Metadata and Database on the given albums after everything is successfully copied over. I make sure the sync direction is "From image metadata to database". That appears to work like a charm. Another option might be to disable auto-scan and just remember to Tools -> Scan for new items regularly. If it really is just a race condition then it is still a bug (shouldn't be adding the file until its state has solidified) but maybe that's not a bug worth fixing since it's such an esoteric case and has known workarounds?
Hi, First video metadata are handle by ffmpeg, not Exiv2 and ExifTool. Second, ffmpeg will try to parse files to extract metadata. If large files are under process to be copied and are uncomplete, the parse will fail and current file date from file system will be used instead. This explain the dysfunction about the date. Gilles Caulier
Normally we append a "digikamtempfile" to the file name during the copying process. The parser ignores these files, so there shouldn't be any problems during the copying process. I'll take a look at it. Maik
Hi, digiKam 7.2.0-beta2 pre-release PKG installer now support BigSur and is compiled with last stable Qt 5.15.2. https://files.kde.org/digikam/ Problem still reproducible with this version. Thanks and happy Christmas in advance Best Regards Gilles Caulier
Kurt, To solve this issue, if it relevant of Exiv2 or ffmpeg, we will have the possibility later to use ExifTool with Video File instead ffmpeg. With DK 7.3.0, ExifTool is now supported has metadata backend. For the moment only the viever in right sidebar use ExifTool, but we plan to use ExifTool to handle special cases or dysfunctions not working with Exiv2 or ffmpeg. If you can share the famous video file taken with your gopro (or a short sample) to test here, it will help to investigate. Best regards Gilles Caulier
Giles, thank you so much for investigating this. You are definitely going down an excellent trail. Unfortunately, it's not feasible for me to upload a representative video sample. As I mentioned in the original report, this behavior almost exclusively happens with large files. The GOPR0026.MP4 given as an example, for instance, is 1.35GB!! Alas, I simply don't see this issue with smaller and more reasonable files.
Git commit b3c56d4cdedb4ad83c7baca89c32d011f1738090 by Gilles Caulier. Committed on 26/04/2021 at 08:58. Pushed by cgilles into branch 'master'. Use EXV container to translate automatically all metadata from ExifTool to Exiv2 Related: bug 421464, bug 416516, bug 360714 M +23 -76 core/libs/metadataengine/exiftool/exiftoolparser.cpp M +2 -5 core/libs/metadataengine/exiftool/exiftoolparser.h M +13 -38 core/tests/metadataengine/exiftool/exiftoolexport_cli.cpp https://invent.kde.org/graphics/digikam/commit/b3c56d4cdedb4ad83c7baca89c32d011f1738090
Git commit 183a05476b1fa485e5143e0bb28d84e3a8daa471 by Gilles Caulier. Committed on 26/04/2021 at 09:08. Pushed by cgilles into branch 'master'. Remove obsolete ExifToolTranslator class. We use now the auto conversion to Exiv2 through EXV container processed by ExivTool Related: bug 421464, bug 416516, bug 360714 M +0 -6 core/libs/metadataengine/CMakeLists.txt M +10 -114 core/libs/metadataengine/exiftool/exiftoolparser.cpp D +0 -81 core/libs/metadataengine/exiftool/exiftooltranslator.cpp D +0 -65 core/libs/metadataengine/exiftool/exiftooltranslator.h D +0 -196 core/libs/metadataengine/exiftool/exiftooltranslator_exif.cpp D +0 -51 core/libs/metadataengine/exiftool/exiftooltranslator_ignoredgroups.cpp D +0 -164 core/libs/metadataengine/exiftool/exiftooltranslator_iptc.cpp D +0 -76 core/libs/metadataengine/exiftool/exiftooltranslator_p.cpp D +0 -73 core/libs/metadataengine/exiftool/exiftooltranslator_p.h D +0 -439 core/libs/metadataengine/exiftool/exiftooltranslator_xmp.cpp M +0 -1 core/libs/widgets/metadata/exiftool/exiftoollistview.cpp M +0 -1 core/tests/metadataengine/exiftool/CMakeLists.txt D +0 -318 core/tests/metadataengine/exiftool/exiftoolxmldb_cli.cpp https://invent.kde.org/graphics/digikam/commit/183a05476b1fa485e5143e0bb28d84e3a8daa471
Git commit 5b097916d8bfa42fef5a2966fa3b34d0793cf0e2 by Gilles Caulier. Committed on 26/04/2021 at 11:02. Pushed by cgilles into branch 'master'. separate ExitTool parser and process classes internal container implementations Related: bug 416516, bug 360714 M +2 -0 core/libs/metadataengine/CMakeLists.txt M +15 -154 core/libs/metadataengine/exiftool/exiftoolparser.cpp M +0 -11 core/libs/metadataengine/exiftool/exiftoolparser.h A +132 -0 core/libs/metadataengine/exiftool/exiftoolparser_p.cpp [License: GPL (v2+)] A +83 -0 core/libs/metadataengine/exiftool/exiftoolparser_p.h [License: GPL (v2+)] M +10 -209 core/libs/metadataengine/exiftool/exiftoolprocess.cpp M +10 -20 core/libs/metadataengine/exiftool/exiftoolprocess.h A +171 -0 core/libs/metadataengine/exiftool/exiftoolprocess_p.cpp [License: GPL (v2+)] A +107 -0 core/libs/metadataengine/exiftool/exiftoolprocess_p.h [License: GPL (v2+)] M +0 -1 core/tests/metadataengine/exiftool/exiftoolmulticore_cli.cpp M +0 -1 core/tests/metadataengine/exiftool/exiftooloutput_cli.cpp M +1 -1 core/tests/metadataengine/exiftool/exiftoolparserout_cli.cpp https://invent.kde.org/graphics/digikam/commit/5b097916d8bfa42fef5a2966fa3b34d0793cf0e2
Git commit 1dff0b016bc15918440a807863757376e297c53e by Gilles Caulier. Committed on 27/04/2021 at 09:27. Pushed by cgilles into branch 'master'. With load EXV chunk method, also backport all possible metadata to XMP. This permit to get all main information from video files not supported by Exiv2, as Quicktime for MP4 Related: bug 416516 M +1 -0 core/libs/metadataengine/exiftool/exiftoolparser.cpp https://invent.kde.org/graphics/digikam/commit/1dff0b016bc15918440a807863757376e297c53e
Git commit febc26188fe4e6037e5c8a9b1840f65729972a47 by Gilles Caulier. Committed on 06/04/2022 at 20:38. Pushed by cgilles into branch 'master'. Add ExifTool backend to load metadata for image or video. Exiv2 is always used in priority. If it fail, ExifTool is used instead. ExifTool has the capability to create an EXV container for Exiv2. All metadata extracted with ExifTool are concatened to the EXV container with all possible translations with Exif, Iptc, and Xmp tags. Of course tags supported by ExifTool but not supported by Exiv2 are lost. Related: bug 385726, bug 416516, bug 360714, bug 264210, bug 419801, bug 419375, bug 103247 M +1 -0 core/libs/metadataengine/CMakeLists.txt M +7 -0 core/libs/metadataengine/dmetadata/dmetadata.h A +94 -0 core/libs/metadataengine/dmetadata/dmetadata_exiftool.cpp [License: GPL (v2+)] M +42 -17 core/libs/metadataengine/dmetadata/dmetadata_fileio.cpp M +7 -6 core/libs/metadataengine/engine/metaengine.h M +5 -0 core/libs/metadataengine/engine/metaengine_fileio.cpp M +4 -2 core/libs/metadataengine/exiftool/exiftoolparser_command.cpp https://invent.kde.org/graphics/digikam/commit/febc26188fe4e6037e5c8a9b1840f65729972a47
ExifTool backend is now used to scan video metadata and date is now properly extracted. Gilles Caulier