Bug 338533

Additional note :
when you move over a thumbnail, the small windows that appears with some infos on the image shows "Created the : ...." but it's not the creation date that is shown, it's the last modification date (2014/08/24 in my case, while the photo was taken on the 2014/08/16)

Mohamed,

Another entry about icon-view sort by date method, recently add with last 4.2.0 release.

Gilles

Nicolas,

Can you provide some photo to reproduce the problem in comment #1 and check which metadata tag is used.

Gilles Caulier

Hello

I was about to send some photos, but then I tried first "album / read again metadata from the images", and this fixed the problem, photos are now sorted in their correct order.
So it seems the problem is not that digikam doesn't use the correct exif tag but that album's order is not refreshed when some new images are added to an album while digikam is already running.

If I recall correctly, I had an album with some images in dir "album_2014/", taken with the same camera. Then, I copied some photos from another camera in this same directory, using "cp" from a shell. Those new photos took the date/time when "cp" was made. Then I ran "exiv2 -T mv *jpg" to update the file's date of all the photos (based on the creation date).
And I think this is where digikam didn't update the sorting order.

Does digikam support when images are changed by another program while the album is shown in digikam ? I thought inodes were monitored and digikam would rescan automatically the album, as if the image was not changed but completely added ?
Or is this something that should be run manually with "read again metadata from the images" ?

It do not support automatically a sync to database with metadata changed externally.

You must sync DB with metadata using relevant digiKam maintenance tool

Gilles Caulier

OK, thanks.
But maybe this is something that could be added ?
I agree that if some modifications are made while digikam is not running, digikam can't guess on next start that it must rescan some metadata.
But in the case where digikam is running and monitoring the files/dir inodes in the albums collection, maybe it could start a metadata's rescan if the OS reports a change in some files ? (update thumbnails, exif, ...)

There is already a report about metadata sync to database when it changed by an external application.

Gilles Caulier

Do you mean that if I change exif dates in a file, then digikam will perform a metadata sync to database and "refresh" data for this image ?

If so, I tried it, and it didn't work. In a folder with ~20 images took in 2014-08-16, I choose "sort by date" under digikam. Then from a shell, I run "exiftool -AllDates='2008:05:21 13:54:13' IMG_20140816_174711.jpg" on one of the image. Expected result is that this image should now be the 1st one with "sort by date" order, but the album view is not modified.
It's only if I select "album / read metadata" that the album view is updated.

If I do the same test again (set date to 2008-05-21) for one image, then move this image outside of the album (from shell "mv IMG_20140816_174711.jpg /tmp"), then album is correctly updated and there's one image less.
But now, if I move back the image in the album ("mv /tmp/IMG_20140816_174711.jpg ."), then the image is still not correctly sorted at the start of the list.
This looks strange to me, shouldn't digikam rescan the new image when it is added, why does it keep some previous data and the wrong sort position ? (does digikam assumes that it's the same image than before just because it's the same name ?)

Nicolas

This file still valid using last digiKam 5.0.0 ?

Gilles Caulier

hello
can't say at the moment, I'm still using kde4 / digikam 4 under mageia 5.
will give it a another look later when mageia 6 is released.

Nicolas

Yes, it is still valid in digikam 5.2.0. Digikam should respect the exif date of creation.

What's about this file using digiKam AppImage bundle 5.4.0 pre release given at this url :

https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM

Gilles Caulier

Hello,

sorry to resurrect this old thread. I think I still have the same problem. 

Some files were modified by an external software (tags or face rectangles were added), so for some reason, the Exif "Creation date and time" was updated to that moment.

However, these pictures already had an "Original date and time", which is left untouched. When digikam sorts files by date, it uses the "Creation date and time" instead of the "Original date and time" (the creation date, as I understand it).

So basically my photos are sorted in the order that they have been last modified.

I can provide some samples if you wish.

I have made a few screenshots showing how the sort date is determined from the metadata: https://imgur.com/a/Scn67Pb

In this example, the correct date should be 2017-02-17, but the date in "File properties" shows 2018-09-09 (the last time this picture was tagged). Examining the EXIF, we can see that under "Image Information" there's the modify date (called "Date and Time" and below that, in "Photograph Information" there's the "Date and Time (original)", which should be the correct display date. IPTC and XMP show the correct creation and modify dates.

On the metadata editor, the "Creation date and time" is the modify date, and the "Original date and time" is the correct date for the picture.

By the way, this is a problem with pictures that weren't taken with a digital camera (scanned, or sent via whatsapp). For pictures with complete EXIF data, even if the modify date is changed, the file is sorted by creation date. So it must have something to do with how date/time is added to photos without previous exif information, possibly because of the lack of a date in the "Photograph Properties" section. 

I hope I didn't make things more confusing than they were before.

More tests:

The behavior is different in pictures without EXIF that had the date/time added manually, than pictures taken from a digital camera that already have EXIF. 

It seems that the date Digikam uses is variable. In some cases, if "Digitization date" is set manually, that's the date that Digikam uses (overriding "Creation date and time" and "Original date and time"), but in some other cases, "Creation date and time" prevails over "Original date and time" and "Digitization date and time".

But I haven't found why yet.

Anyway, if Digikam would sort pictures by "Original date and time" by default, would solve many of these issues.

IWBR,

The rules to get date time-stamp from image to populate the database, can be relevant of advanced metadata settings page from configuration dialog. Did you take a look in this settings ?

Gilles Caulier

The function is hardcoded and is located here:

https://cgit.kde.org/digikam.git/tree/core/libs/dmetadata/metaengine_image.cpp#n526

Last I changed it, so that erroneous EXIF, IPTX or XMP entries are recognized, that at least 2x the same date must be found. If only one entry exists, it must have a valid time. It is also possible to see in which order the date is searched. If this function does not find a valid date, the file date will be used.

Maik

It's more complex than I though then.

In my particular case, I have a bunch of pictures sent via whatsapp that do not have metadata in them, but their filename includes the date. Then I wrote a script to enter EXIF date using the "exiftool -aldates" command. This creates a "Create Date", a "Modify Date" and "Date/Time Original" (in exiftool's words). 

So a picture without previous metadata that has been passed through the script will look like this:

user@Equip-3:~/Escriptori$ exiftool IMG-20180303-WA0021.jpg | grep Date
File Modification Date/Time     : 2018:09:13 14:32:54+02:00
File Access Date/Time           : 2018:09:13 14:32:54+02:00
File Inode Change Date/Time     : 2018:09:13 14:32:54+02:00
Modify Date                     : 2018:03:03 00:21:00
Date/Time Original              : 2018:03:03 00:21:00
Create Date                     : 2018:03:03 00:21:00


However, when an external program modifies an image (usually by tagging something in it), the Modify Date changes, and that seems to be the value that Digikam uses as the picture date. It looks like this in exiftool:

user@Equip-3:~/Escriptori$ exiftool IMG-20170217-WA0011.jpg | grep Date
File Modification Date/Time     : 2018:09:09 13:54:25+02:00
File Access Date/Time           : 2018:09:13 14:28:51+02:00
File Inode Change Date/Time     : 2018:09:13 14:28:51+02:00
Modify Date                     : 2018:09:09 13:54:25
Date/Time Original              : 2017:02:17 12:35:26
Date Created                    : 2017:02:17
Date/Time Created               : 2017:02:17 12:35:26+00:00

In Digikam, the "Creation date and time" appears as 9/9/2018 13:54:25, and the "Original date and time" is 17/2/2017 12:35:11. So in this case, Digikam's is using what exiftools (and the other picture manage software) considers the Modify date as the main picture date. Note that "Creation date and time" in Digikam corresponds to the "Modify date" in exiftool. By the way, to further complicate things, the creation date in IPTC is "17/02/2017 12:35:25" and the creation date in XMP is 09/09/2018 13:54:26". I guess they have been synchronized from the EXIF date at some point (I honestly could do without them).

I think there's some confusion on how Digikam calls the different date fields. "Creation date and time" should be called "Modify date" and "Original date and time" should be called Creation date and time". 

Then a possible solution (that I've seen in another software) would be to be able to sort pictures by "Original date" (when pictures were taken) or by "Modify date" (when pictures were last changed).

digiKam do not use exiftool, but Exiv2 shared library.

To compare properly, please use Exiv2 CLI tool, not Exiftool. This can give some differences.

Gilles Caulier

I see. This is what exiv2 returns for the same picture (the second one).


user@Equip-3:~/Escriptori$ exiv2 -pa IMG-20170217-WA0011.jpg
Exif.Image.Software                          Ascii       7  Picasa
Exif.Image.DateTime                          Ascii      20  2018:09:09 13:54:25
Exif.Image.ExifTag                           Long        1  78
Exif.Photo.ExifVersion                       Undefined   4  2.20
Exif.Photo.DateTimeOriginal                  Ascii      20  2017:02:17 12:35:26
Exif.Photo.PixelXDimension                   Long        1  1600
Exif.Photo.PixelYDimension                   Long        1  900
Exif.Photo.ImageUniqueID                     Ascii      33  3b7453056144d41f3492b4d48afcc271
Iptc.Envelope.ModelVersion                   Short       1  4
Iptc.Envelope.CharacterSet                   String      3
Iptc.Application2.RecordVersion              Short       1  4
Iptc.Application2.DateCreated                Date        8  2017-02-17
Iptc.Application2.TimeCreated                Time       11  12:35:26+00:00
Xmp.xmp.ModifyDate                           XmpText    25  2018-09-09T13:54:25+02:00
Xmp.exif.DateTimeOriginal                    XmpText    25  2017-02-17T12:35:26+00:00



So exiftool calls "Modify Date" to what exiv2 calls DateTime, and exiftool calls "Date/Time Original" and "Date/Time Created" to what exiv2 calls "DateTimeOriginal". I can see the confusion.

In any case, some programs change "Exif.Image.DateTime" when the picture is edited, and use the "Exif.Photo.DateTimeOriginal" for sorting purposes.

I also checked the official EXIF documentation at https://www.exif.org/Exif2-2.PDF to see what is the "official" purpose of each exif tag.

DateTime 
The date and time of image creation.  In this standard it is the date and time the  file was changed. 

DateTimeOriginal
The date and time when the original image data was generated.

DateTimeDigitized
The date and time when the image was stored as digital data. If, for example, an image was captured by DSC and at the same time the file was recorded, then the DateTimeOriginal and DateTimeDigitized will have the same contents.


So it is correct to use the DateTime as the last modification date, and DateTimeOriginal as the original creation date of the picture.

Ok, we can give DateTimeOriginal a higher priority. Just take a look...

Maik

Git commit d69523ba54d640b9918c69853bea03b893e75874 by Maik Qualmann.
Committed on 13/09/2018 at 21:07.
Pushed by mqualmann into branch 'master'.

give DateTimeOriginal a higher priority
this fix does not break Bug 386959

M  +18   -18   core/libs/dmetadata/metaengine_image.cpp

https://commits.kde.org/digikam/d69523ba54d640b9918c69853bea03b893e75874

The IMG-20170217-WA0011.jpg image to test would be good...

Maik

Created attachment 114944 [details]
IMG-20170217-WA0011.jpg

File used as an example that has been mentioned in the comments. There are two dates in this image: 2017-02-17 and 2018-09-2018. The first is the creation/original date of the picture, the second is the last modified date (today). 

Ideally, Digikam should sort by date using 2017-02-17, but uses 2018-09-2018 instead.

This is how it looks in exiv2:

user@Equip-4:~/Desktop/proves$ exiv2 IMG-20170217-WA0011.jpg -pa
Exif.Image.XResolution                       Rational    1  72
Exif.Image.YResolution                       Rational    1  72
Exif.Image.ResolutionUnit                    Short       1  inch
Exif.Image.Software                          Ascii       7  Picasa
Exif.Image.DateTime                          Ascii      20  2018:09:13 23:49:42
Exif.Image.YCbCrPositioning                  Short       1  Centered
Exif.Image.ExifTag                           Long        1  142
Exif.Photo.ExifVersion                       Undefined   4  2.31
Exif.Photo.DateTimeOriginal                  Ascii      20  2017:02:17 00:11:00
Exif.Photo.DateTimeDigitized                 Ascii      20  2017:02:17 00:11:00
Exif.Photo.ComponentsConfiguration           Undefined   4  YCbCr
Exif.Photo.FlashpixVersion                   Undefined   4  1.00
Exif.Photo.ColorSpace                        Short       1  Uncalibrated
Exif.Photo.ImageUniqueID                     Ascii      33  3b7453056144d41f3492b4d48afcc271
Iptc.Envelope.ModelVersion                   Short       1  4
Iptc.Envelope.CharacterSet                   String      3
Iptc.Application2.RecordVersion              Short       1  4
Iptc.Application2.Keywords                   String     12  Digikam test
Xmp.xmp.ModifyDate                           XmpText    25  2018-09-13T23:49:42+02:00
Xmp.dc.subject                               XmpBag      1  Digikam test


And this is the same pictures in exiftool:

user@Equip-4:~/Desktop/proves$ exiftool IMG-20170217-WA0011.jpg
ExifTool Version Number         : 10.80
File Name                       : IMG-20170217-WA0011.jpg
Directory                       : .
File Size                       : 260 kB
File Modification Date/Time     : 2018:09:13 23:49:42+02:00
File Access Date/Time           : 2018:09:13 23:49:42+02:00
File Inode Change Date/Time     : 2018:09:13 23:49:42+02:00
File Permissions                : rwxrwxrwx
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
Exif Byte Order                 : Big-endian (Motorola, MM)
X Resolution                    : 72
Y Resolution                    : 72
Resolution Unit                 : inches
Software                        : Picasa
Modify Date                     : 2018:09:13 23:49:42
Y Cb Cr Positioning             : Centered
Exif Version                    : 0231
Date/Time Original              : 2017:02:17 00:11:00
Create Date                     : 2017:02:17 00:11:00
Components Configuration        : Y, Cb, Cr, -
Flashpix Version                : 0100
Color Space                     : Uncalibrated
Image Unique ID                 : 3b7453056144d41f3492b4d48afcc271
XMP Toolkit                     : XMP Core 5.1.2
Subject                         : Digikam test
Current IPTC Digest             : c60939fa3093030de3dd3bcbfbec1709
Envelope Record Version         : 4
Coded Character Set             : UTF8
Application Record Version      : 4
Keywords                        : Digikam test
IPTC Digest                     : c60939fa3093030de3dd3bcbfbec1709
Image Width                     : 1600
Image Height                    : 900
Encoding Process                : Progressive DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:4:4 (1 1)
Image Size                      : 1600x900
Megapixels                      : 1.4

This test image would have digiKam-0.5.9 correctly sorted with the date 2017-02-17, because Exif.Photo.DateTimeDigitized contains the same date. Look in comment 20, Exif.Photo.DateTimeDigitized does not exist there. This image must show 2017-02-17 as the creation date.

Maik

You are right. I think I might have added the digitized time at some point, in order to find out what Date fields had priority. The tests I did previously do not have the Digitized date field.

For the test I have removed from your image Exif.Photo.DateTimeDigitized. Without the patch from comment 23, the date of 2018 will be recognized as the creation date. With the patch recognizes the correct date of 2017, so it is fixed for this example.

Maik

Thank you very much! I'll try it as soon as the next digikam snapshot is released.

I will work while this week end to update all the bundles with current git/master code.

Gilles Caulier

I tried today's snapshot, and now it seems to work just fine. Digikam gives now more priority to the "Original date and time" than the "Creation date and time".

Thank you all!