Bug 246401

Summary: Improve date based sort of camera interface: use more time precision from Exif when shots are taken with less than 1s
Product: [Applications] digikam Reporter: JD Rogers <rogersjd>
Component: Import-SortAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: caulier.gilles, marcel.wiesweg, rogersjd, vivo75+kde
Priority: NOR    
Version: 1.2.0   
Target Milestone: ---   
Platform: Ubuntu   
OS: Linux   
Latest Commit: Version Fixed In: 2.1.0
Sentry Crash Report:

Description JD Rogers 2010-08-01 05:07:56 UTC
Version:           1.2.0 (using KDE 4.4.2) 
OS:                Linux

Man, I love digikam. It just keeps getting better. The advanced rename feature is better than ever, and now I use [date:yyyyMMddThhmmss]_##
Which is great except when I have more than one shot in a second. The order gets reversed in this case. The result is periodic shots out of order which made for some really nifty evidence of breaking the laws of physics until I realized what was going on.  :-)

Simplified example: I have a sequence of pics which are taken at time/date t0, t1, t1, t2. The rename then gives me t0_01, t1_03, t1_02, t2_03 where the to images taken at the same second are reversed relative to the rest of the sequence. 

Thanks,
JDR

Reproducible: Always

Steps to Reproduce:
1. Take a fast sequence of photos with more than 1 shot per second.
2. Rename on download from camera using [date:yyyyMMddThhmmss]_##

Actual Results:  
Inverted order of the _## part for pics taken in the same second.

Expected Results:  
The sequence number increments in the same order for pics taken at different times and within date/time resolution. In other words, the images file number stored on the camera
Comment 1 JD Rogers 2010-08-01 17:03:36 UTC
Woops, too trigger happy with the submit. 

I just checked how the images are listed on the camera in the import window and it seems that the camera filenames (IMG_4707.JPG, IMG_4708.JPG, etc) are in the correct order. If I take a series of phots as fast as possible while slowly turning, it is easy to see the order that *should* be in the sequence, and the camera filenames are in the correct order. 

When I view the thumbnails in the import window, the sequence is reversed for groups of 2-3 images at a time. I believe this is the problem, since selected the images then assigns sequence number based on the order of selection?

Hope that makes sense. Please let me know if you need any more information.

I don't think this is related to 230553 where {unique} increments on select/deselect, but perhaps it is.

Thanks,
JDR
Comment 2 JD Rogers 2010-12-18 18:55:19 UTC
This inverted order is still a problem with digikam 1.6 and when viewing the files on the camera in the import window, I can select "show last photo first" or not and images captured within the same second still show in reversed order relative to all other photos in both cases. 

For example, if images 20 and 21 were captured within the same second but the rest were seperated by at least one second, "show last photo first" results in: IMG_7923.JPG  IMG_7922.JPG  IMG_7920.JPG  IMG_7921.JPG
while unchecking the reverse order shows:
IMG_7921.JPG  IMG_7920.JPG  IMG_7922.JPG  IMG_7923.JPG

Importing the photos with advanced rename then applies this incorrect ordering when assigning numbers to the filenames.

Now, I was curious if the reason for digikam behavior is that the camera is writing to flash card in reverse order for fast sequences, and digikam is just using the directory order, however, mounting the card with a card reader and using "ls -U" shows
IMG_7920.JPG  IMG_7921.JPG  IMG_7922.JPG  IMG_7923.JPG
i.e. correct order. 

I am finally getting around this by using [date:ssyyyyMMddThhmmss]_[file] which appends the camera numbering and keeps the order correct. However, it would be really nice to fix this so that fast sequences of photoscan be kept in the correct order without this workaround.
Comment 3 Andi Clemens 2010-12-18 19:41:49 UTC
If the images are not in the correct order in the icon view, then it is not a problem with the renaming but with the camera import tool.
Comment 4 JD Rogers 2010-12-18 22:45:33 UTC
True.. now that I have explored this more, it should be listed as a bug in import rather than advanced rename. Is it possible to edit the component field of the bug? I just tried with no luck.
Comment 5 caulier.gilles 2011-08-09 09:38:36 UTC
JD,

Can you provide some image sample (at least 6) taken with a brief sort of time with your camera, to try to solve this issue.

Note : I hope that your camera store msec information in Exif about time stamp. There are a specific field in Exif for that, else time resolution is limited to 1s...

Gilles Caulier
Comment 6 caulier.gilles 2011-08-09 13:03:57 UTC
JD,

I taken some items quicly with my camera and i fixed the problem :

http://www.flickr.com/photos/digikam/6025713094/sizes/o/in/photostream/

With this screenshot, digiKam 2.1.0 is able to sort items from camera interface using milliseconds time-stamp stored in Exif metadata.

But this require to use Exif Date/Time stamp from metadata as you can see in camera interface settings, not File System Date/Time stamp which don't store milli-second information. This increase connection time to camera, to list all items in icon-view.

Gilles Caulier
Comment 7 caulier.gilles 2011-08-09 13:10:08 UTC
Marcel,

I don't know how date-time stamp is stored in DB and if when can store milliseconds info as well, but this problem is only fixed in camera gui. In Collection, digiKam is not able to sort item taken quickly.

I think we must take a care about in the future.

Gilles Caulier
Comment 8 caulier.gilles 2011-08-09 13:11:55 UTC
At least, the solution to extract millisecond info fropm Exif is implemented in DMetadata. It's easy to use to it with database if necessary to register this info.

Francesco, i CC you too for info...

Gilles Caulier
Comment 9 caulier.gilles 2011-08-09 13:12:48 UTC
Git commit abc7ddc41ffcdf89f7607e96112277aba717de82 by Gilles Caulier.
Committed on 09/08/2011 at 15:04.
Pushed by cgilles into branch 'master'.

Camera interface is now able to sort item taken quicly (less than 1 second) using millisecond time stamp from Exif metadata.
BUGS: 246401

M  +35   -0    libs/dmetadata/dmetadata.cpp
M  +9    -0    libs/dmetadata/dmetadata.h
M  +2    -0    utilities/cameragui/devices/dkcamera.cpp

http://commits.kde.org/digikam/abc7ddc41ffcdf89f7607e96112277aba717de82
Comment 10 Marcel Wiesweg 2011-08-09 17:24:10 UTC
We use the SQL data type DATETIME. For SQLite, this is simply a string . This string is created using QDateTime::toString/fromString with Qt::ISODate: "2011-08-09T19:14:09.123"
QTime has a field for microseconds. QDateTime::toString will never append the microseconds, but fromString correctly parses them. This means with a custom toString(), including the microseconds would be fully backwards compatible.

Francesco, any considerations for MySQL and the DATETIME type? The code generating a QString in AlbumDB is the same.
Comment 11 caulier.gilles 2011-08-09 17:25:53 UTC
Marcel,

One words... Excellent... (:=)))

Gilles
Comment 12 JD Rogers 2011-08-09 21:27:23 UTC
Hi Gilles, 
I'm too slow. You've been busy on bugs of mine lately and apparently with great success! Sorry I didn't get back to you sooner, but thanks for the fix. I'm not even sure my camera reports ms in the exif, but I can always work around it until I get a newer camera. :-)
Thanks,
JD
Comment 13 Francesco Riosa 2011-08-10 23:46:54 UTC
(In reply to comment #10)
> We use the SQL data type DATETIME. For SQLite, this is simply a string . This
> string is created using QDateTime::toString/fromString with Qt::ISODate:
> "2011-08-09T19:14:09.123"
> QTime has a field for microseconds. QDateTime::toString will never append the
> microseconds, but fromString correctly parses them. This means with a custom
> toString(), including the microseconds would be fully backwards compatible.
> 
> Francesco, any considerations for MySQL and the DATETIME type? The code
> generating a QString in AlbumDB is the same.

no support for milliseconds or better in mysql, need to fake it with an additional INT column :(