Bug 177242

To summarize: wish is to auto-create date based albums in import dialog / camera dialog based on creation date instead of modification date.

We currently use mtime because the gphoto2 api only allows access to this property.
For USB cameras, which can also include sticks, and the import dialog though, it seems sensible to use the creation date as it is easily available.
Opinions?

Marcel,

>For USB cameras, which can also include sticks, and the import dialog though, it >seems sensible to use the creation date as it is easily available. Opinions? 

I'm fully agree. Also, look into digikam/cameragui B.K.O component, i think there is 1 or 2 entries relevant of this problem...

Gilles

Generally it is better to use creation date: that is what user really expects.
Personally I am for it BUT there will be big inconsequence with devices accessed by gphoto2 interface. IMO in future it should be "fixed" by some hack around gphoto2, eg.: download photos to some temporary folder and later reimport them and use creation date for sub-folders.

Marcel,

files which can be relevant:

http://bugs.kde.org/show_bug.cgi?id=151275
http://bugs.kde.org/show_bug.cgi?id=146764

Gilles

SVN commit 897995 by cgilles:

digiKam from KDE3 branch : Camera GUI improvements: USB mass Storage camera: items date management:
- Use metadata from THM files if available to get item informations.
- Video files date are extracted from THM if possible. In this case video files are sorted by date properlly.
- If THM files are not available, we trying to extract date from orginal file metadata.
- If metadata are not available, we use file creation time stamp instead file Modification time stamp.


 M  +103 -60   umscamera.cpp
 M  +8 -8      umscamera.h


--- branches/extragear/kde3/graphics/digikam/utilities/cameragui/umscamera.cpp #897994:897995
@@ -7,7 +7,7 @@
 * Description : USB Mass Storage camera interface
 *
 * Copyright (C) 2004-2005 by Renchi Raju <renchi@pooh.tam.uiuc.edu>
- * Copyright (C) 2005-2007 by Gilles Caulier <caulier dot gilles at gmail dot com>
+ * Copyright (C) 2005-2008 by Gilles Caulier <caulier dot gilles at gmail dot com>
 *
 * This program is free software; you can redistribute it
 * and/or modify it under the terms of the GNU General
@@ -101,8 +101,10 @@
    if (!list)
        return false;

-    QFileInfoListIterator it( *list );
+    QFileInfoListIterator it(*list);
    QFileInfo *fi;
+    QFileInfo thmlo, thmup;
+    DMetadata meta;

    while ((fi = it.current()) != 0 && !m_cancel)
    {
@@ -112,38 +114,68 @@

        if (!mime.isEmpty())
        {
+            QSize      dims;
+            QDateTime  dt;
            GPItemInfo info;
-            QSize dims(-1, -1);
+            thmlo.setFile(folder + QString("/") + fi->baseName() + QString(".thm"));
+            thmup.setFile(folder + QString("/") + fi->baseName() + QString(".THM"));

-            if (getImageDimensions)
+            if (thmlo.exists())
            {
-                if (mime == QString("image/x-raw"))
+                // Try thumbnail sidecar files with lowercase extension.
+                meta.load(thmlo.filePath());
+                dt   = meta.getImageDateTime();
+                dims = meta.getImageDimensions();
+            }
+            else if (thmup.exists())
+            {
+                // Try thumbnail sidecar files with uppercase extension.
+                meta.load(thmup.filePath());
+                dt   = meta.getImageDateTime();
+                dims = meta.getImageDimensions();
+            }
+            else if (mime == QString("image/x-raw"))
+            {
+                // If no thumbnail sidecar file available , try to load image metadata with Raw files.
+                meta.load(fi->filePath());
+                dt   = meta.getImageDateTime();
+                dims = meta.getImageDimensions();
+            }
+            else
+            {
+                meta.load(fi->filePath());
+                dt   = meta.getImageDateTime();
+                dims = meta.getImageDimensions();
+
+                if (dims.isNull())
                {
-                    DMetadata metaData(fi->filePath());
-                    dims = metaData.getImageDimensions();
-                }
-                else
-                {
-                    KFileMetaInfo meta(fi->filePath());
-                    if (meta.isValid())
+                    // In all others case, try KFileMetaInfo.
+                    KFileMetaInfo kmeta(fi->filePath());
+                    if (kmeta.isValid())
                    {
-                        if (meta.containsGroup("Jpeg EXIF Data"))
-                            dims = meta.group("Jpeg EXIF Data").item("Dimensions").value().toSize();
-                        else if (meta.containsGroup("General"))
-                            dims = meta.group("General").item("Dimensions").value().toSize();
-                        else if (meta.containsGroup("Technical"))
-                            dims = meta.group("Technical").item("Dimensions").value().toSize();
+                        if (kmeta.containsGroup("Jpeg EXIF Data"))
+                            dims = kmeta.group("Jpeg EXIF Data").item("Dimensions").value().toSize();
+                        else if (kmeta.containsGroup("General"))
+                            dims = kmeta.group("General").item("Dimensions").value().toSize();
+                        else if (kmeta.containsGroup("Technical"))
+                            dims = kmeta.group("Technical").item("Dimensions").value().toSize();
                    }
                }
            }

+            if (dt.isNull())
+            {
+                // If date is not found in metadata, use file time stamp
+                dt = fi->created();
+            }
+
            info.name             = fi->fileName();
            info.folder           = !folder.endsWith("/") ? folder + QString("/") : folder;
            info.mime             = mime;
-            info.mtime            = fi->lastModified().toTime_t();
+            info.mtime            = dt.toTime_t();
            info.size             = fi->size();
-            info.width            = dims.width();
-            info.height           = dims.height();
+            info.width            = getImageDimensions ? dims.width()  : -1;
+            info.height           = getImageDimensions ? dims.height() : -1;
            info.downloaded       = GPItemInfo::DownloadUnknow;
            info.readPermissions  = fi->isReadable();
            info.writePermissions = fi->isWritable();
@@ -173,10 +205,9 @@
        return true;

    // THM files: try to get thumbnail from '.thm' files if we didn't manage to get
-    // thumbnail from Exif. Any cameras provides *.thm files like JPEG files with RAW files.
-    // Using this way is always speed up than ultimate loading using DImg.
-    // Nota: the thumbnail extracted with this method can be in poor quality.
-    // 2006/27/01 - Gilles - Tested with my Minolta Dynax 5D USM camera.
+    // thumbnail from Exif. Any cameras provides *.thm files like JPEG files with RAW/video files.
+    // Using this way speed up thumbnailization and limit data transfered between camera and computer.
+    // NOTE: the thumbnail extracted with this method can provide a poor quality preview.

    QFileInfo fi(folder + QString("/") + itemName);

@@ -191,7 +222,6 @@
           return true;
    }

-
    // Finaly, we trying to get thumbnail using DImg API (slow).

    DImg dimgThumb(QFile::encodeName(folder + QString("/") + itemName));
@@ -215,8 +245,7 @@

 bool UMSCamera::downloadItem(const QString& folder, const QString& itemName, const QString& saveFile)
 {
-    m_cancel = false;
-
+    m_cancel     = false;
    QString src  = folder + QString("/") + itemName;
    QString dest = saveFile;

@@ -226,7 +255,7 @@
    if ( !sFile.open(IO_ReadOnly) )
    {
        DWarning() << "Failed to open source file for reading: "
-                    << src << endl;
+                   << src << endl;
        return false;
    }

@@ -271,7 +300,7 @@

 bool UMSCamera::setLockItem(const QString& folder, const QString& itemName, bool lock)
 {
-    QString src  = folder + QString("/") + itemName;
+    QString src = folder + QString("/") + itemName;

    if (lock)
    {
@@ -312,10 +341,9 @@
 }

 bool UMSCamera::uploadItem(const QString& folder, const QString& itemName, const QString& localFile,
-                           GPItemInfo& itemInfo, bool getImageDimensions)
+                           GPItemInfo& info, bool getImageDimensions)
 {
-    m_cancel = false;
-
+    m_cancel     = false;
    QString dest = folder + QString("/") + itemName;
    QString src  = localFile;

@@ -367,45 +395,60 @@

    // Get new camera item information.

+    DMetadata meta;
    QFileInfo fi(dest);
-    QString mime = mimeType(fi.extension(false).lower());
+    QString   mime = mimeType(fi.extension(false).lower());

    if (!mime.isEmpty())
    {
-        QSize dims(-1, -1);
+        QSize     dims;
+        QDateTime dt;

-        if (getImageDimensions)
+        if (mime == QString("image/x-raw"))
        {
-            if (mime == QString("image/x-raw"))
+            // Try to load image metadata with Raw files.
+            meta.load(fi.filePath());
+            dt   = meta.getImageDateTime();
+            dims = meta.getImageDimensions();
+        }
+        else
+        {
+            meta.load(fi.filePath());
+            dt   = meta.getImageDateTime();
+            dims = meta.getImageDimensions();
+
+            if (dims.isNull())
            {
-                DMetadata metaData(fi.filePath());
-                dims = metaData.getImageDimensions();
-            }
-            else
-            {
-                KFileMetaInfo meta(fi.filePath());
-                if (meta.isValid())
+                // In all others case, try KFileMetaInfo.
+                KFileMetaInfo kmeta(fi.filePath());
+                if (kmeta.isValid())
                {
-                    if (meta.containsGroup("Jpeg EXIF Data"))
-                        dims = meta.group("Jpeg EXIF Data").item("Dimensions").value().toSize();
-                    else if (meta.containsGroup("General"))
-                        dims = meta.group("General").item("Dimensions").value().toSize();
-                    else if (meta.containsGroup("Technical"))
-                        dims = meta.group("Technical").item("Dimensions").value().toSize();
+                    if (kmeta.containsGroup("Jpeg EXIF Data"))
+                        dims = kmeta.group("Jpeg EXIF Data").item("Dimensions").value().toSize();
+                    else if (kmeta.containsGroup("General"))
+                        dims = kmeta.group("General").item("Dimensions").value().toSize();
+                    else if (kmeta.containsGroup("Technical"))
+                        dims = kmeta.group("Technical").item("Dimensions").value().toSize();
                }
            }
        }

-        itemInfo.name             = fi.fileName();
-        itemInfo.folder           = !folder.endsWith("/") ? folder + QString("/") : folder;
-        itemInfo.mime             = mime;
-        itemInfo.mtime            = fi.lastModified().toTime_t();
-        itemInfo.size             = fi.size();
-        itemInfo.width            = dims.width();
-        itemInfo.height           = dims.height();
-        itemInfo.downloaded       = GPItemInfo::DownloadUnknow;
-        itemInfo.readPermissions  = fi.isReadable();
-        itemInfo.writePermissions = fi.isWritable();
+        if (dt.isNull())
+        {
+            // If date is not found in metadata, use file time stamp
+            dt = fi.created();
+        }
+
+        info.name             = fi.fileName();
+        info.folder           = !folder.endsWith("/") ? folder + QString("/") : folder;
+        info.mime             = mime;
+        info.mtime            = dt.toTime_t();
+        info.size             = fi.size();
+        info.width            = getImageDimensions ? dims.width()  : -1;
+        info.height           = getImageDimensions ? dims.height() : -1;
+        info.downloaded       = GPItemInfo::DownloadUnknow;
+        info.readPermissions  = fi.isReadable();
+        info.writePermissions = fi.isWritable();
    }

    return true;
--- branches/extragear/kde3/graphics/digikam/utilities/cameragui/umscamera.h #897994:897995
@@ -5,21 +5,21 @@
 *
 * Date        : 2004-12-21
 * Description : USB Mass Storage camera interface
- *
+ *
 * Copyright (C) 2004-2005 by Renchi Raju <renchi@pooh.tam.uiuc.edu>
- * Copyright (C) 2005-2007 by Gilles Caulier <caulier dot gilles at gmail dot com>
+ * Copyright (C) 2005-2008 by Gilles Caulier <caulier dot gilles at gmail dot com>
 *
 * This program is free software; you can redistribute it
 * and/or modify it under the terms of the GNU General
 * Public License as published by the Free Software Foundation;
 * either version 2, or (at your option)
 * any later version.
- *
+ *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
- *
+ *
 * ============================================================ */

 #ifndef UMSCAMERA_H
@@ -49,7 +49,7 @@
    void cancel();

    void getAllFolders(const QString& folder, QStringList& subFolderList);
-    bool getItemsInfoList(const QString& folder, GPItemInfoList& infoList, bool getImageDimensions = true);
+    bool getItemsInfoList(const QString& folder, GPItemInfoList& infoList, bool getImageDimensions=true);
    bool getThumbnail(const QString& folder, const QString& itemName, QImage& thumbnail);
    bool getExif(const QString& folder, const QString& itemName, char **edata, int& esize);

@@ -57,8 +57,8 @@

    bool downloadItem(const QString& folder, const QString& itemName, const QString& saveFile);
    bool deleteItem(const QString& folder, const QString& itemName);
-    bool uploadItem(const QString& folder, const QString& itemName, const QString& localFile,
-                    GPItemInfo& itemInfo, bool getImageDimensions=true);
+    bool uploadItem(const QString& folder, const QString& itemName, const QString& localFile,
+                    GPItemInfo& info, bool getImageDimensions=true);

    bool cameraSummary(QString& summary);
    bool cameraManual(QString& manual);
@@ -68,7 +68,7 @@

    void listFolders(const QString& folder, QStringList& subFolderList);

-private :
+private:

    bool m_cancel;
 };

(In reply to comment #5)
> SVN commit 897995 by cgilles:
> 
> digiKam from KDE3 branch : Camera GUI improvements: USB mass Storage camera:
> items date management:
> - Use metadata from THM files if available to get item informations.
> - Video files date are extracted from THM if possible. In this case video files
> are sorted by date properlly.
> - If THM files are not available, we trying to extract date from orginal file
> metadata.
> - If metadata are not available, we use file creation time stamp instead file
> Modification time stamp.
> 
> 

Yes, this is exactly about wish is. But only for usb mass storage devices?

>Yes, this is exactly about wish is. But only for usb mass storage devices? 

yes, only USB Mass storage, because Gphoto2 do not provide an easy way to do the same : some drivers do not provide a way to extract Exif info and only give Modification time stamp of file system.

Gilles Caulier

Jaroslav,

It will be nice if you can test my patch using current digiKam implementation from svn (0.9.5-beta3)

Note : I will also patch KDE4 version...

Gilles Caulier

hmm,

you could try:

CameraFileInfo info;
gp_camera_file_get_info( camera, folder, filename ,&info, context);
if (info.file.fields & GP_FILE_INFO_MTIME) ... use info.file.mtime   

Marcus,

Already done of course, but it's not enough (:=)

http://lxr.kde.org/source/extragear/graphics/digikam/utilities/cameragui/gpcamera.cpp#531

Gilles

(In reply to comment #7)
> >Yes, this is exactly about wish is. But only for usb mass storage devices? 
> 
> yes, only USB Mass storage, because Gphoto2 do not provide an easy way to do
> the same : some drivers do not provide a way to extract Exif info and only give
> Modification time stamp of file system.
> 
> Gilles Caulier
> 

What about local disk filesystem? Look at my situation - old unsorted photos on my harddisk which I want to sort. They have exif data and ctime too (I think they have :D ). 

I think it's filesystem dependent, so there is no difference between USB mass storage and harddrive, only in filesystem.

(In reply to comment #8)
> >Jaroslav,
>
> It will be nice if you can test my patch using current digiKam implementation from svn (0.9.5-beta3)
>
>Note : I will also patch KDE4 version...
>
>Gilles Caulier 

I have a lot of work now, but if I find some free time, i will test it immediately.

SVN commit 898140 by cgilles:

backport commit #897995 from KDE3 branch
BUG: 177242
BUG: 151275
BUG: 146764
CCBUG: 136897


 M  +1 -2      cameracontroller.cpp  
 M  +1 -2      gpcamera.cpp  
 M  +60 -84    umscamera.cpp  
 M  +4 -4      umscamera.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=898140

One additional question. When it will be merged to general available versions of KDE, or digikam?

PS: Thank you.

>When it will be merged to general available versions of KDE, or digikam? 

Sorry ? I don't understand...

Gilles

(In reply to comment #14)
> >When it will be merged to general available versions of KDE, or digikam? 
> 
> Sorry ? I don't understand...
> 
> Gilles
> 

Sorry, my mistake - bad formulation. In this time I have information what I want, so excuse me please for my pointless coments :(

Created attachment 29535 [details]
Photo created in 2004 restored in 2008 using scp

Created attachment 29536 [details]
Photo created in 2004 restored in 2008 using scp

- If metadata are not available, we use file creation time stamp instead file
Modification time stamp.

I suspect this should use the lesser value of ctime or mtime and not just ctime only.

When I restore a photo from a remote archive using rsync or scp, it will have the ctime set to the current time (now: when the inode is set) ie 2008, but the mtime will be match the mtime on the original file system (ie 2004).

This is causing me issues with my imported files without metadata are always set to ctime, whilst mtime is generally more accurate for a backup file.  See attachment:

Propably the best will be to give user choice to make decision whitch timestamp to use (mtime/ctime).