Bug 107316

Summary: Option: Only download new photos from camera
Product: [Applications] digikam Reporter: Grzegorz Jaskiewicz <gj>
Component: Import-Gphoto2Assignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: wishlist CC: caulier.gilles, f.faber-pro, kde, marcus, maximedelorme, rivolaks
Priority: NOR    
Version: 0.7.3   
Target Milestone: ---   
Platform: unspecified   
OS: Linux   
Latest Commit: Version Fixed In:
Attachments: A sample camera XML config file from my computer (.kde/share/apps/digikam/cameras.xml)

Description Grzegorz Jaskiewicz 2005-06-13 08:59:55 UTC
Version:           0.7.3-beta2 (using KDE 3.4.89 (>= 20050508), compiled sources)
Compiler:          gcc version 3.4.4 20050314 (prerelease) (Debian 3.4.3-12)
OS:                Linux (i686) release 2.6.11.7

the dialog that allows me to download photos off camera is really neat, apart from thing. I have to 1) select all new, 2) download all new. Usually I do it (or actually always) to download new pictures from camera, so I think there should be an option in Download that would allow me to do it. 

Btw, great great great app :-)
Comment 1 Vitalijus 2006-01-01 20:51:49 UTC
i think in digikam database you could add something like files HASH, when you connect camera, program get all the files hash list in camera, then check who of it is in database, and in the next step displays it,
but could be checkbox "Show only new" or something like this
Comment 2 caulier.gilles 2006-07-19 15:33:07 UTC
*** Bug 124414 has been marked as a duplicate of this bug. ***
Comment 3 caulier.gilles 2006-08-24 15:59:09 UTC
*** Bug 131959 has been marked as a duplicate of this bug. ***
Comment 4 caulier.gilles 2006-08-24 18:51:15 UTC
Vitalijus,

Your solution require a change in database structure (a new table dedicaced to store camera log or something like that). I won't changes anything in database to 0.9.0 release.

Secondo, an hash key for each picture from camera must be computed using the properties provided by all camera drivers : filename, file size, file date, and camera folder. We cannot use a md5 sum with gphoto2 camera driver (and PTP) because this will require to download the pictures to compute the sum!

This is have no sence because we want to identify the un-downloaded files before to process loading on computer. Of course, md5 sum can be used with USM camera, but it cannot be the universal solution...

Gilles Caulier
Comment 5 caulier.gilles 2006-08-24 18:56:29 UTC
This is my proposal for 0.9.0 release :

Each camera settings (port, path, internal name, and title) is store in a little XML file in the ~/.kde/ settings area. We can add a new XML property from each camera to store the last acces performed by the camera gui to download pictures.

Like this we can identify all pictures with a future date than the last date used to download pictures.

Note: if the camera gui is used with to download pictures, the date in XML file isn't updated.

What do you think about ?

Gilles Caulier
Comment 6 caulier.gilles 2006-08-24 19:01:00 UTC
Important : I have forget to said than the libgphoto2 API provide a flag to identify the download status of a picture, but this flag is _ONLY_ supported by Canon gphoto2 drivers, not ghoto2 PTP driver and digiKam UMS driver. (Marcus fix me if i'm wrong (:=)))

In fact, this libgphoto2 tag cannot be used here...

Gilles Caulier
Comment 7 caulier.gilles 2006-08-24 19:04:50 UTC
In comment #5, i would said : "the last camera access _date_ of course.

Gilles Caulier
Comment 8 caulier.gilles 2006-08-24 20:45:17 UTC
In comment #5, i would said also : "Note: if the camera gui is _not_ used to download pictures"

Gilles Caulier 
Comment 9 Achim Bohnet 2006-08-24 20:49:13 UTC
Mhmm, die xml file reminded me: IMHO it's really sad that kio-camera
and kipi camera as well as digikam don't store camera info in a common
file.   Maybe using a kde config file is the better solution for this
task, because parameter unknown to one application are not deleted.

Otherwise sounds fine to me.  We just have to be careful that the download
dialog does not get overcrowed ;)

Achim
Comment 10 caulier.gilles 2006-08-24 20:50:33 UTC
Achim,

The XML camera settings file is used only by digiKam !

Gilles
Comment 11 Achim Bohnet 2006-08-24 20:54:53 UTC
Gilles,

yes I know.  And that it's only used by digikam _is_ a problem.
I value the integration in KDE core very high.  We need to
spread this to the extragear apps too!  I know it's not easy
to find an consensus.  Therefore a kde config file is imho
better so other can implement features as they need.

Achim
Comment 12 Marcus Meissner 2006-08-24 21:19:05 UTC
regarding #c6 ... correct. only canon cameras support it.

regarding the XML question ... well, what settings? A USB camera can just be
autodetect, there is not much more to store, or what else are you storing?
Comment 13 caulier.gilles 2006-08-24 21:28:13 UTC
Created attachment 17491 [details]
A sample camera XML config file from my computer (.kde/share/apps/digikam/cameras.xml)
Comment 14 caulier.gilles 2006-08-24 21:32:21 UTC
Like you can see, there is one line by camera to describe the setting like this:

 <item port="usb:" title="Olympus C-2100UZ" model="Olympus C-2100UZ" path="/" />

My solution is to implemente something like this:

 <item port="usb:" title="Olympus C-2100UZ" model="Olympus C-2100UZ" path="/" path="/" lastaccess="20060824T213412"/>

Gilles
Comment 15 caulier.gilles 2006-08-25 14:28:18 UTC
SVN commit 577049 by cgilles:

digikam from trunk : Camera Gui improvement : first approach to itentify the NEW pictures taken by a camera.

digiKam now store in the camera XML config file the "last camera used date". Each camera are tagged with a date value between camera gui sessions if some pictures have been donwloaded to computer. If no pictures are donloaded with camera gui, the date is unchanged.

In fact, all picture dates in the camera witch are most recent than the "last camera used date" will be displayed with a little yellow start to indentify the most recent pictures taken with the camera.

To use properlly this little feature, you need to set the right date in your camera !

Of course, this way cannot be used to identify the pictures "already downloaded" from camera to computer. For that, we need to have a full camera activity log somewhere (perhaps using the digiKam database). 

I'm waiting your comments about to implement camera log feature in digiKam.

CCBUGS: 107316

 M  +17 -14    digikam/cameradragobject.cpp  
 M  +10 -9     digikam/cameradragobject.h  
 M  +27 -8     digikam/cameralist.cpp  
 M  +2 -0      digikam/cameralist.h  
 M  +40 -25    digikam/cameratype.cpp  
 M  +10 -7     digikam/cameratype.h  
 M  +5 -5      digikam/digikamapp.cpp  
 M  +1 -1      utilities/cameragui/cameraiconitem.cpp  
 M  +16 -1     utilities/cameragui/cameraiconview.cpp  
 M  +1 -0      utilities/cameragui/cameraiconview.h  
 M  +29 -6     utilities/cameragui/cameraui.cpp  
 M  +2 -1      utilities/cameragui/cameraui.h  
 M  +1 -0      utilities/cameragui/gpiteminfo.h  
 M  +19 -6     utilities/setup/setupcamera.cpp  
Comment 16 caulier.gilles 2006-08-25 14:49:19 UTC
*** Bug 88253 has been marked as a duplicate of this bug. ***
Comment 17 FACORAT Fabrice 2006-08-25 15:01:16 UTC
IMHO you should store info like USB ID/Vendor Id, or a value unique to each model because someone can have severals "Olympus C-2100UZ" ( or someone else with also a  "Olympus C-2100UZ" may come on your computer ) and your system will be confused. We'd better no report too false informations.
I don't have a USB camera to test, but IMHO you can find this info
Comment 18 Mikolaj Machowski 2006-08-28 12:11:36 UTC
> To use properlly this little feature, you need to set the right date in
> your camera !


Is it possible to check if camera time is in sync with computer clock
and warn user if this is not the case. That would be good improvement.
Anyway - great solution for problem.
Comment 19 caulier.gilles 2006-08-28 20:52:07 UTC
#18 : I will take a look if gphoto2 API can provide a way to set camera date... but how to do with UMS camera ? 

Gilles
Comment 20 Mikolaj Machowski 2006-08-29 02:28:38 UTC
I'm not sure if Digikam should try to *set* camera date. Such intrusion
could be seen as aggressive and user can has his reasons for "bad" date
on camera (BTW - camera time can be good, what if computer date is
bad?). I am asking only(?) for checking if dates are in sync. If yes,
everything is OK and marking can proceed. If not, Digikam should notify
user about that fact. Of course if it would be possible to sync date it
would be great. For example when returning from other time zone it could
be great way to return to home settings. But warning would be enough.
Comment 21 Benoit DUMONT 2006-08-29 21:59:43 UTC
This is working for me but is there a way to make this work also for the dynamic entries in the media inside "browse entry".
For example, when I plug my camera, KDE is asking me if I want to download my picture using digikam. If I do it directly, the new files will not be displayed as the corresponding camera has no entry in the xml file.
If I close the download window and open it again using the static camera entry that I created, it works fine.
Comment 22 caulier.gilles 2006-08-29 22:09:49 UTC
#21 > yes, dynamic camera detection do not yet support this feature...

Gilles
Comment 23 caulier.gilles 2006-09-07 13:32:42 UTC
SVN commit 581730 by cgilles:

digikam from trunk : do not show the "unknow downloaded" icon tag on camera gui item until a right solution to identify the already downloaded images is implemented

CCBUGS: 107316 

 M  +7 -1      cameraiconitem.cpp  


--- trunk/extragear/graphics/digikam/utilities/cameragui/cameraiconitem.cpp #581729:581730
@@ -294,14 +294,20 @@
             downloaded = SmallIcon( "button_cancel" );
             break;
         }
+        /* TODO: see B.K.O #107316 : disable temporally the unknow download status until     
+                 a new method to identify the already downloaded pictures from camera is 
+                 implemented.
+  
         case GPItemInfo::DownloadUnknow:
         {
             downloaded = d->pixmapUnknowPicture;
             break;
         }
+        */
     }
 
-    p.drawPixmap(rect().width() - downloaded.width() - 5, 5, downloaded);
+    if (!downloaded.isNull())
+        p.drawPixmap(rect().width() - downloaded.width() - 5, 5, downloaded);
 
     // If camera item is locked (read only), draw a "Lock" icon.
     if (d->itemInfo->writePermissions == 0) 
Comment 24 Marcus Meissner 2006-10-13 18:43:17 UTC
i added code to handle "New" images for CANON cameras in the PTP2 driver of libgphoto2 too.

WIll be in libgphoto2 2.3.0 release (not released yet).
Comment 25 Mikolaj Machowski 2007-08-22 12:05:23 UTC
 I think this wish was implemented. Maybe sometimes isn't working 100% but this belongs to realm of bugs. As a wish this is IMO fulfilled. The only remaining issue I see there is no "magic" button/menu option: download new photos. You have to make two steps a) select new photos b) download selected photos. Awkward.

ps. For me it is working 100% (Fujifilm FinePix S6500fd and Olympus C-350).

Any voices against closing?
Comment 26 Arnd Baecker 2007-08-22 12:12:18 UTC
I am not using this feature, but the "magic" button sounds like a good
idea and possibly should be implemented before closing this bug, 
because otherwise it will be forgotten.
Comment 27 caulier.gilles 2007-08-22 12:27:07 UTC
Mik,

I'm against to close this bug. During my holidays, i have found a very advanced method to implement fully this feature (witch will work in all cases, especially if photograph has more than one CF card for one camera). I will post details later in this room...

Gilles
Comment 28 Mikolaj Machowski 2007-08-22 12:48:06 UTC
Gilles, you were supposed to relax yourself :)

----------------------------------------------------
CLUBNETIC SUMMER PARTY 2007
House, club, electro. Najlepsza kompilacja na letnie imprezy!
http://klik.wp.pl/?adr=http%3A%2F%2Fadv.reklama.wp.pl%2Fas%2Fclubnetic.html&sid=1266
Comment 29 Christian Mayer 2007-10-23 12:53:10 UTC
I've allways missed such an feature (actually it allways annoyes me when I'm downloading new pictures...)

For me it's important that this feature handles my CF and SD cards in my USB card reader as I'm using that method for my two cameras (I'm not directly pluging them in)

So basicly the solution for me would be to remember for each card individually the last loaded picture name or/date.

BTW: I only need two previews/thumbnails then: the last downloaded picture and the first "fresh" one, so I can check if it works correctly. All other thumbnails should only be shown on request (see also Bug #138299) to speed up the process of downloading the pictures.
Comment 30 Mikolaj Machowski 2008-04-03 18:32:37 UTC
Looks like this feature stops from working when constantly switching between two (or more) cameras. When I download images from camera1 make some photos and make new download new images are marked correctly but all knowledge goes to hell when using various cameras.
Comment 31 Adrian von Bidder 2008-05-27 16:25:05 UTC
... hmmmm ... for features like this, it would probably be worthwhile *always* identifying the storage medium and not the camera.

 * people may have multiple cards for their cameras
 * people may use the same card one time from the camera and another time from a cardreader.

... and: info about already downloaded pictures should certainly be moved into the digikam database if digikam moves towards being able to share digikam databases between multiple users.  A "family camera" might be plugged in into any of the accounts using the digikam database.

... and I'm not thinking about somebody using the same storage card with multiple cameras.  Many cameras will probably get hopelessly confused if anybody does that.
Comment 32 caulier.gilles 2008-11-02 01:44:34 UTC
I'm happy to said that all is done. It will be available with the next 0.10.0-beta6 release.

To be more precise, digiKam now use database to identify which files are already
downloaded from the camera.

Old version do not use Database but a flag from Gphoto2 drivers (Canon only).
It's a uncomplete way that do not support USB mass storage camera. Now, with
Database flags all camera type are supported.

The flag use a MD5 sum using media ID and some others image properties as file name, date and size to identify on the right way all items downloaded.

Gilles Caulier
Comment 33 Arnd Baecker 2008-11-06 11:57:01 UTC
Gilles, shouldn't this one be closed now?
Comment 34 caulier.gilles 2008-11-06 12:20:51 UTC
Arnd,

Yes, technically this file can be closed.

I will be more precise how uploaded images detection work:

- The database record for each item downloaded these infos:
 * a MD5 fingerprint (QString).
 * File name as QString.
 * File date as QDateTime.
 * File size in Kb as integer.

 See this method for details: 

http://lxr.kde.org/source/extragear/graphics/digikam/libs/database/downloadhistory.h#060
 
- With USB Mass Camera, i use Solid interface UUID of camera media to compute the MD5 fingerprint in Database:

http://lxr.kde.org/source/extragear/graphics/digikam/utilities/cameragui/umscamera.cpp#080

 Like this, we are able to detect if file is already downloaded if media is plugged in another device as a card reader instead the camera.

- With Gphoto2 camera, it's more complex, because there is no way to get camera media UUID. I have ask to Marcus from Gphoto2 team, and he confirm this point.
So i use another way, less powerfull: 

http://lxr.kde.org/source/extragear/graphics/digikam/utilities/cameragui/gpcamera.cpp#148

 I compute fingerprint using camera settings : mount path, port, and model. This way restrict uploaded images detection to camera device, not camera media. If you plug memory card in a card reader, digiKam will not reconize already downloaded items.

 I know that Gphoto2 provide a way to get camera media informations as it defined in gphoto2-filesystem.h:

/** 
 * \brief Storage information flags.
 *                                  
 * Bitmask to specify which entries of the filesystem
 * storage information is set.                       
 */                                                  
typedef enum {                                       
        GP_STORAGEINFO_BASE             = (1<<0),       /**< \brief The base directory. 
                                                         * Usually / if just 1 storage is attached.
                                                         */                                        
        GP_STORAGEINFO_LABEL            = (1<<1),       /**< \brief Label of the filesystem.       
                                                         * Could also be a DOS label.              
                                                         */                                        
        GP_STORAGEINFO_DESCRIPTION      = (1<<2),       /**< \brief More verbose description. */   
        GP_STORAGEINFO_ACCESS           = (1<<3),       /**< \brief Access permissions. */         
        GP_STORAGEINFO_STORAGETYPE      = (1<<4),       /**< \brief Hardware type. */              
        GP_STORAGEINFO_FILESYSTEMTYPE   = (1<<5),       /**< \brief Filesystem type. */            
        GP_STORAGEINFO_MAXCAPACITY      = (1<<6),       /**< \brief Maximum capacity in kbytes */  
        GP_STORAGEINFO_FREESPACEKBYTES  = (1<<7),       /**< \brief Free space in kbytes. */       
        GP_STORAGEINFO_FREESPACEIMAGES  = (1<<8)        /**< \brief Free space in images. */       
} CameraStorageInfoFields;                                                                         

/**
 * \brief Hardware storage types.
 *                               
 * Type of hardware this storage is on. The types and values
 * are the same as the PTP standard uses (PTP_ST_xxx).      
 */                                                         
typedef enum {                                              
        GP_STORAGEINFO_ST_UNKNOWN       = 0,    /**< \brief Unknown storage type. */
        GP_STORAGEINFO_ST_FIXED_ROM     = 1,    /**< \brief A fixed ROM storage. */ 
        GP_STORAGEINFO_ST_REMOVABLE_ROM = 2,    /**< \brief A removable ROM storage. */
        GP_STORAGEINFO_ST_FIXED_RAM     = 3,    /**< \brief A fixed RAM storage. (e.g. SDRAM) */
        GP_STORAGEINFO_ST_REMOVABLE_RAM = 4     /**< \brief A removable RAM storage. (any kind of cards etc) */
} CameraStorageType;                                                                                           

/**
 * \brief Storage access modes.
 *                             
 * The modes we can access the storage with. Uses the same
 * types and values as the PTP standard (PTP_AC_xxx).     
 */                                                       
typedef enum {                                            
        GP_STORAGEINFO_AC_READWRITE             = 0,    /**< \brief Storage is Read / Write. */
        GP_STORAGEINFO_AC_READONLY              = 1,    /**< \brief Storage is Ready Only. */  
        GP_STORAGEINFO_AC_READONLY_WITH_DELETE  = 2     /**< \brief Storage is Ready Only, but allows Delete.*/
} CameraStorageAccessType;                                                                                     

/**
 * \brief Filesystem hierarchy types.
 *                                   
 * The type of the filesystem hierarchy the devices uses.
 * Same types and values as the PTP standard defines (PTP_FST_xxx).
 */                                                                
typedef enum {                                                     
        GP_STORAGEINFO_FST_UNDEFINED            = 0,    /**< \brief Undefined or unknown filesystem hierarchy. */
        GP_STORAGEINFO_FST_GENERICFLAT          = 1,    /**< \brief Generic flat storage (all in 1 directory). */
        GP_STORAGEINFO_FST_GENERICHIERARCHICAL  = 2,    /**< \brief Generic tree hierarchy. */                   
        GP_STORAGEINFO_FST_DCF                  = 3     /**< \brief DCIM style storage. */                       
} CameraStorageFilesystemType;                         

/**
 * \brief Storage information structure.
 *                                     
 * This structure contains the information of a specific camera storage.
 * Only the members as specified by the \a fields member are valid.     
 */                                                                     
typedef struct _CameraStorageInformation {                              
        CameraStorageInfoFields         fields; /**< \brief Bitmask of struct members that are specified. */
        char                            basedir[256];   /**< \brief Basedirectory of the storage. Will be "/" if just 1 storage on the camera. */                                                                                                           
        char                            label[256];     /**< \brief Label of the storage. Similar to DOS label. */            
        char                            description[256];/**< \brief Description of the storage. */                           
        CameraStorageType               type;           /**< \brief Hardware type of the storage. */                          
        CameraStorageFilesystemType     fstype;         /**< \brief Hierarchy type of the filesystem. */                      
        CameraStorageAccessType         access;         /**< \brief Access permissions. */                                    
        uint64_t                        capacitykbytes; /**< \brief Total capacity in kbytes. */                              
        uint64_t                        freekbytes;     /**< \brief Free space in kbytes. */                                  
        uint64_t                        freeimages;     /**< \brief Free space in images (guessed by camera). */              
} CameraStorageInformation; 

But, like you can see, there is no universal and uniq information about camera media...
Note that Solid interface cannot help here. It cannot play indeep yet with Gphoto2 camera device.

So, the thread still open if somebody as a better way to proposal.

Gilles
Comment 35 caulier.gilles 2008-11-13 18:25:20 UTC
This bug can be considerated fixed now, since implementation for fine, accordingly with restrictions given in #34 about GPhoto2 camera.

Gilles Caulier