Bug 268831 - Interrupted-restarted import results in empty file
Summary: Interrupted-restarted import results in empty file
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Import-MainView (show other bugs)
Version: 1.8.0
Platform: Ubuntu Linux
: NOR major
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-03-18 17:24 UTC by Adam Porter
Modified: 2016-07-15 16:14 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In: 5.1.0


Attachments
Screenshot of file-replace dialog (54.22 KB, image/png)
2011-03-18 17:24 UTC, Adam Porter
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Adam Porter 2011-03-18 17:24:19 UTC
Created attachment 58148 [details]
Screenshot of file-replace dialog

Version:           1.8.0 (using KDE 4.6.1) 
OS:                Linux

I was downloading images from my iPhone when my system froze hard.  I don't know what caused that.  Anyway, I rebooted and went back to the import dialog.

1.  The image it had just finished downloading when the system froze (not the file which was "downloading" on the screen when it froze) is a zero-byte file in the album directory.  Digikam didn't notice that the file on the camera (iPhone) differs from the file in the album on-disk, and so it indicates it is not a new photo.

2.  When I select and download the photo anyway, it asks me if I want to replace the file on disk with the file from the camera.  But the replace dialog shows that both files are zero-byte files, even though the file on the camera is an intact JPEG file.  I clicked Overwrite and it correctly overwrote the zero-byte file with the file from the camera--but the dialog should have shown the file from the camera correctly.

3.  The file it was downloading when the system froze somehow ended up on-disk, but the metadata (EXIF, etc) was not imported into Digikam's db.  I had to manually reread the metadata from the image file.

Conclusion: I realize that system freezes aren't supported behavior, however Digikam should be more robust.  When importing photos, it should look at more than the filename to determine if an image is new.  It should at the very least check the filesize.  It ought to check some metadata too, like the EXIF creation date.  A checksum would be nice, though since it would be slower I suppose it should be optional.  Personally, I'd trade the extra time for extra reliability, knowing that Digikam will always import all photos successfully without any false duplicates.

It also seems to me that Digikam should not mark an image in the db as having had its metadata imported until it actually has.

Reproducible: Didn't try




OS: Linux (i686) release 2.6.35-28-generic
Compiler: cc
Comment 1 Adam Porter 2011-03-18 17:49:24 UTC
This bug is worse than I realized.  A total of 12 images ended up as zero-byte files on the disk in the album directory.  But:

1.  Digikam shows them as not-new images in the import dialog.

2.  In the on-disk album view, Digikam displays thumbnails and metadata for all of these images which are zero bytes on the disk!  Only when double-clicking the thumbnail does Digikam say "Cannot display preview for..."

So unless the user opens the album directory in Dolphin and verifies for himself that the files are intact, he may cheerfully delete his only copies of the images from his camera.

I am using ext4, by the way.

Perhaps fortuitously, I was reading a Debian bug about dpkg running on ext4 filesystems today and I believe Theodore Ts'o's solution may be applicable here:

http://article.gmane.org/gmane.linux.debian.devel.bugs.general/771761

I suspect Digikam isn't even calling fsync as images are downloaded from the camera.  That would be ok if it didn't think they were downloaded successfully, because they could be redownloaded/reimported after rebooting.  But Digikam puts both the thumbnails and EXIF metadata of unwritten files into the database, and displays them in the album view as if the images are intact on-disk, leaving the poor, unsuspecting user with no idea that the files were never written to the disk.

I'm sure we can all agree that Digikam should always fail safely and assume nothing when it comes to protecting users' data, right?  Personally, I hate having to babysit software to make sure it does the right thing.  :)
Comment 2 caulier.gilles 2011-07-06 11:22:11 UTC
We need feedback using a recent version. 2.0.0 RC is out, please test...

Thanks in advance

Gilles Caulier
Comment 3 caulier.gilles 2011-08-04 13:39:18 UTC
Adam,

I recommend to test with current implmentation from git master (future digiKam 2.1.0) where i fixed a lots of bugs...

Thanks in advance

Gilles Caulier
Comment 4 caulier.gilles 2011-12-18 13:53:25 UTC
Adam, we need a fresh feedback using digiKam 2.4...

Gilles Caulier
Comment 5 Adam Porter 2011-12-19 13:47:49 UTC
I'm sorry, I don't have 2.4 available in my repos, and I don't have
time to build it right now.

This is a bug that anyone can test.  Just start an import and then
pkill -9 Digikam, then restart Digikam and see what happened.

Really, this is a matter of importing files atomically, is it not?  If
you haven't programmed it to work atomically yet, the bug will still
exist, won't it?

On Sun, Dec 18, 2011 at 07:53, Gilles Caulier <caulier.gilles@gmail.com> wrote:
> https://bugs.kde.org/show_bug.cgi?id=268831
>
>
> Gilles Caulier <caulier.gilles@gmail.com> changed:
>
>           What    |Removed                     |Added
> ----------------------------------------------------------------------------
>             Status|UNCONFIRMED                 |RESOLVED
>         Resolution|                            |WAITINGFORINFO
>
>
>
>
> --- Comment #4 from Gilles Caulier <caulier gilles gmail com>  2011-12-18 13:53:25 ---
> Adam, we need a fresh feedback using digiKam 2.4...
>
> Gilles Caulier
>
> --
> Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You reported the bug.
Comment 6 caulier.gilles 2012-06-22 08:53:48 UTC
Official digiKam 2.6.0 release is out since few days now :

http://www.digikam.org/drupal/node/656

Please, check if this entry still valid, or update report accordingly.

Thanks in advance.

Gilles Caulier
Comment 7 caulier.gilles 2014-05-24 06:18:14 UTC
digiKam 4.0.0 is out. Please check if this file still reproducible on your computer

Thanks in advance

Gilles Caulier
Comment 8 caulier.gilles 2014-09-01 07:45:10 UTC
Adam,

We need a fresh feedback here using last digiKam 4.2.0.

Gilles Caulier
Comment 9 caulier.gilles 2015-06-28 09:42:24 UTC
New digiKam 4.11.0 is available :

https://www.digikam.org/node/740

Can you reproduce the problem with this release ?

Gilles Caulier
Comment 10 caulier.gilles 2015-08-24 05:16:24 UTC
digiKam 4.12.0 is out :

https://www.digikam.org/node/741

We need a fresh feedback using this release please...
Thanks in advance.
Comment 11 caulier.gilles 2016-07-15 16:14:45 UTC
With digiKam 5.0.0, this problem is not reproducible.
I close this file now. Don't hesitate to re-open if necessary.
Gilles Caulier