Bug 291627

Summary:	NEF data loss after geotaging with Exiv2 - Use ExifTool instead.
Product:	[Applications] digikam	Reporter:	serge
Component:	Metadata-ExifTool	Assignee:	Digikam Developers <digikam-bugs-null>
Status:	RESOLVED FIXED
Severity:	major	CC:	ahuggel, caulier.gilles
Priority:	NOR
Version:	2.5.0
Target Milestone:	---
Platform:	Gentoo Packages
OS:	Linux
Latest Commit:	https://invent.kde.org/graphics/digikam/commit/232040bdd51ba615ba3c2ac04e4846a3cf80c1af	Version Fixed In:	8.0.0
Sentry Crash Report:
Attachments:	Structure of a corrupted NEF (otput of exiftool -htmlDump)

Description serge 2012-01-16 06:27:35 UTC

Version:           2.5.0 (using KDE 4.7.3) 
OS:                Linux

When geotaging of big portion of NEF images (>100) with writing the metadata into the images, digikam often crashes. Very approximately I usually have ~ 1 crash corresponding to ~ 50 images. Before with digikam 1.9 this never lead to data corruption. Now, with digikam 2.5 or digikam 2.4.1 I discover several corrupted nef files. Perhaps crashes are another bug, lets speak here about nef structure corruption. My NEFs are from Nikon D90.

Reproducible: Sometimes

Steps to Reproduce:
1) Enable writing the metadata into RAW
2) Geotag the big portion of NEF files (~ 500)
3) If digikam crashes, find the first unchanged file and resume the process of geotagging
4) When all files are geotagged, run 

ALL=`find -iname "*NEF"
for arg in $ALL; do exiftool -htmlDump $arg | grep Error; done

to find the broken files in the current directory.

Actual Results:  
Yesterday I geotagged ~ 500 files and 4 ones were corrupted. 

Expected Results:  
This is serious data loss - these NEF are not recoverable anymore: see output of exiftool -htmlDump

This should be fixed.

Comment 1 serge 2012-01-16 06:32:16 UTC

Created attachment 67876 [details]
Structure of a corrupted NEF (otput of exiftool -htmlDump)

The small piece of RAW data is absent.

Comment 2 serge 2012-01-16 06:33:45 UTC

Put the cursor onto "(SubIFD1:Strip data)" at the bottom to see the error.

Comment 3 serge 2012-01-16 06:35:20 UTC

If necessary I will provide the broken NEF and the original one.

Comment 4 caulier.gilles 2012-01-16 08:03:57 UTC

yes, we need original and corrupted image to compare.

I CC Andreas Huggel who can analys metadata issue with Exiv2.

I suppose that you have " write metadata to RAW file" option enabled in digiKam configuration panel. Right ?

Gilles Caulier

Comment 5 serge 2012-01-16 09:28:17 UTC

(In reply to comment #4)
> yes, we need original and corrupted image to compare.

http://www.pdmi.ras.ru/~serge/nef.zip

> 
> I CC Andreas Huggel who can analys metadata issue with Exiv2.
> 
> I suppose that you have " write metadata to RAW file" option enabled in digiKam
> configuration panel. Right ?

I usually enable this before geotaging and disable just after

Comment 6 caulier.gilles 2012-01-16 09:32:37 UTC

And which Exiv2 version you use exactly ? Got to Help/Components Info dialog for details.

Note : If you don't enable metadata writing in RAW option, geolocation data are written only in digiKam database, which ca be enough and suitable as well.

Gilles Caulier

Comment 7 serge 2012-01-16 09:37:35 UTC

(In reply to comment #6)
> And which Exiv2 version you use exactly ? Got to Help/Components Info dialog
> for details.
> 

0.21

I did not upgraded exiv2 when upgraded gigikam up to 2.4.1 from 1.9

So it seems that this is digikam2 issue

> Note : If you don't enable metadata writing in RAW option, geolocation data are
> written only in digiKam database, which ca be enough and suitable as well.
> 
> Gilles Caulier

Yes, I know this, thanks

Comment 8 caulier.gilles 2012-01-16 09:48:09 UTC

No, i don't think it's a digiKam issue. digiKam do not play with metadata directly. all is delegate to Exiv2.

But do you mean that with 1.9 release geolocation metadata work fine with NEF but not 2.x ?

Gilles Caulier

Comment 9 serge 2012-01-16 09:55:40 UTC

> But do you mean that with 1.9 release geolocation metadata work fine with NEF
> but not 2.x ?
> 

Yes, definitely. I use the same procedure with digikam 1.9 many times during a have of a year. I just have checked with exiftool that there were no broken files. The problems began after upgrade up to digikam 2.4.1

Comment 10 caulier.gilles 2012-01-16 10:01:24 UTC

And are you sure that Exiv2 used by digiKam still the same between 1.9 and 2.x release.

I recommend to take a look into Help/Component Info dialog in 1.9 version, to be sure. Both Exiv2 release can be installed on your system...

Gilles Caulier

Comment 11 serge 2012-01-16 10:08:41 UTC

Yes, I have checked carefully with /var/log/portage and took a look into Help/Component. 

The last upgrade of exiv2 was at 2011-02-04

The digikam 2.4.1 was installed at 2011-12-09 together with libkexiv2-4.6.1 (which was upgraded from 4.5.5)

Now I have digikam 1.5.0 and libkexiv2-4.7.3

Both digikam 2.4.1 and 2.5.0 are seem to be affected. Digikam 1.9 is not affected.

Comment 12 serge 2012-01-16 10:12:11 UTC

Sorry this not quite right. 

The digikam 2.4.1 was installed at 2011-12-09 together with libkexiv2-4.7.3
(which was upgraded from 4.6.1)

Comment 13 Andreas Huggel 2012-01-16 10:39:24 UTC

Just for the record, we have at least one other issue with crippled NEFs, bug 283681.

-ahu.

Comment 14 caulier.gilles 2012-06-22 08:51:53 UTC

Official digiKam 2.6.0 release is out since few days now :

http://www.digikam.org/drupal/node/656

Please, check if this entry still valid, or update report accordingly.

Thanks in advance.

Gilles Caulier

Comment 15 serge 2012-08-11 08:03:35 UTC

I cannot reproduce this anymore: 

digikam 2.7.0
exiv2 0.23
kde 4.9

Comment 16 caulier.gilles 2022-04-08 05:27:20 UTC

Git commit 232040bdd51ba615ba3c2ac04e4846a3cf80c1af by Gilles Caulier.
Committed on 08/04/2022 at 05:24.
Pushed by cgilles into branch 'master'.

More files closed with ExifTool backend used to write metadata to RAW files instead Exiv2.
Related: bug 193228, bug 235171, bug 236127, bug 305823, bug 317745, bug 234181, bug 366406, bug 168064, bug 134487
FIXED-IN: 8.0.0

M  +14   -4    NEWS

https://invent.kde.org/graphics/digikam/commit/232040bdd51ba615ba3c2ac04e4846a3cf80c1af