| Summary: | Metadata lost when converting from png to jpeg (IPTC thumbnail too big) | ||
|---|---|---|---|
| Product: | [Applications] digikam | Reporter: | Paweł Marciniak <pave> |
| Component: | Plugin-Bqm-Convert | Assignee: | Digikam Developers <digikam-bugs-null> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | ahuggel, avvisi, caulier.gilles, pave |
| Priority: | NOR | ||
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Platform: | unspecified | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed In: | 5.0.0 | |
| Sentry Crash Report: | |||
| Attachments: |
The jpeg file that causes problems with svn exiv2
The difference in metadata before and after running convert The same picture as attachment 1 with fixed IPTC IRB sizes |
||
This looks similar to the bug reported last month. http://bugs.kde.org/show_bug.cgi?id=150801 See comment #1 Geoff, It is similar, but your report applies to digiKam core, while mine has to do with batch process plugin. Anyway, the bug described here is rather an ImageMagick bug, so I'm not sure if on the side of KIPI we are able to do anything more than some ugly workaround. Pawel, Look my commits #738759. In convertimagesdialog.cpp, i have add a new virtual method named processDone(). This one is called by main batch process loop when a conversion from a file to another one is done. Code is commented in this method. Uncomment it and test. Just update too libkexiv2. I have patched this one to report the Exiv2 error code on the console. Try to convert PNG files generated by digiKam image Editor (with IPTC preview data) to JPEG and look the result. Here it doesn't properlly... sometimes. Sound like a race condition failure somewhere. Gilles SVN commit 738840 by cgilles: Kipi-plugins from KDE3 branch : Batch Process : restore metadata properlly with JPEG file during conversion. Pawel, This patch sound like properlly under KDE3.5, but not under KDE 3.4. I suspect a bug in KProcess or something like that. BUG: 152210 M +1 -2 convertimagesdialog.cpp WebSVN link: http://websvn.kde.org/?view=rev&revision=738840 Pawel, Please test and give me your feedback. Thanks in advance Gilles Unfortunately, there's still a problem when Exiv2 is supposed to load the metadata corrupted by convert program. The image->readMetadata() call in kexiv2.cpp:381 says: Warning: IPTC dataset Iptc.Application2.Preview has invalid size 120301; skipped. Warning: Failed to decode IPTC metadata. It's happening during construction of the metaTgt object in convertimagesdialog.cpp. So the object is broken and further call to metaTgt.applyChanges() throws Exiv2 error 22: Input data does not contain a valid image. The first solution that comes to my mind is to use source metadata as a base for removing the preview, but then we loose the changes to metadata that convert program probably does. As to the race condition you mentioned, I already reported one in bug #152208 . Perhaps that's the same beast... That's strange! The problem with metaTgt I described happens only with svn version of exiv2. With stable 0.15 it works fine... Anyway, maybe we could just copy the source metadata after stripping the preview? Seems like convert doesn't change metadata much. I did a comparison and only few Canon-specific exif tags were changed. Pawel, Yes, i can reproduce the problem here. With Exiv2 0.15 all is fine, and with 0.16-svn, something fail... Andreas, What do you think about ? Gilles To the original bug report: Since Exiv2 v0.15, reading of Exif and IPTC metadata is decoupled, i.e., if reading IPTC fails, it will not affect reading of Exif. Re comment #6: The Exiv2 error 22 in the svn version sounds like a bug, since it worked in 0.15. I'll look into this, can you please send me such a corrupted JPG? Re comment #8: Can convert handle makernotes, what does "change a few canon-specific tags" mean? Are there any valid makernote tags left? Created attachment 22180 [details]
The jpeg file that causes problems with svn exiv2
Created attachment 22181 [details]
The difference in metadata before and after running convert
This is a diff between exif dumps (exiv2 -v -b pr -p h) of png and jpeg file. I
rearranged some parts of the dumps in order to help the diff program exclude
identical (but relocated) sections.
Thanks for this image. I can reproduce an Exiv2 Error 22 exception with this image. The problem is rather fundamental: The image has its IPTC data split over two APP13 Photoshop 3.0 segments it seems (and thus answers this old question: http://bugs.kde.org/show_bug.cgi?id=130525#c32). Exiv2 can only deal with one such segment in a JPEG. I've opened Exiv2 issue #533 (http://dev.robotbattle.com/bugs/view.php?id=533) to fix this. However, I get exactly the same behaviour in 0.16-pre1 and 0.15 and none of the previous revisions can handle this case. So can you please elaborate what exactly is the situation which you only observe with 0.16-pre1 but not with 0.15, maybe there is another issue here? -ahu. Created attachment 22410 [details] The same picture as attachment 1 [details] with fixed IPTC IRB sizes The original image contains two APP13 Photoshop 3.0 segments, both contain an IRB with IPTC data. However, both of these IRBs have wrong sizes (0x0adf for both). The new attachment is the same image as the original but the IRB sizes are fixed (0xffdc and 0x0b04). Using some new code (not checked-in yet) to parse the JPEG segments, I can now get all the IPTC datasets out of this image with exiv2, including the preview image (68198 bytes JPEG thumbnail). However, exiftool doesn't like the way the IPTC data is split over the two IRBs, and I have no specification which says how it is actually supposed to be done. So I'm not sure yet what it should be. That should read "same picture as attachment 22180 [details]", above. Nice feature :)
Paweł, if you still have it, could you please also send me the original PNG file which you converted to the JPEG posted as attachment 22180 [details]?
Thanks,
Andreas
Andreas, You is back... Welcome... we suspect a little problem in Exiv2 0.16 against 0.15. Thanks to review it Gilles well, it looks like my comments didn't make it to the list because i wasn't registered with kde-imaging. i've fixed that now. pls see my earlier comment 13 above (and comment 14 for a better explanation): i can reproduce the problem but it happens with all versions. need more input on the issue which happens only with the latest version. andreas In response to Andreas' comment #16, I uploaded the png file at http://pandora.cs.put.poznan.pl/~pmarciniak/img_7469.png The file is over 7 megs in size and B.K.O. doesn't accept such big attachments. A little late (a LOT of work recently), sorry for that, but I hope this still helps. Best, Paweł Pawel, This file still valid using kipi-plugins 0.1.6 and a recent version of Exiv2 library ? Gilles Caulier Just repro'ed it with digikam 0.9.4, kipi-plugins 0.1.6 and libexiv2 0.17.1 Console output when opening the converted jpg image: Warning: IFD0 tag 0x0001 has invalid Exif type 0; using 7 (undefined). Warning: IPTC dataset Iptc.Application2.Preview has invalid size 68198; skipped. Marcel, with jpeg, we don't create iptc preview, but in case of converting PNG to JPEG, and if PNG has already iptc preview, this one is not removed (in editor code). I think we need to always remove it with JPEG (to be sure) Pawel, this warning come from Exiv2 which skip iptc preview when size is > to 64 Kb. Andreas, Can you confirm that all other iptc tags are not removed in this case ? Gilles Caulier >> Warning: IPTC dataset Iptc.Application2.Preview has invalid size 68198; skipped. > Andreas, Can you confirm that all other iptc tags are not removed in this case ? Yes, all other IPTC tags are not affected by this. -ahu. Pawel, This file still valid using kipi-plugins 2.4 ? Gilles Caulier Hi Gilles, Unfortunately I don't have access to a system where I could test it. Feel free to resolve the bug if nobody else is able to repro it. Thanks, Pawel New Kipiplugins 4.11.0 is available : https://www.digikam.org/node/740 Can you reproduce the problem with this release ? Gilles Caulier BatchProcessImage is not maintained since a while and is obsolete now. It will be removed with 5.0.0. Use digiKam BQM instead... |
Version: svn (using KDE KDE 3.5.8) Installed from: Unlisted Binary Package OS: Linux A preview (an embeded and reduced JPEG image) that is set by digiKam in IPTC to speed up preview loading (F4) can be larger than 64Kb (see iptc/iim paper) and a jpeg section (used to embeded iptc in jpeg file) is limited to 64Kb. There is no size limitation with PNG (or TIFF). Therefore, when png files are created at the time of downloading images from camera, too big previews are generated. After converting such pngs to jpegs with batch convert and trying to open new jpegs in digiKam, no metadata are shown (neither IPTC nor EXIF) and the console says: Warning: IPTC dataset Iptc.Application2.Preview has invalid size 67904; skipped. IPTC data should be stripped during conversion ('convert +profile iptc') so that at least EXIF is readable in the newly created jpegs.