Bug 151552

Marcel,

Sound like a problem in DImg cache mechanism. right ?

Arnd,

I'm not sure, but i think there is already a similar report in B.K.O, perhaps in Image Editor component...

Gilles

Maybe this one is related?
  http://bugs.kde.org/show_bug.cgi?id=113797

krienke,

What's news about this file ? it still valid using digiKam 0.9.4 ?

Gilles Caulier

0.9.5.svn: unfortunately this is still problem.

Mik,

And with 0.10.0 ? A lots of code have been re-written and perhaps with Qt4, problem disapear...

Gilles Caulier

unfortunately this is still problem
Version 0.10.0-beta7 (rev.: 893451)

I can confirm for the thumbnail view, but not for preview/imageeditor.

Just tried with 0.10. Its not the latest svn version I have (about 4 weeks old), but I could still reproduce the bug both for icon preview as well as in image editor.

Rainer

I have committed a number of changes that touches this area. Some functionality was missing and some bugs need to be fixed. I have tested some but not all situations.

There is a number of situations where updating of thumbnails/preview will _not_ work:

- it relies on KDirWatch watching file changes. If this does not work on your system, you wont have updates
- I chose not to install watches for thumbnails. If you overwrite a file that is displayed as thumbnail (and not as preview/in image editor), and change it without deleting it first (e.g. with "cp"), the KDirWatch may not notice this.
There were problems with a high number of file watches in the past.
- If you edit your file containing an IPTC preview with a program that does not know about such things, the thumbnail may never change if the preview did not change

krienke,

Please test with official release of 0.10.0-beta7 or better current implementation from svn. Thanks in advance

Gilles Caulier

Hi,

I still see a problem in this area with digikam 0.10.0-beta8. Thumbnails and embedded preview do not get updated after image editing actions at all, as long as 
"Misc options -> embedded preview loads full size" is unchecked. It doesn't make a difference if the editing action is done from the main menu or from the image editor. If I switch to full size preview everything is fine.

My setup:
KDE Version  0.10.0-beta8 (KDE 4.1.87 (KDE 4.1.87 (KDE 4.2 >= 20090101)) "release 3.1", KDE:KDE4:UNSTABLE:Desktop / openSUSE_11.1) on Linux (x86_64) release 2.6.27.7-9-default

All the best

Thomas

To be more precise: the embedded preview gets updated. The thumbnails most of the time do not. Even the thumbnails within the image editor do not follow changes done in the editor.

Open image in image editor, select area, crop, save + overwrite
=> Preview image, preview in lighttable, thumbnail in icon view, in preview thumbnail bar, in lighttable thumbnail bar, in editor thumbnail bar is updated.

Can anyone else please check?

Note that this is 100% _not_ KDirWatch dependent.

Thomas, please check if your pictures have embedded previews. See in right sidebar, Metadata panel, IPTC, select "Full list", look for "Preview Data" (with binary data)

worksforme, svn info: 909172

Works perfectly fine for me, all widgets are automatically updated.

Marcel,

I have the feeling that you are right on track with your idea that the problem
may be related to internal thumbnails. The bug is quite subtile, and I had
difficulties to reproduce it myself. At the beginning, using freshly imported
images, all thumbnail updates seemed to work just fine.

Please try these steps: 

- I started with fresh png files without internal previews, in my case created
with ImageJ
- import in digikam
- open an image in the digikam image editor, crop (resulting in a landscape
image), save (overwrite). Now the image has an internal preview as shown in the
IPTC data. 
- select image thumbnail in album view, do a "Rotate left" using the main menu.
The thumbnail redraws at the end of the rotation, but it stays unrotated
(landscape).
- If I open this image again in the editor, it is landscape in the editors
thumbnail and portrait in the editor. 
- Now I do a crop in the editor to make the image square and save, overwriting
the original again. 
- All thumbnails in editor, album view, and light table still show the
landscape image.


My system: OpenSuse 11.1,  KDE Version 4.1.87 (KDE 4.1.87 (KDE 4.2 >=
20090101)) "release 3.1", Version 0.10.0-beta8 (OpenSuse rpm)

Hope this helps


Thomas

Gilles, this is a very interesting and strange problem. To reproduce:

1) Take a PNG image.
2) exiv2 -di to remove Iptc (metadata plugin doesnt do it for me at the moment)
3) Open in editor, rotate (or crop, change color, any effect), Save
=> thumbnail is fine
4) Close digikam and start again (to rule out any caching problem)
5) Open in editor, rotate (or crop, change color, any effect), Save
=> thumbnail is unchanged everywhere, shows result of step 3
6) Close digikam and start again
7) Open in editor, rotate (or crop, change color, any effect), Save
=> thumbnail is updated, shows result of step 5!!

Repeat as long as you want, thumbnail always shows the result of the step before. 

Digikam was closed in the meantime => it must be somewhere in the image.

Control with exiv2 -PIkyct and compare the byte number with the one displayed from setImagePreview debug message.
Control with Gwenview that the image is really saved as the latest version.
Latest SVN version of exiv2.

What's that???

Marcel,

In first, you talking about preview image (JPEG) saved in IPTC, not Exif thumbnail. digiKam never use Exif thumbnail to render thumb in icon view.

I know this problem. i remember to have already see it with KDE3 code...

I have already hacking indeep but without sucess. All sound fine in code.

What's i' sure : when editor has transformed an image, IPTC preview is recomputed when save files is called. The new preview image is a scaled version using QImage:

http://lxr.kde.org/source/extragear/graphics/digikam/utilities/imageeditor/canvas/dimginterface.cpp#622

If you save this QImage to a dumy png file, you will see that preview is fine.
Preview is passed to DMetadata container and IPTC byte array containing preview to DImg object.

Later, PNGLoader is called by DImg to save really image to file. Here, there is 2 way :

Exiv2 < 0.18 => PNG iptc byte array is saved using libPNG, thrue PNGLoader code.
Exiv2 >= 0.18 = PNG iptc byte array is saved by Exiv2 since i have added PNG writting mode in library.

I have never tested if this problem still exists with Exiv2 >= 0.18, but in both case iptc must be written in PNG.

If i remember my investiguations, iptc data (outside IPTC preview) are written and updated in iptc, but preview sometime failed to be updated to iptc byte array.

For me, it's sound like a bug in Exiv2, but i'm not sure. After investiguations, i cannot see where the problem can be in libkexiv2.

Gilles Caulier

PS: Andreas, i CC you for info...

http://digikam3rdparty.free.fr/TEST_IMAGES/METADATA/magic-color.png

This picture is an example of the problem. Open it in the image editor (not preview) and see that the thumbnail is wrong. Iptc preview size as with exiv2 is 84665.
Now edit the image in any way, and you will see a thumbnail of size 87821 appear that looks slightly broken because I inserted 0xdeadbeef (at position 60000) with a once modified libkexiv2. When this 0xdeadbeef (broken thumbnail) appears on your computer, this is a proof that there are _two_ previews inside the Iptc data, of which only one is shown, and adding a third one deletes the first and pushes the second to position one.

(Note: closing digikam in between steps is essential to create such cases)

SVN commit 920100 by cgilles:

force to remove IPTC preview before to update it...
CCBUG: 151552


 M  +7 -8      dimginterface.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=920100

Marcel, 

I can reproduce the problem here with test image. incredible.

To be sure, with commit #920100, IPTC preview tags are always removed before to update it. This don't fix the problem.

I think about a bug in Exiv2. Andreas ?

Gilles

Andreas,

Some pointers for your investigations :

When digiKam image editor save image, code which render IPTC preview is given below (cod eis not updated to LXR):

...
    // Get image Exif/IPTC data.
    DMetadata meta;
    meta.setExif(d->image.getExif());
    meta.setIptc(d->image.getIptc());
    meta.setXmp(d->image.getXmp());

    // Update IPTC preview.
    // NOTE: see B.K.O #130525. a JPEG segment is limited to 64K. If the IPTC byte array is
    // bigger than 64K during of image preview tag size, the target JPEG image will be
    // broken. Note that IPTC image preview tag is limited to 256K!!!
    // There is no limitation with TIFF and PNG about IPTC byte array size.

    // Before to update IPTC preview, we remove it.
    meta.removeIptcTag("Iptc.Application2.Preview");
    meta.removeIptcTag("Iptc.Application2.PreviewFormat");
    meta.removeIptcTag("Iptc.Application2.PreviewVersion");

    QSize previewSize = d->image.size();
    previewSize.scale(1280, 1024, Qt::KeepAspectRatio);
    QImage preview;
    // Ensure that preview is not upscaled
    if (previewSize.width() >= (int)d->image.width())
        preview = d->image.copyQImage();
    else
        preview = d->image.smoothScale(previewSize.width(), previewSize.height(), Qt::IgnoreAspectRatio).copyQImage();

    // With JPEG file, we don't store IPTC preview.
    // NOTE: only store preview if pixel number is at least two times bigger
    if (/* (2*(previewSize.width() * previewSize.height()) < (int)(d->image.width() * d->image.height())) &&*/
        (mimeType.toUpper() != QString("JPG") && mimeType.toUpper() != QString("JPEG") &&
         mimeType.toUpper() != QString("JPE"))
       )
    {
        // Non JPEG file, we update IPTC preview
        meta.setImagePreview(preview);
    }

    // Update Exif thumbnail.
    QImage thumb = preview.scaled(160, 120, Qt::KeepAspectRatio, Qt::SmoothTransformation);
    meta.setExifThumbnail(thumb);

    // Update Exif Image dimensions.
    meta.setImageDimensions(d->image.size());

    // Update Exif Document Name tag with the original file name.
    meta.setExifTagString("Exif.Image.DocumentName", getImageFileName());

    // Update Exif Orientation tag if necessary.
    if( setExifOrientationTag )
        meta.setImageOrientation(DMetadata::ORIENTATION_NORMAL);

    // Store new Exif/IPTC/XMP data into image.
    d->image.setExif(meta.getExif());
    d->image.setIptc(meta.getIptc());
    d->image.setXmp(meta.getXmp());
...
DMetadata is a devirated class of libkexiv2. It include some digiKam internal methods not shared with the rest of KDE. 

http://lxr.kde.org/source/extragear/graphics/digikam/libs/dmetadata/dmetadata.h

d->image is an instance of DImg container used to store image data + image metadata. 

http://lxr.kde.org/source/extragear/graphics/digikam/libs/dimg/dimg.h

We clear Iptc preview tags, and render preview as reduced image. This one is passed to DMetadata container. Unforget here that we save PNG image.

http://lxr.kde.org/source/KDE/kdegraphics/libs/libkexiv2/libkexiv2/kexiv2image.cpp#841

After that, we update IPTC metadata to DImg instance, and we can start to save really the image to PNG file. In this case, DIMG call a dedicated image writter which use libpng :

http://lxr.kde.org/source/extragear/graphics/digikam/libs/dimg/loaders/pngloader.cpp#556

In this code, you will see similar code than Exiv2 to make metadata PNG chunks.
In fact i must to update this image writter to use libpng to save metadata if Exiv2 < 0.18 else Exiv2 as well. But here this is not the problem i think.

Gilles

SVN commit 921012 by cgilles:

libkexiv2 from trunk: add test program to add/update IPTC preview to image.
Dedicated to investigate B.K.O #151552
CCBUG: 151552
CCMAIL: ahuggel@gmx.net
CCMAIL: marcel.wiesweg@gmx.de


 M  +1 -0      CMakeLists.txt  
 A             test (directory)  
 AM            test/CMakeLists.txt  
 AM            test/setiptcpreview.cpp   [License: GPL (v2+)]


WebSVN link: http://websvn.kde.org/?view=rev&revision=921012

Marcel,

With my new test program included in libkexiv2 from trunk, i cannot reproduce the problem with your png test image. 

Test program load image, rotate it, and generate iptc preview as image editor do. preview is saved in image and is extracted again to be save to an external PNG file.

Your viewpoint ?

Gilles

SVN commit 921033 by cgilles:

digiKam from trunk : force to save metadata with Exiv2 in all case. Since Exiv2 is out PNG is upport in writting mode.
CCBUG: 151552


 M  +2 -3      pngloader.cpp  
 M  +1 -1      pngloader.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=921033

Marcel,

With my commit #921033, the problem disappear... Why ? because i suspect somthing is become wrong in DImg::PNG loader to save metadata chunk... and i don't know what... overwriting with exiv2 as a second pass solve the problem...

krienke,

Can you test again using last code from svn ?

Gilles Caulier

Ah yes this is the first time that this bug makes any sense to me. Our own implementation of writing metadata blocks to PNG is the problem!
So with your fix now, we could remove the whole code block line 740-810?

>So with your fix now, we could remove the whole code block line 740-810? 

yes, but only if Exiv2 >= 0.18 is used to complie libkexiv2

Gilles

EXIV2_TEST_VERSION is not available outside libkexiv2 (and that's fine I think).
I think we can use KExiv2::supportMetadataWritting("image/png") to test this?

yes, of course marcel...

Gilles

SVN commit 922679 by mwiesweg:

Use manual writing with libpng only when exiv2 version does not support it.
(99% if changes are only intendation change)

BUG: 151552

 M  +62 -55    pngloader.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=922679