Bug 233094

One thing to rule out is a problem with libpgf. For this test, change in libs/threadimageio/thumbnailcreator.cpp, line 498,
    dbInfo.type             = DatabaseThumbnail::PGF;
to
    dbInfo.type             = DatabaseThumbnail::PNG;

From now on you will create PNG thumbnails in your database. Please check if that makes any difference.

If there is a difference, this problem will be relevant of libpgf running under PPC computer. Please confirm... In this case, i will contact libpgf team for feedback

Gilles Caulier

Yeah, success! It is libpgf again. Thanks for the quick reaction, Marcel and Gilles! Now I've also got the best digiKam ever, finally. :-)

Ok thanks. I CC libpgf team for info.

Gilles Caulier

Raphael,

Please look at this file. Sound like another libpgf problem under PPC computer. Here, Vlado see thumbnails blured in digiKam icon view. It's typically a compression problem with libpgf, i think.

Code relevant in digiKam is there :

http://lxr.kde.org/source/extragear/graphics/digikam/libs/threadimageio/pgfutils.cpp#83

This method is used to create a bytearray of a compressed PGF image
computed from a QImage instance. This bytearray is stored in digiKam Database as well.

To extract PGF bytearray from database and create a QImage, we use this method :

http://lxr.kde.org/source/extragear/graphics/digikam/libs/threadimageio/pgfutils.cpp#53

The rest of thumbnail process in digiKam is Qt4 based code, and the problem is not there.

About libpgf compiled in digiKam core, we use this CMake code :

http://lxr.kde.org/source/extragear/graphics/digikam/CMakeLists.txt#1002

There is no special compilation flag set for libpgf.

Finally, lipgf code is there :

http://lxr.kde.org/source/extragear/graphics/digikam/libs/3rdparty/libpgf/

Thanks in advance for your help

Gilles Caulier

Thank you for reporting and all the helpful details.
A PPC will be at my disposal starting April 6th. I'll look into it immediately.

Raphael Schweizer

Raphael,

Great. If you need more info, ask to Vlado. he has a PPC computer (I don't have it).

If you want to compile digiKam, just post your questions here. All developers are in this room.

Gilles Caulier

Gilles,

Investigation has started.
Using Ubuntu 9.10 and the current pgf codec w/ console I could not find a difference between big and little endian encoded images so far. I'm currently trying to compile my own svn version of digiKam (repository version is at 1.0.0, which crashes on PPCs, as we know :-D) CMake says:
--
CMake Error at libkipi/CMakeLists.txt:60 (INSTALL):
 install FILES given no DESTINATION!
CMake Error at libkipi/pics/CMakeLists.txt:1 (KDE4_INSTALL_ICONS):
 Unknown CMake command "KDE4_INSTALL_ICONS".
--
What am I doing wrong?

I suppose digiKam calculates the thumnail images once and scales these images afterwards? So if the thumbnails look OK after generation and get blurry after scaling, where does libpgf come in? I do, however, not understand why the problem disappears if dbInfo.type = DatabaseThumbnail::PNG

Raphael Schweizer

To compile digiKam, you need to compile and install shared libraries from kdegraphics, and after compile digiKam and kipi-plugins. Look instruction here :

http://www.digikam.org/drupal/download/KDE4

Using svn trunk version is the best, because it's current code.

To solve external dependencies, look in README files, all is explained indeep. It's not difficult...

About PNG stuff, yes, this is strange. PNG encoder is libpng based (through Qt4 API), and PGF codec is dropped to render thumbs. This is why i suspect libpgf here.

Gilles Caulier

My suggestion for compiling digiKam in Ubuntu 9.10:

Get the digiKam 1.2.0 tarball and follow instructions on that website:
http://www.digikam.org/drupal/download?q=download/tarball

You can get all build dependencies for digiKam by doing
# apt-get build-dep digikam

It is also possible to create proper deb packages, but compiling that way takes longer, because it creates both a normal and a debug version:
http://mail.kde.org/pipermail/digikam-users/2010-March/010083.html

I don't think it is necessary to experiment with the latest code, if the problem is already there in version 1.2.0.

Raphael, I hope you can reproduce and solve my problem (although it is not urgent for me, thanks to the PNG workaround)! I experienced different results on Debian and on Ubuntu 10.04 - digiKam on Ubuntu 9.10 might behave differently again.

Vlado,

I has a question for you ? Can you test again using PGF codec on your computer, and change compression level of PGF image. There is only an hardcoded value to change in source code here :

http://lxr.kde.org/source/extragear/graphics/digikam/libs/threadimageio/thumbnailcreator.cpp#504

The last parameter '4' is PGF compression level. Using smaller value compress lesser. '0' value is lossless compression as PNG.

Can you test with 3, 2, 1, and 0 to see if blurring effect disappear ?

Note : Of course don't forget to revert changes done following #1 from Marcel to use PGF codec.

Thanks in advance

Gilles

Created attachment 42593 [details]
comparison between PGF compression levels 4 and 3

Gilles, you seem to be on the right track. Changing PGF compression levels has a significant effect, as can be seen in the attached comparison between levels 4 (default) and 3. I did these tests with digiKam 1.2 on Debian sid / PPC. I also checked level 0 (before level 3), but I can't see a significant difference between 0 and 3, so I did not bother attaching a level 0 screen-shot.

Vlado,

Thanks to comparing compression levels. i will set PGF compression to 3 immediately in source code.

Raphael,

Any comments about difference between PGF compression level 3 and 4 ? Why the image become blurred ? It's a side effect of wavelets compression ? If yes, why i cannot reproduce i never see that with JPEG 2000 ?

Gilles Caulier

Gilles,

starting with compression level 4 downsampling is used (see DownsampleThreshold in http://lxr.kde.org/source/extragear/graphics/digikam/libs/3rdparty/libpgf/PGFtypes.h#53), which may explain a small difference. Judging from Vlado's screenshots there must be something else. Perhaps libpgf is reporting wrong level sizes on PPCs, such that scaling (http://lxr.kde.org/source/extragear/graphics/digikam/libs/threadimageio/pgfutils.cpp#129) does not work correctly. I'll check that.

Raphael Schweizer

Created attachment 42783 [details]
PGF thumbail channel map patch

Gilles,

the blurriness issue is finally demystified:

1) on big endian systems, libpgf did a wrong YUV transform of thumbnails (missing ARGB / BGRA map) which led to increased data size
2) libPGF crashed when a ARGB PGF was loaded (short channel map).

a) loadPGFScaled only works with (A)RGB PGFs (some mapping PGF.Mode <-> QImage.Mode is needed)
b) Judging from screenshots thumbnail images are stored 256x256 but displayed 254x254 (with black border). Resizing introduces blurriness.

Issues 1) & 2) are solved with the attached patch. a) and b) are outside the scope of this bug.

Please note that a rebuild of the thumbnails is needed on big endian systems.

Raphael Schweizer

Thanks Raphael,

I will patch digiKam.

Yes thumbs are 256x256 and are downscaled to 254x254. I thinl that image must be cropped instead. Marcel ?

Gilles

SVN commit 1114868 by cgilles:

patch from Raphael Schweizer to play properly with PGF image under PowerPC computer
CCBUGS: 233094


 M  +17 -4     pgfutils.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1114868

Referring to this code?
void ThumbnailLoadThread::slotThumbnailLoaded(const LoadingDescription& description, const QImage& thumb)

        pix = QPixmap(w + 2, h + 2);
        QPainter p(&pix);
        p.setPen(QPen(Qt::black, 1));
        p.drawRect(0, 0, w + 1, h + 1);
        p.drawImage(1, 1, thumb);

The pixmap is created two pixels larger than the loaded thumbnail. No scaling is done here.

Thanks Marcel,

So nothing can be done anymore in this file. I close it now.

Gilles Caulier

Marcel,

please excuse me for my wrong accusation.
On my work machine the thumb strip on top (when viewing thumbs and preview) can be expanded just so far that the thumbnail is at most 254x254 (which led to the wrong conclusion). In a VM I just used, it as at most 246px. Maybe you could allow resizing the upper part of the UI to allow "native" sized thumbnails.

Raphael

Raphael,

In comment #14, you talk about a threshold for downsampling image. Currently it set to level 3. 

It's possible to add an option to disable this feature in compression process if quality is more than 3 ?

Gilles

Gilles,

it is possible to set a higher threshold in http://lxr.kde.org/source/extragear/graphics/digikam/libs/3rdparty/libpgf/PGFtypes.h#53 Set it to 31 to completely disable downsampling. However, no one should be able to distinguish visually images compressed on q4 with downsampling en- or disabled.

I recommend using the PGFconsole to experiment with encoding settings and observing PSNR / size changes.

Raphael

Created attachment 42801 [details]
Modified to write debug image files

Vlado,

though we did find some PPC / big endian issues we couldn't reproduce the dramatic quality loss you demonstrated in comment #12

I would be pleased to investigate further. Do you mind compiling digiKam using the attached pgfutils.cpp (and comparing it's results to the PGFconsole [SVN: https://libpgf.svn.sourceforge.net/svnroot/libpgf]). pgfutils is modified to write to "/tmp/":
a) writeQImage.bmp (the thumbnail image we encode)
b) writeStream.pgf (the encoded thumbnail)
c) readStream.pgf (the pgf read from the thmbnails db)
d) readQImage.bmp (the decoded bitmap, returned to digiKam)
I'm interested in all those files (+ a) encoded with the console and afterwards decoded + the original image.)

In order to make this work you'll have to create an album with only the image of interest, load this album (-> a,b) and resize the thumb (-> c,d)

I know this is a lot of work, please let me know if I can assist in any way.

Raphael

Ok, Raphael, your instructions for investigating the issue further seem clear enough to me. I should be able to produce these debug files on Sunday, but probably not earlier.

SVN commit 1115266 by mwiesweg:

Adjust the ImageThumbnailBar so that showing thumbnails up to full 256 is possible.
For this we need to find out pixel-exactly which viewport size (we know the max value for
that) corresponds to which widget size (which we can set a max value for).
It seems to work to just take the difference of the two at one point of time.
Additionally, add the two pixels used for the highlighting border.

CCBUG:  233094

 M  +16 -4     digikam/imagethumbnailbar.cpp  
 M  +1 -0      digikam/imagethumbnailbar.h  
 M  +2 -1      digikam/imagethumbnaildelegate.cpp  
 M  +8 -0      libs/threadimageio/thumbnailloadthread.cpp  
 M  +2 -0      libs/threadimageio/thumbnailloadthread.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1115266

Created attachment 42874 [details]
patch to digiKam 1.2.0's pgfutils.cpp to create debug files in /tmp

Raphael,
using your patched pgfutils.cpp from 2010-04-15 (comment 23) I also could not reproduce the blurriness issue. The reason seems to be that you patched the current SVN version of pgfutils.cpp, which already includes your patches from 2010-04-14 (comment 15)! Isolating the debug patch and merging the two relevant sections into digiKam 1.2.0's original pgfutils.cpp led to PGF files in /tmp that where buggy, showed in strange colours in showfoto, and even lead to a program crash (although that crash could also have been caused by something else). I'll send you the debug images (~ 500 KByte) in a private e-mail.

In case anyone is curious to (better) see the picture I mainly used for comparison:
http://www.vlado-do.de/photos/image.php?src=australia_2008&img=2008-11-21__17-48-38

Gilles, judging from the tests I ran today with digiKam 1.2.0 source code it seems not necessary to reduce the PGF thumbnails' compression level (in thumbnailcreator.cpp), if the current pgfutils.cpp (from SVN) is used. So you might want to undo that compression level change in SVN before the next digiKam release.

Vlado,

thank you for creating those files. I'm glad to hear that the issues are resolved in the newest version.
The strange colors are caused by the missing int[] map. PGF internally works with YUV images and by default imports / exports BGRA bitmaps. If the caller supplies / needs ARGB format, we have to supply the corresponding mapping (ARGB -> BGRA: [3, 2, 1, 0]). As these are missing from 1.2.0 digiKam, libPGF does a wrong YUV transformation and the images end up 'blueish'. However, I can't explain how this is linked to the blurriness. The crash was most likely caused by a 'short' map, which is also fixed in SVN.
Encoding the debig_svn/writeQImage.bmp with the windows version of libPGF now gives the same result as on your PPC. Therefore, digiKam 1.3.0 should work as expected.

Raphael

>Gilles, judging from the tests I ran today with digiKam 1.2.0 source code it
>seems not necessary to reduce the PGF thumbnails' compression level (in
>thumbnailcreator.cpp), if the current pgfutils.cpp (from SVN) is used. So you
>might want to undo that compression level change in SVN before the next digiKam
>release.

Yes, i will take a look about. it's in my TODO list

Gilles

SVN commit 1116312 by cgilles:

restore PGF quality 4 for thumb compression. 
Please check if thumbnails are not to blured there with this value, else, use only 3 instead...
CCBUGS: 233094


 M  +4 -3      pgfutils.h  
 M  +3 -2      thumbnailcreator.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1116312

To Vlado,

PGF compression quality is restored to 4. Please checkout svn trunk again, and re-generate all tumbnails using "Tool/Rebuild Thnumbnails" option. Check if blur effect is back or not.

Thanks in advance

Gilles Caulier

Marcel, 

I'm not sure if i found a bug about color management and thumbs view.

There i generated all thumbs with CM enabled for thumbs, and disabling lead CM option do not change something. i use a dumy ICC profile as wide-gamut to see the difference, anyway sRGB do not give visible effect.

Do you store thumbs in DB with CM correction or as well, and you apply CM only when thumbs are displayed ?

Vlado, Just to be sure, do you use Color Management on your system ?

Gilles

Marcel,

in revision 1116346 resizing the thumbnailbar to allow 256x256 images is not possible for me. Tried using work machine (resolution 1600x1200) and VM (do. & 1400x1050).

Raphael

SVN commit 1116583 by mwiesweg:

For the preview thumbbar, also adjust the thumbnail size to reach 256

CCBUG: 233094

 M  +1 -1      digikam/imagethumbnaildelegate.cpp  
 M  +7 -0      libs/threadimageio/thumbnailloadthread.cpp  
 M  +1 -0      libs/threadimageio/thumbnailloadthread.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1116583

Gilles, I currently can't compile SVN digiKam. Recently you answered someone's question "But svn requires kde 4.5?" with "yes digiKam from trunk depand of libkdcraw from trunk which is KDE 4.5" on digikam-users.

Anyway I now compiled digiKam 1.2 with pgfutils.cpp from SVN (and no other modifications) and the thumbnail quality seems good again. I tried this in Debian and Ubuntu. So I guess the big-endian problem is properly fixed now.

To answer you other question, Gilles: I don't use color management on my system.

Thanks Vlado,

Yes, current code from svn need libkdcraw and libkexiv2 from trunk which will be published with KDE 4.5.

But libkdcraw and libkexiv2 do not depand of KDE 4.5 and compile fine with older KDE version. Just checkout code from kdegraphics/libs, compile and install these libraries, and recompile digiKam /kipi-plugins from trunk as well. 

More details here :

http://www.digikam.org/drupal/sharedlibs

http://www.digikam.org/drupal/download?q=download/KDE4

Here i use Mandriva 2010.0 which use KDE 4.3.x. No problem.

Gilles Caulier

Created attachment 42934 [details]
Maximum thumbnail size on Ubuntu 1600x1200

Marcel,

as of rev. 1117090 I still can't resize the thumbnailbar to hold images of 256px (see attached, cropped screenshot). Do you need additional information?

Raphael

Gilles,

after you gave these more detailed instructions I tried to compile libkdcraw and libkexiv2 from trunk in my Debian unstable system. libkdcraw compiles all right, but libkexiv2 does not. Something seems to go wrong with the configuration, but there is only a warning "No cmake_minimum_required command is present". Still the Makefile generated by cmake is just 3.7 KByte, whereas the Makefile for libkdcraw is about 14 KByte. Calling "make" for libkexiv2 does not seem to do anything.

I could give it another try, if you have any suggestions, but I'm confident it is not necessary for the libpgf problem anymore.

You have to checkout the whole kdegraphics tree from that svn that contains libs/libkexiv2 as a subfolder, call cmake directly in kdegraphics, cd tp libs/libkexiv2 and call make, make install there.

Raphael: This affects only the thumbbar, in the main icon view you can reach 256?
Which widget style do you use, which KDE/Qt version?
There is no really clean solution, because I know the size the viewport shall have, while I can only set the size of the whole widget, including border, scrollbar etc. I tested here but maybe it doesn't work in other settings.

Ok, the problem is definitely fixed now. Today I tested digiKam from SVN (rev.: 1117432) on Debian/PPC, and thumbnails are looking great again.