Bug 193967

Summary: VERY fast loading of thumbnails [PATCH]
Product: [Applications] digikam Reporter: Mikolaj Machowski <mikmach>
Component: Thumbs-ImageAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: wishlist CC: marcel.wiesweg
Priority: NOR    
Version: 1.0.0   
Target Milestone: ---   
Platform: Compiled Sources   
OS: Linux   
Latest Commit: Version Fixed In: 1.0.0
Sentry Crash Report:
Attachments: %s/Smooth/Fast/ Transformation
FastTransformation - 137px thumbnail size
256px thumbnail size
hack'n'slash patch for 'cheatscale'
Thumbnails settings panel from Gwenview for KDE3

Description Mikolaj Machowski 2009-05-25 01:28:44 UTC
Version:            (using Devel)
Compiler:          gcc4.3.2 
OS:                Linux
Installed from:    Compiled sources

With this simple patch loading of thumbnails is much, much faster. Of course it comes with price - slightly lower quality of thumbs (but only on display, generated thumbnails are of good quality).

It is done by simple change from SmoothTranformation to FastTransformation.

I understand that this trade off isn't good for everyone but digiKam experience becomes much smoother after removing of Smooth transformation ;)

Several possible solutions:
- use that patch as is and accept slightly lower quality (really visible only with smallest thumbnails of 80px - and really, who could expect super quality from those thumbs ;)
- make option in config dialog a) lower quality of thumbs, faster loading b) high quality of thumbs, slower browsing of albums
- load thumbnails in two passes: first time with FastTransformation, second, after eg. 2-3 seconds of inactivity with Smooth transformation
Comment 1 Mikolaj Machowski 2009-05-25 01:30:05 UTC
Created attachment 33985 [details]
%s/Smooth/Fast/ Transformation
Comment 2 Mikolaj Machowski 2009-05-25 02:33:58 UTC
I was too fast. First replace is harmful -> it generates lower quality thumbs. Have to test it more - but second part is still valid!
Comment 3 caulier.gilles 2009-05-25 06:30:47 UTC
SVN commit 972480 by cgilles:

optimize QImage scaled 2nd pass to use fast transform instead smooth transform. Quality still here.
BUG: 193967


 M  +7 -8      thumbnailcreator.cpp  
 M  +3 -0      thumbnailcreator_p.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=972480
Comment 4 Andi Clemens 2009-05-25 09:40:20 UTC
For me thumbs have bad quality now, Gilles i guess yours are ok because they have been already created before?
What when removing .thumbnails folder?

I remember some blog entry on the QtLabs homepage where some way was introduced to have good AND fast scalings. Need to search for this...

Andi
Comment 5 Andi Clemens 2009-05-25 09:44:43 UTC
Ah, here it is:
http://labs.trolltech.com/blogs/2009/01/26/creating-thumbnail-preview/
Comment 6 Andi Clemens 2009-05-25 09:50:15 UTC
Also, this cheat scale seems to be even faster then FastTransformation, but looks like SmoothTransformation.
Comment 7 caulier.gilles 2009-05-25 09:52:42 UTC
Andi,

Of course, i have tested to remove ./thumbnails (:=))) It still good quality without anti-aliasing if thumbs size is less than 256. At 256, smoothscale version is used (from cache).

What do you mean by bad quality exactly ? Can you take a screenshot ?

Gilles Caulier
Comment 8 caulier.gilles 2009-05-25 10:00:32 UTC
Andi,

The question is : why i cannot see any signifiant differences here when size < 256 ?

Gilles
Comment 9 Andi Clemens 2009-05-25 10:31:37 UTC
Created attachment 33988 [details]
FastTransformation - 137px thumbnail size
Comment 10 Andi Clemens 2009-05-25 10:32:12 UTC
Created attachment 33989 [details]
256px thumbnail size
Comment 11 Andi Clemens 2009-05-25 10:33:18 UTC
As you can see in the above screenshots, the outlines are very jaggy.
Comment 12 Mikolaj Machowski 2009-05-25 16:11:03 UTC
Andi - for me 256px thumbnail is smooth as previously. Smaller thumbs *are* jagged but not to such extent. Only in 80px I see something like on your screenshot and only on pictures with high contrast. What is your graphic card and drivers? (I have old Nvidia with legacy, proprietary OpenGL drivers)

Gilles - maybe some misunderstanding? My idea/patch was only simple replacement of Smooth with Fast, there is no second pass and writing of it is beyond my capabilities.

Personally this trade off - lower quality of thumbnails for faster loading of them is preferable - even welcomed with enthusiasm. Previously loading of whole screen of thumbs could take 4-5 seconds. Now filling the screen with thumbs take about 1 second. Judging from number of wishes for preloading of thumbs speed is important for users.
Comment 13 Andi Clemens 2009-05-25 16:20:55 UTC
1 second? When generating the thumbnails? Then my system is crap, although I have a MacBookPro Core 2 Duo. It takes 8 seconds and more for one page (smallest thumbsize, approx 40 thumbs per page).
But if you mean the loading of the thumbs that are already cached / generated, this is not faster for me.
The approach in the above Qt Labs blog entry should be even faster then FastTransform AND look better, so maybe we should try this method?

Andi
Comment 14 Mikolaj Machowski 2009-05-25 17:41:53 UTC
Below I will attach copy'n'paste patch with "cheat" method from Ariya.

1) his method doesn't care about aspect ratio so you will see squares but as quick tool for tests it is enough
2) it is slightly slower than fast transformation - he probably tested it on big images where it is easier to gain performance, we are going only for scaling 256px downwards to 80px; on my screen smallest thumbs (55 pieces) take  3 seconds to fill the screen; It would be interesting to see what this method gives for generation of real thumbs; all times are for already generated thumbs and I see difference between loading of pregenerated thumbs before and now (Gilles patch doesn't touch generation of thumbs at all)
3) my computer: Sempron 2200, 756MB RAM (233 MHz), GeForce2 MX 400; Qt4.5.1, KDE4.3svn
4) bit tricky legal issues: file of origin where cheatScale is coming from is under GPL2 or 3, while thumbnailcreator.cpp is GPL2+
5) what is taking so much time is loading of thumbnails in sequence; would it be possible to load them simultaneously (thread per thumb or at least few threads per whole screen)?
Comment 15 Mikolaj Machowski 2009-05-25 17:44:05 UTC
Created attachment 33995 [details]
hack'n'slash patch for 'cheatscale'
Comment 16 Marcel Wiesweg 2009-05-25 18:03:48 UTC
We should be careful, if we use FastTransformation and users notice that thumbnails look "bad" then it makes a really bad impression on the average user.
On my system, thumbnail loading is not significantly bothering me at all. Creating takes its time, but loading is fast enough. I usually use thumbnail size 160 and I never "wait" for thumbnails to appear once they are created.
Comment 17 Andi Clemens 2009-05-25 18:14:03 UTC
Yes, here too. I don't see a difference once the thumbs are generated. The creation is slow, this could be faster (maybe with this cheat method, so it would also look nice?)

Andi
Comment 18 caulier.gilles 2009-05-25 18:21:17 UTC
#16 : I'm agree with Marcel. We must take a care to not make a bad impression
about thumbs.

#14 : agree with Mik, but speed-up improvements can be see on Single CPU
computer. On Double core, nothing (tested on 8 differents computers)

What do you think about this way :

1/ Always compute cache with SmoothScale to 256x256.
2/ In second pass,

- If size is 256, do nothing.
- If 256 > size >= 128 : use cheatscale
- If size < 128 : use FastScale (we cannot see details indeep here).

Or perhaps alwyas use cheatscale if size < 256. In all case, no need to to
touch thumb if size = 256.

Gilles
Comment 19 Mikolaj Machowski 2009-05-25 19:05:28 UTC
BTW - very, very interesting link about generation of thumbnails:

http://www.4p8.com/eric.brasseur/gamma.html
Comment 20 caulier.gilles 2009-05-25 19:24:19 UTC
Andi,

Another way to try is to use DImg instead QImage with Fast Scale algorithm taken from imlib2...

Gilles
Comment 21 Mikolaj Machowski 2009-05-25 19:48:48 UTC
Gilles

1) as Andi wrote it would be good to test how generation of thumbs with cheatScale is going
2) I think I understand misunderstanding. For you second pass is when thumbnails were generated and only loaded from disk; for me second pass means that when loading thumbs they are loaded in two steps - first with FastTransformation, second time, after 2-3 seconds of inactivity in Album GUI, with SmoothTransformation; in your terminology it would 2.5 or 3 pass.

About sizes:
size == 256 do nothing - as far as I understand this works now
size < 128 - in this area side effects of FastTransformation are really visible so here users will be hit hard in quality

When 3 people told me that they see no difference I recompiled digiKam for each of three variants (Smooth, Fast, cheat) to test again. Well, I see difference between Smooth and Fast - maybe not so dramatic as I perceived yesterday - but there is. May it be not worth to sacrifice quality? For me trade-off is worth but for most of users of this solution it is not.

So, I propose to revert this whole thing but still investigate use of cheatScale for .thumbnails generation.
Comment 22 Marcel Wiesweg 2009-05-26 18:43:39 UTC
> So, I propose to revert this whole thing 

I strongly agree. For me all thumbnails look pixellated with the current solution.
Comment 23 caulier.gilles 2009-05-26 19:15:55 UTC
SVN commit 973246 by cgilles:

revert 
CCBUGS: 193967


 M  +1 -2      thumbnailcreator.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=973246
Comment 24 caulier.gilles 2009-05-26 19:17:00 UTC
SVN commit 973247 by cgilles:

revert 
CCBUGS: 193967


 M  +1 -2      digikamthumbnail.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=973247
Comment 25 caulier.gilles 2009-05-28 12:35:27 UTC
SVN commit 973970 by cgilles:

try cheat scale. It's really faster now ???
CCBUGS: 193967


 M  +8 -2      thumbnailcreator.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=973970
Comment 26 caulier.gilles 2009-05-28 12:50:57 UTC
Let's me see if thumbnails creation is a little bit faster for you now...

I still convinced that a lots of time is used to save png file to .thumbnails cache. 

Just do a simple test :

1/ start digiKam 
2/ go to a console and remove ~./thumbnails (let's open digiKam session)
3/ now navigate into albums : it's really fast

Why ? Because ~./thumbnails dir disappear and digiKam cannot save in cache. There is no more access disk.

Gilles
Comment 27 caulier.gilles 2009-05-28 12:55:08 UTC
Created attachment 34073 [details]
Thumbnails settings panel from Gwenview for KDE3

Look like Gwenview has an option to disable thumbs cache synchronization. This is why it can be very fast to generate thumbnails on icon view.
Comment 28 Mikolaj Machowski 2009-05-28 17:10:08 UTC
@26, Gilles - this trick doesn't work for me, digiKam here creates .thumbnails dir and writes there :)
@25, Gilles - yes it is faster - one thing I missed previously, in this place digiKam creates thumbs only for non-jpg files so I see difference for TIFFs. JPEGs are handled by local copy of jpegutils.
Comment 29 Mikolaj Machowski 2009-05-28 17:47:08 UTC
@27, Gilles - made simple tests by commenting out code responsible for writing of qimage to disk. There are speed gains when viewing images for first time but when going to the same folder second time it is approximately two times slower than with support of .thumbnails cache. For image viewer like Gwenview it is acceptable but not for full blown image organizer like digiKam.

Could be done for Showview.