Bug 135484 - Thumbnail generation for multible images can cause severe overload
Summary: Thumbnail generation for multible images can cause severe overload
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Geolocation-Engine (show other bugs)
Version: unspecified
Platform: Fedora RPMs Linux
: NOR crash
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-10-12 03:36 UTC by Birkir A. Barkarson
Modified: 2018-03-23 21:03 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In: 0.1.0


Attachments
Debug statements (4.10 KB, patch)
2006-10-13 12:24 UTC, Marcel Wiesweg
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Birkir A. Barkarson 2006-10-12 03:36:48 UTC
Version:           svn (using KDE KDE 3.5.4)
Installed from:    Fedora RPMs
OS:                Linux

Since the last update thumbnail generation in gpssync is working.  However if many images are selected for update (I selected about 20 I think) and gpssync is launched then multible processes are launched to generate thumbnails for the list.  In my case these processes  overloaded my machine severly, so much that I couldn't even close the gpssync window.  (eventually had to terminate the processes via ssh from another machine).
Comment 1 caulier.gilles 2006-10-12 08:55:49 UTC
SVN commit 594746 by cgilles:

kipiplugins from trunk : GPSSync : missing to pass parent instance to QObject. 
This can cause broken dispatch events.

Please, let's me hear if this patch fix the problem. I have located others implementations 
where this problem exist.

CCBUGS: 135484


 M  +1 -1      gpslistviewitem.cpp  


--- trunk/extragear/libs/kipi-plugins/gpssync/gpslistviewitem.cpp #594745:594746
@@ -65,7 +65,7 @@
 };
 
 GPSListViewItem::GPSListViewItem(KListView *view, QListViewItem *after, const KURL& url)
-               : KListViewItem(view, after)
+               : QObject(view), KListViewItem(view, after)
 {
     d = new GPSListViewItemPriv;
     d->url = url;
Comment 2 caulier.gilles 2006-10-12 08:59:58 UTC
Gerhard,

We have see this problem on your computer during previous week end.

Also, i suspect a similar problem into multithreaded image loader from digiKam core implemented by Marcel, witch can cause large latency on gui. This is not yet fixed in digiKam. I waiting your feedback about this plugin fix for that.

Gilles
Comment 3 caulier.gilles 2006-10-12 09:08:16 UTC
Marcel,

I suspect exactly the same problem into Digikam::LoadSaveThread constructor.

On Gerhard computer, when a large picture is loaded to editor, the gui is frozen and there is no active progress bar. In fact the image is loaded using the parent instance, not a separate thread.

Also, in digiKam core, i suspect a similar problem with the thumbnails interface. On Gerhard computer, when you display album content using mouse, the thumb are loaded using parent instance. The gui is also frozen.

Please give me your viewpoint...

Note : it difficult to identify this problem on my computers. All work fine under Mandriva 2005/6/7 and Suse 10.1. I suspect a specific Qt lib compilation option used with this thread witch frok properly by default all childs thread.

Gilles
Comment 4 Gerhard Kulzer 2006-10-12 10:41:46 UTC
Hi Gilles,
your patch solved the thumb generation in the gpssync plugin for me. I tried it with 100 images and the thumbs were generated within 2 sec, no machine blocking anymore. Great!

The loading of big pictures on my computer has some nuances: when I load a picture from the main window, no progress bar is shown, the GUI is blocked until the picture is fully loaded, no kio shows up with top. 
When I then pass (with pgdown for example) from one picture to another within the editor, the kio_digikamprev is invoked and the progress bar is visible. It seem to me that the loading is quicker too.

Gerhard
Comment 5 caulier.gilles 2006-10-12 10:49:20 UTC
Thanks Gerhard. I will patch digiKam core now.

Gilles
Comment 6 caulier.gilles 2006-10-12 11:00:27 UTC
Marcel, 

Digikam::LoadSaveThread class need to use the parent instance to dispatch event to QObject via constructor. 

There are a lots of class dervated from this class. This implementation have been written by you. I let's you to fix it.

Please CCBUGS in this file your fix. Gerhard will ask you if all work fine.

Thanks in advance

Gilles
Comment 7 caulier.gilles 2006-10-12 11:11:36 UTC
Marcel,

I suspect the same problem in: 

- Digikam::AlbumThumbnailLoader 
- Digikam::ImageAttributeWatch

...constructors witch are derivated from QObject.

Gilles
Comment 8 Marcel Wiesweg 2006-10-12 12:36:15 UTC
Gilles:
Hm, I don't quite understand. There is no parent QObject for either of these classes.

Gerhard: 
"when I load a picture from the main window" means you click on a thumbnail, and the image editor is opened?
Forget about any kioslaves then, loading is done in a different thread. The ioslave is used when you press F3 in the album thumbnail view.

"the GUI is blocked" means what is also called frozen, i.e., no repainting is done, the main window is blocked as well, if you move any other window on top of any digikam window, the damaged area will not be repainted, no mouse-over animation happens if you move the cursor over the menu bar or tool bar buttons, etc.?
The other version of "blocked" is that the digikam main window is unaffected, but the GUI of the Image editor window is disabled (greyed out).

You described a difference between newly opening the image editor and changing the picture from the opened image editor.
Does this occur everytime you open the window, or only the very first time during a digikam run?
Comment 9 caulier.gilles 2006-10-12 12:52:24 UTC
Marcel,

Look the difference here :

http://websvn.kde.org/trunk/extragear/libs/kipi-plugins/gpssync/gpslistviewitem.cpp?r1=text&tr1=594449&r2=text&tr2=594746&makepatch=1&diff_format=h

Before my commit, the QObject instance used to create the GPSListViewItem object is null ! this is want mean that, in special cases (when Qt is compiled with special options i think), the main Qt event will be not available to the object. I have just connected the child QObject instance to the parent QObject instance, witch is a KListView in GPSSync plugin.

About digiKam core implementation, i think that we need to pass a parent QObject instance in object constructors to receive all events dispatched by the main Qt loop.

I have already talk about this point in the past with Renchi when i have implemented the CDArchiving kipi-plugins. With this tool, we have seen a problem to recieve the KProcess events to drive K3b.

Gilles
Gilles
Comment 10 Marcel Wiesweg 2006-10-12 13:56:26 UTC
This is very strange to me. What should be the reason for this? I postEvent(QObject *receiver, QEvent*) with a very specific receiver. The QObject::event implementation of QWidget might propagate certain events (key, mouse) up the parent hierarchy. But a general check for receiver->parent() with QCustomEvents? I wouldn't expect it even in an alpha version of Qt ;-)

Moreover, sending signals is independent from event delivery (only in Qt4 we will enjoy queued connections).

Let me try to investigate this with Gerhard, if the main thread hangs somewhere, if events really are not delivered.
Comment 11 Birkir A. Barkarson 2006-10-12 14:01:26 UTC
It seems better now, doesn't totally lock up although it does take a while to build the thumbnails.  My images are often large from 3 - 10 MB each so it does drag the system on a large number of photographs.  
Actually I would question the need for thumbnails here.  I didn't even notice that there were supposed to be thumbnails (until it was fixed) because I just always select the images from roughly the same location and then open up the gpssync dialog.  At the least wouldn't it be sensible to just generate thumbnails for the images that are in view? 
Comment 12 Gerhard Kulzer 2006-10-12 18:02:40 UTC
Am Donnerstag, 12. Oktober 2006 12:36 schrieb Marcel Wiesweg:
[bugs.kde.org quoted mail]
Yes
> Forget about any kioslaves then, 
> loading is done in a different thread. The ioslave is used when you press
> F3 in the album thumbnail view.
>
> "the GUI is blocked" means what is also called frozen, i.e., no repainting
> is done, the main window is blocked as well, if you move any other window
> on top of any digikam window, the damaged area will not be repainted, no
> mouse-over animation happens if you move the cursor over the menu bar or
> tool bar buttons, etc.? The other version of "blocked" is that the digikam
> main window is unaffected, but the GUI of the Image editor window is
> disabled (greyed out).
>
> You described a difference between newly opening the image editor and
> changing the picture from the opened image editor. Does this occur
> everytime you open the window, or only the very first time during a digikam
> run?

It seems that every time I open an image from the main window (by clicking on 
it), the parent digikam process is involved only, not just the first time. If 
I move within the IE from picture to picture the kio_digikamprev is called. 
Also, if I open a picture in IE, close it and open the following one by 
clicking kio_digikamprev is invoked (probably a look-ahead feature).

But anyway, the loading is much faster now (using the version of last night), 
a 36 MB png opens instantly.

Gerhard
Comment 13 Marcel Wiesweg 2006-10-13 12:24:26 UTC
Created attachment 18107 [details]
Debug statements

So is there a problem or not? ;-)
If there is, you find attached a patch that outputs debug messages from main
thread and loading thread. Output should look like

digikam: MainThread: ManagedLoadSaveThread::load, creating load request for
/media/fotos/Andalusien 2004/pict0053.jpg
digikam: Main thread: Now leaving ManagedLoadSaveThread::load
digikam: Loading image in thread
digikam: Doing the real work in thread, loading /media/fotos/Andalusien
2004/pict0053.jpg
digikam: /media/fotos/Andalusien 2004/pict0053.jpg : JPEG file identified
digikam: Posting loaded event for /media/fotos/Andalusien 2004/pict0053.jpg to
0x867b528
digikam: Received event of type 1000
digikam: Received event of type 1000
digikam: Received event of type 1000
digikam: Received event of type 1000
digikam: Main thread: This is the loaded event. Now the signal is sent from
LoadSaveThread
digikam: Main thread: DImgInterface::slotImageLoaded. The image data has
arrived, now display it

It's most interesting if you observe delays and watch the console at the same
time to see when the delay occurs.

Btw: The image editor does _not_ use the digikampreview ioslave. What you
observe is: When you change the picture in IE, it notifies the main view, which
marks the new picture as active, which triggers the start of the preview
ioslave.