Bug 232054

Strange... I had to introduce a QThread::wait(), when the KFileMetaDataWidget gets destroyed, but this should not be executed in combination with the Information Panel (only when closing Dolphin).

Do you have tooltips enabled in your environment? Because of the wait() I've to adjust the Dolphin code slightly.

I'll do some tests next week, by temporary adding a very expensive, blocking loop in the KLoadFileMetaDataThread. Even in this case Dolphin may never hang...

Neither tooltips nor the "Preview" image on the Information panel is enabled. I use the Information Panel only to display additional Meta Data.

Created attachment 42332 [details]
backtrace while it hung

I catched a backtrace while it hung when I clicked on a small image (2K file size) to view it. While this backtrace indicates a DBus hang when called from KToolInvokation, I also see the same hangs when just hovering over icons, but not reproducable. I am sure it is DBus related also, as the 15 sec timeout seems to resolve the lock.

Thanks Christoph for the update! I also did some investigations in the meantime. After doing some minor fixes in KFileMetaDataWidget & related classes I locally added an expensive loop in the KLoadFileMetaDataThread, which blocks the thread for around 7 seconds and results in 100 % CPU load. After this I turned on tooltips and the Information Panel to check, whether there is a thread related blocking of the Dolphin UI. Result after a few days of testing: No blocking of the user interface is done by the KFileMetaDataWidget thread code. Of course everything gets slower because of the 100 % CPU load, but no blocking is done. So I hope, that the recent locks are not directly related to the KFileMetaWidget code... If you plan to turn on tooltips, please update kdelibs + Dolphin in parallel, as quite some fixes have been done there.

But I also faced some locks on trunk, which sometimes where not view-related at all. E. g. when opening the Dolphin preferences recently, the UI was blocked for around 10 seconds - very strange... I'll leave my "blocking code" there locally and will try to get a backtrace too if the next lock occurs.

Created attachment 42686 [details]
another backtrace while it hung

This backtrace does not involve KRun, it apparently hung while receiving a timer event during normal browsing.

SVN commit 1120270 by ppenz:

Tag related fixes:
- Restore click-on-tags functionality
- Assure that the tags-widget state gets applied to the resources

General fixes:
- Don't start a massupdate-job if an internal data change has been done (this might be the root cause of bug #232054)
- Don't invoke ResourceManager::init() in the context of the load-meta-data thread

CCBUG: 232054
CCMAIL: frank78ac@googlemail.com

 M  +43 -26    kfilemetadataprovider.cpp  
 M  +1 -1      kfilemetadataprovider_p.h  
 M  +12 -0     kfilemetadatawidget.cpp  
 M  +2 -0      kfilemetadatawidget.h  
 M  +1 -2      kloadfilemetadatathread.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1120270

Thanks for the update, Peter. Unfortunately, with kdelibs r1120299, kdebase r1120300, the D-Bus hangs still happen.

Same problem here on Arch Linux, using KDE 4.4.3

*** Bug 240939 has been marked as a duplicate of this bug. ***

Still in 4.4.80

http://randomguy3.wordpress.com/2010/06/01/d-bus-threading-issues/ has some infos about this issue

I just downloaded D-Bus packages which are supposed to be build with the patch mentioned in comment #11, restarted the session, and I still experience the 15 second hangs when browsing and clicking on files.

As far as I understand, the patch should only fix thread-related crashes with Soprano, but I have Strigi and Nepomuk disabled.

Thanks Christoph for testing the D-Bus related issues!

> As far as I understand, the patch should only fix thread-related
> crashes with Soprano, but I have Strigi and Nepomuk disabled.

In KDE 4.5 Dolphin also gets the meta data for files, if Strigi (to be more precise: the Strigi indexer) and Nepomuk are disabled. This is done by using KFileMetaInfo in a separate thread. KFileMetaInfo uses Strigi internally (only the indexer got disabled) and my guess is, that Strigi might be affected too.

However looking at the backtraces, the issue seems to be not related to Strigi/Nepomuk at all. Does disabling tooltips and the Information Panel in Dolphin "fix" the 15 second hangs? If this is the case, then this issue is not related to the KFileMetaData classes...

I've set Sebastian Trüg to CC, as is more familiar with this issue.

just have experienced it with 4.5-Beta2

still in 4.4.90

*** Bug 243077 has been marked as a duplicate of this bug. ***

Peter, bug 243077 is not a duplicate. During the hangs I experience, Dolphin does not use the CPU, it just waits for the D-Bus timeouts.

Thanks Christoph, I've adjusted the state of bug 243077 to NEW now. In this case bug 243077 might be a Strigi issue...

maybe is the same, investigating the 100% CPU I see that almost is IOWait and, thanks to the kde forum guys, with iotop I can see   
 TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND             
 4675 be/4  gio      87.03 M/s  0.00 B/s  ?unavailable?       dolphin -icon system-file-manager -caption Dolphin

and the time that the access to the disk seems to be the same for reading all the 1GB file (the video file that mouse overing; about 12 sec with 1.1GB file, so with 90.00 M/s average in 12 seconds he reads 1.1 GB, the file size). The issue becomes dramatic when you mouse over more than one big file, because the IO is duplicated, triplicated and so on.  I notice that if I wait the IO reading, the second time I mouse over the file there isn't IO read anymore and the system remains stable. Strigi is disabled.

@Giovanni: Thanks for the update! Disabling strigi in the system settings only disables the "strigi indexer", when reading meta data kdelibs internally still uses the Strigi API. My guess is that the root cause is this: http://sourceforge.net/tracker/?func=detail&aid=2830904&group_id=171000&atid=856302 The fix should not be too tricky, but I did not have the time yet to investigate into this issue :-(

Same bug here with kde 4.5 rc2...

Dolphin hangs when single clicking on images/videos/...

ArchLinux.

Same with kde 4.5 rc3...

Really annoying bug... Dolphin became unusable...

This should be marked as BLOCKER.

4.5 was defintivly the last cycles I did beta-testing for, there are already enough testers but bugs rot their way into releases regulary.

I am able to consistently reproduce this bug in 4.4.95 by either mousing over several documents to generate a preview in the Information panel, or by right-clicking a file and choosing Properties. Dolphin will freeze for about 15-20 seconds, then continue on, or crash. This behavior only occurs with the Information panel open. When it is closed, Dolphin seems stable.

Unfortunately, I can't provide any useful backtraces, so I am not much help.

I'm having the same problem with Gwenview and opening JPG files on KDE 4.4.92 (4.5RC2) - it like one for few files freezes Gwenview for at least 10 seconds then it gets back to normal for another few files. I don't use Dolphin only occasionally so I haven't checked the problem there.

I'm getting the log output from Gwenview:
gwenview(10439) Gwenview::SemanticInfoDirModel::setData: Semantic info cache for KUrl("file:///media/disk-1/Zdjecia-2010_07_21/CZERWIEC 2010/S6303089.JPG") is invalid 
QPainter::begin: Paint device returned engine == 0, type: 2
QPainter::begin: Paint device returned engine == 0, type: 2

gwenview(10439) Gwenview::SemanticInfoDirModel::setData: Semantic info cache for KUrl("file:///media/disk-1/Zdjecia-2010_07_21/CZERWIEC 2010/S6303056.JPG") is invalid 
gwenview(10439) Gwenview::SemanticInfoDirModel::setData: Semantic info cache for KUrl("file:///media/disk-1/Zdjecia-2010_07_21/CZERWIEC 2010/S6303055.JPG") is invalid
...

I have Nepomuk and Strigi enabled but these files are on external USB disk (which condition is good, it's not read error or sth).

Program received signal SIGSEGV, Segmentation fault.
0x00007fffecb65939 in ?? () from /usr/lib64/libdbus-1.so.3
(gdb) bt
#0  0x00007fffecb65939 in ?? () from /usr/lib64/libdbus-1.so.3
#1  0x00007fffecb46add in ?? () from /usr/lib64/libdbus-1.so.3
#2  0x00007fffecb5845d in ?? () from /usr/lib64/libdbus-1.so.3
#3  0x00007fffecb4a49e in ?? () from /usr/lib64/libdbus-1.so.3
#4  0x00007ffff45ab8e3 in ?? () from /usr/lib64/qt/lib/libQtDBus.so.4
#5  0x00007ffff4288953 in QObject::event(QEvent*) () from /usr/lib64/qt/lib/libQtCore.so.4
#6  0x00007ffff511d79c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /usr/lib64/qt/lib/libQtGui.so.4
#7  0x00007ffff5123c8b in QApplication::notify(QObject*, QEvent*) () from /usr/lib64/qt/lib/libQtGui.so.4
#8  0x00007ffff5e45086 in KApplication::notify(QObject*, QEvent*) () from /usr/lib64/libkdeui.so.5
#9  0x00007ffff4278f0c in QCoreApplication::notifyInternal(QObject*, QEvent*) () from /usr/lib64/qt/lib/libQtCore.so.4
#10 0x00007ffff42a5c82 in ?? () from /usr/lib64/qt/lib/libQtCore.so.4
#11 0x00007ffff42a27f8 in ?? () from /usr/lib64/qt/lib/libQtCore.so.4
#12 0x00007fffef554f9e in g_main_context_dispatch () from /usr/lib64/libglib-2.0.so.0
#13 0x00007fffef558958 in ?? () from /usr/lib64/libglib-2.0.so.0
#14 0x00007fffef558a80 in g_main_context_iteration () from /usr/lib64/libglib-2.0.so.0
#15 0x00007ffff42a24c3 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/qt/lib/libQtCore.so.4
#16 0x00007ffff51cc6ce in ?? () from /usr/lib64/qt/lib/libQtGui.so.4
#17 0x00007ffff4277832 in QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/qt/lib/libQtCore.so.4
#18 0x00007ffff4277c0c in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/qt/lib/libQtCore.so.4
#19 0x00007ffff427c89b in QCoreApplication::exec() () from /usr/lib64/qt/lib/libQtCore.so.4
#20 0x00007ffff7b8b98f in kdemain () from /usr/lib64/libkdeinit4_dolphin.so
#21 0x00007ffff29e2b6d in __libc_start_main () from /lib64/libc.so.6
#22 0x0000000000400789 in _start ()
(gdb)

(In reply to comment #26)
> Program received signal SIGSEGV, Segmentation fault.
> 0x00007fffecb65939 in ?? () from /usr/lib64/libdbus-1.so.3
> (gdb) bt
> #0  0x00007fffecb65939 in ?? () from /usr/lib64/libdbus-1.so.3
> #1  0x00007fffecb46add in ?? () from /usr/lib64/libdbus-1.so.3
> #2  0x00007fffecb5845d in ?? () from /usr/lib64/libdbus-1.so.3
> #3  0x00007fffecb4a49e in ?? () from /usr/lib64/libdbus-1.so.3
> #4  0x00007ffff45ab8e3 in ?? () from /usr/lib64/qt/lib/libQtDBus.so.4
> #5  0x00007ffff4288953 in QObject::event(QEvent*) () from
> /usr/lib64/qt/lib/libQtCore.so.4
> #6  0x00007ffff511d79c in QApplicationPrivate::notify_helper(QObject*, QEvent*)
> () from /usr/lib64/qt/lib/libQtGui.so.4
> #7  0x00007ffff5123c8b in QApplication::notify(QObject*, QEvent*) () from
> /usr/lib64/qt/lib/libQtGui.so.4
> #8  0x00007ffff5e45086 in KApplication::notify(QObject*, QEvent*) () from
> /usr/lib64/libkdeui.so.5
> #9  0x00007ffff4278f0c in QCoreApplication::notifyInternal(QObject*, QEvent*)
> () from /usr/lib64/qt/lib/libQtCore.so.4
> #10 0x00007ffff42a5c82 in ?? () from /usr/lib64/qt/lib/libQtCore.so.4
> #11 0x00007ffff42a27f8 in ?? () from /usr/lib64/qt/lib/libQtCore.so.4
> #12 0x00007fffef554f9e in g_main_context_dispatch () from
> /usr/lib64/libglib-2.0.so.0
> #13 0x00007fffef558958 in ?? () from /usr/lib64/libglib-2.0.so.0
> #14 0x00007fffef558a80 in g_main_context_iteration () from
> /usr/lib64/libglib-2.0.so.0
> #15 0x00007ffff42a24c3 in
> QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) ()
> from /usr/lib64/qt/lib/libQtCore.so.4
> #16 0x00007ffff51cc6ce in ?? () from /usr/lib64/qt/lib/libQtGui.so.4
> #17 0x00007ffff4277832 in
> QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from
> /usr/lib64/qt/lib/libQtCore.so.4
> #18 0x00007ffff4277c0c in
> QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from
> /usr/lib64/qt/lib/libQtCore.so.4
> #19 0x00007ffff427c89b in QCoreApplication::exec() () from
> /usr/lib64/qt/lib/libQtCore.so.4
> #20 0x00007ffff7b8b98f in kdemain () from /usr/lib64/libkdeinit4_dolphin.so
> #21 0x00007ffff29e2b6d in __libc_start_main () from /lib64/libc.so.6
> #22 0x0000000000400789 in _start ()
> (gdb)

in kde 4.5. RC3 Dolphin 1.5, i want delete a folder with shift+del

Jeffrey: I don't think this is related to this issue. Please search for another appropriate bug-report or open a new one.

Same issue here, but closing the information panel solve the problem temporary, just like Marc.

Created attachment 49586 [details]
Make MpegAnalyzer respect the max stream read size

This patch applies to strigi svn trunk in kde's kdesupport module. It would help a lot if one of you could test if it solves your problems with video files.

@Sebastian: Thanks a lot for the patch, I owe you a beer the next time we meet (hopefully soon at a Nepomuk workshop?). I've less time during the next days to test the patch, but can do this at the weekend (of course it would be great, if somebody else could verify this also before ;-)) I hope I'll find some time to fix some related Strigi crashes too: As Dolphin now uses KFileMetaInfo as fallback when Nepomuk is disabled, those Strigi issues (that have been there always) popup up a lot more frequently now...

*** Bug 246020 has been marked as a duplicate of this bug. ***

as you can read in https://bugs.kde.org/show_bug.cgi?id=246020 I’m affected by this bug too…
Btw: I do not have Nepomuk nor Strigi activated. Both are disabled in the systemsettings.

@Panagiotis: this has nothing to do with Nepomuk or its strigi integration. This is Dolphin's own strigi integration which you cannot disable. But we now have a way to fix it and if all goes well that fix will be in 4.5.1.

So all the 4.5 users will have to work with Dolphin freezing every now and then for 10-20s until 4.5.1 comes out?

Its just a personal opinion, but KDE's beta/rc stages are way too short.
Do you think Microsoft would ship windows with one of its core-componnets broken like this? Well, they also take more than 2 month for stabilization.

>This behavior only occurs with the
>Information panel open. When it is closed, Dolphin seems stable.

I don't use "Information panel" and dolphin freeze/crash...

But at work, i can't reproduce this bug :(

A short update, as in the meantime 3 different issues are reported here:

- There is an issue in libdbus, that is one root cause for the freeze in
Dolphin and also other applications (see
https://bugs.freedesktop.org/show_bug.cgi?id=17754). The issue is fixed already
and I hope that the packagers will provide an updated package, as the whole KDE
SC is affected by this, not only Dolphin. In Dolphin the issue gets triggered
more often in KDE SC 4.5, as Dolphin asynchronously fetches meta data also for
non-indexed files as soon as tooltips are enabled or the Information Panel is
enabled.

- Dolphin uses KFileMetaInfo from kdelibs to retrieve meta data of non-indexed
files (= if Nepomuk and Strigi are turned off). A limit of 64 KB is set in
KFileMetaInfo, so that all meta information analyzers never read more than 64
KB of a file to get the meta information. However there are some analyzers,
that don't respect this size limit, which of course gets problematic for a 2 GB
video file. In this case the UI of Dolphin should not completely freeze, as the
analyzes is done asynchronously. But I observed that some of the analyzers go
completely mad and decrease the performance of the whole system until it feels
like frozen. The analyzers are part of Strigi (they are used also by
KFileMetaInfo if Strigi is turned off, as turning off Strigi only turns off the
indexing functionality of Strigi). I'm not very familiar with the Strigi code
yet, but I'll try to fix those issues with a high priority.

- Dolphin will crash if a Strigi analyzer crashes. Similar like before: The
analyzer must be fixed.

I'm very unhappy with the current situation, as of course Dolphin will get
blamed for this. But from looking at the bug reports, most of the freezes are
related to the libdbus issue and I hope that the recent fix there will improve
the situation.

@Bellegarde:
> I don't use "Information panel" and dolphin freeze/crash...

Did you enable tooltips? If turning off tooltips does not help, then this should be a completely different issue.

*** Bug 246136 has been marked as a duplicate of this bug. ***

The DBus issues are a major pain.

I've tested Sebastian's patch. It stops the crashes on my system.

There are a couple of issues with KLoadFileMetadataThread. I'll email you ( Peter & Trueg ) about them. I think I might even be able to patch them up.

Would it be possible to delay KDE-4.5 until this issue is solved?
Its really a huge problem with crashes and hangs, kde-4.5.0 shouldn't be shipped broken like that.

SVN commit 1158299 by trueg:

Make sure we never let libstreamanalyzer (Strigi) read more than 64k in quick analysis mode.
This is necessary since many Strigi plugins simply ignore the config.
While at it I started cleaning up the whole KFileMetaInfo code. Classes were scattered over
different files, KDE style guides were not followed, things were unnecessarily complicated.
This is only the first step mainly to make the Dolphin freeze go away. Next steps include
replacing PredicateProperties. It was never implemented and is not necessary now since we have
Nepomuk::Types::Property. Whatever features PredicateProperties wanted to expose at some point
in time (as I said: nothing besides the name is implemented) we should rather do that in
Nepomuk::Types::Property.

BUG: 232054
FIXED-IN: 4.5.1

 M  +157 -83   kfilemetainfo.cpp  
 M  +2 -2      kfilemetainfo.h  
 D             kfilemetainfo_p.h  
 M  +24 -24    kfilemetainfoitem.cpp  
 M  +8 -5      kfilemetainfoitem.h  
 A             kfilemetainfoitem_p.h   kfilemetainfo_p.h#1158298 [License: LGPL (v2+)]
 M  +3 -1      kfilewriteplugin.cpp  
 A             kfilewriteplugin_p.h   [License: LGPL (v2+)]
 M  +16 -56    predicateproperties.cpp  
 M  +4 -2      predicateproperties.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1158299

I agree with Clemens Eisserer: Shipping KDE 4.5.0 with an unstable filemanager is not a good idea, IMHO it is a showstopper. Tagging for 4.5.0 seems not to be finished, can this fix be included there as well?

I agree with Clemens Eisserer and Stefan Fleckenstein, too. It should be considered as showstopper for 4.5.0. Many KDE users will upgrade from 4.4.x to 4.5.0 expecting stable and polished desktop and can be frustrated about this bug.

I agree with Clemens Eisserer, Stefan Fleckenstein and Tomasz Czapiewski. This problem should be fixed in 4.5.0
This version of KDE is very stable and good, unfortunately this bug affects an important desktop component making it very little usable. Releasing KDE 4.5 with this bug would give a bad taste to the user making it seem unstable and broken. Should we pay this price just to respect a planned releasing date?

We've notified the release team and the kde-packager mailing list to make sure that the patches find their way into the distributions' 4.5.0 packages.

But there was a recent decision to delay the 4.5.0 release anyway, see

http://mail.kde.org/pipermail/release-team/2010-August/004050.html

I'd like to thank you for delaying KDE 4.5 release for getting fixes for these bugs to 4.5.

I do not have time right now to compile trunk to test the new version, so I cannot confirm if the changes fix the hangs. Can someone that could reproduce them confirm? What if the user still has the unpatched D-Bus version? From just looking at the patch, it only enforces a file size limit.

Are there any patches applied to 4.5 branch that are not in trunk? Because with todays trunk (kdelibs 1160128, kdebase 1160144) I still got the hang twice in a short time. Once when clicking on an image (about 100K in size), once when clicking on a folder. I have to disable information panel again :(

Reenabled the Information Panel after a fresh compilation/installation of trunk (kdelibs r1160350, kdebase r1160534), and the hangs are still there. So far I got them when
- right clicking on a folder
- clicking on a small text file to drag it
- by hovering over a small .diff file

Reopening the bug until someone clarifies if there are fixes in 4.5 branch which are not forward ported to trunk.

Adding blocker keyword, as requested by multiple commenters.

@Christoph: I can confirm that all related fixes to this issue from the 4.5 branch also have been ported to trunk. As mentioned in comment #37 the dbus-related hangs require a recent version of libdbus. Can somebody else still reproduce hangs with a recent trunk version or the tagged 4.5.0 version? I did a lot of tests during the last days and also have run a test program which opens several instances of KFileMetaDataWidget in parallel for verification, but everything runs fine here.

as it seems unconfirmed (comment #52), ignoring the 4.5.0 release blocker. please investigate/fix for 4.5.1

Kde 4.5 (from repository) - Opensuse 11.3
libDBus 1.3.2 (from repository)

In my system, this bug seems resolved.

Thanks

I updated to D-Bus 1.3.2 and it worked without a freeze so far. Either the patches for D-Bus 1.2.24 mentioned in comment #11 were not sufficient alone, or were not really part of the packages that were advertised/created for this purpose. Sorry for the confusion and thanks for all the dolphins.

(On a side note, the official pages say that 1.3.x are unstable D-Bus versions, and users should use 1.2.24 instead. How ironic.)

*** Bug 233838 has been marked as a duplicate of this bug. ***

so people need an unstable dbus version in order to get a working file manager?

*** Bug 249990 has been marked as a duplicate of this bug. ***

*** Bug 250030 has been marked as a duplicate of this bug. ***

*** Bug 250036 has been marked as a duplicate of this bug. ***

*** Bug 250165 has been marked as a duplicate of this bug. ***

*** Bug 250293 has been marked as a duplicate of this bug. ***

*** Bug 250295 has been marked as a duplicate of this bug. ***

*** Bug 252342 has been marked as a duplicate of this bug. ***

I still experience those hangs - even after updating to libdbus-1.4.0 :/

(In reply to comment #65)
> I still experience those hangs - even after updating to libdbus-1.4.0 :/

I am noticed too that dolphin hangs with dbus 1.3.1 but not so often.

Not happened to me though, I'm using 1.4.0.

Actually the dbus bug seems fixed after 1.3.1, so it maybe normal as you are using 1.3.1.

As I said, I still see this bug with dbus-1.4.0!
It is now a lot better than it has been with 1.2.?, but its still annoying to have dolphin/plasma freeze from time to time for ~20s.

Installing libdbus 1.4.0 solved issues with Konqueror for me.
I have not experienced any similar problems with Dolphin as well, though I rarely use it.

Debian testing, KDE 4.5.1 from external repository, libdbus 1.4.0 from Debian's experimental.

Also I have Nepomuk disabled in SystemSettings.