Bug 317676 - Dolphin freezes/hangs when copying large files
Summary: Dolphin freezes/hangs when copying large files
Status: RESOLVED FIXED
Alias: None
Product: kio
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: 4.10.1
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: David Faure
URL:
Keywords:
: 256291 317678 (view as bug list)
Depends on:
Blocks:
 
Reported: 2013-04-01 11:21 UTC by Michael Zanetti
Modified: 2014-03-25 11:49 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In: 4.10.5


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Zanetti 2013-04-01 11:21:10 UTC
Dolphing hangs and is completely unuseable for hours when copying large files. Copying files over network is especially hit hard by this.



Reproducible: Always

Steps to Reproduce:
1. Mount a NFS share
2. Copy a large file to the share
Actual Results:  
Dolphin is unuseable until copying has finished.

Expected Results:  
It should not freeze :)
Comment 1 Frank Reininghaus 2013-04-02 09:09:01 UTC
Thanks for the bug report! To understand what the problem is, we need to know where in the code the freeze occurs. You could help us by running Dolphin in gdb to find out:

1. In a terminal, enter 'gdb dolphin'.
2. At the gdb prompt, enter 'run'.
3. Reproduce the freeze.
4. Switch to the terminal, press Ctrl+Z.
5. At the gdb prompt, enter 'thread apply all backtrace'.

Paste or attach the backtrace here. Thanks for your help.
Comment 2 Michael Zanetti 2013-04-03 23:32:16 UTC
Here you go. I might add that pressing ctrl+z took quite a while to actually suspend the process. When dolphing hangs, it still sometimes reacts to input events. For example, clicking on a non-active tab takes about a minute until the tab is activated but then it hangs again. Like if the input event is queued in the same queue thats actually performing the copy task and it has to wait until the currently queued data chunks are copied before it handles the click. Stopping the process with Ctrl+Z shows the same behavior. 

-------------------- Backtrace ------------------------------

(gdb) thread apply all backtrace

Thread 11 (Thread 0x7fffd9762700 (LWP 8529)):
#0  __GI___nptl_death_event () at events.c:31
#1  0x00007fffef571160 in start_thread (arg=0x7fffd9762700) at pthread_create.c:356
#2  0x00007ffff786be1d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 10 (Thread 0x7fffd9ffb700 (LWP 8519)):
#0  0x00007ffff7864233 in select () at ../sysdeps/unix/syscall-template.S:81
#1  0x00007ffff36e3132 in QProcessManager::run (this=0x7ffff3a62540 <processManager()::processManager>) at io/qprocess_unix.cpp:245
#2  0x00007ffff3607bec in QThreadPrivate::start (arg=0x7ffff3a62540 <processManager()::processManager>) at thread/qthread_unix.cpp:338
#3  0x00007fffef570f8e in start_thread (arg=0x7fffd9ffb700) at pthread_create.c:311
#4  0x00007ffff786be1d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 3 (Thread 0x7fffdb84e700 (LWP 8505)):
#0  0x00007ffff785f3cd in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0x00007fffef09e2bc in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#2  0x00007fffef09e3e4 in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#3  0x00007ffff3733036 in QEventDispatcherGlib::processEvents (this=0x7fffcc0008c0, flags=...) at kernel/qeventdispatcher_glib.cpp:426
#4  0x00007ffff370338f in QEventLoop::processEvents (this=this@entry=0x7fffdb84dd90, flags=...) at kernel/qeventloop.cpp:149
#5  0x00007ffff3703618 in QEventLoop::exec (this=0x7fffdb84dd90, flags=...) at kernel/qeventloop.cpp:204
#6  0x00007ffff3605410 in QThread::exec (this=<optimized out>) at thread/qthread.cpp:542
#7  0x00007ffff36e4edf in QInotifyFileSystemWatcherEngine::run (this=0xa6d650) at io/qfilesystemwatcher_inotify.cpp:256
#8  0x00007ffff3607bec in QThreadPrivate::start (arg=0xa6d650) at thread/qthread_unix.cpp:338
#9  0x00007fffef570f8e in start_thread (arg=0x7fffdb84e700) at pthread_create.c:311
#10 0x00007ffff786be1d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Thread 2 (Thread 0x7fffdc71a700 (LWP 8504)):
---Type <return> to continue, or q <return> to quit---
#0  0x00007ffff785f3cd in poll () at ../sysdeps/unix/syscall-template.S:81                                                                                                                                  
#1  0x00007fffef09e2bc in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0                                                                                                                                 
#2  0x00007fffef09e3e4 in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0                                                                                                           
#3  0x00007ffff3733036 in QEventDispatcherGlib::processEvents (this=0x7fffd40008c0, flags=...) at kernel/qeventdispatcher_glib.cpp:426                                                                      
#4  0x00007ffff370338f in QEventLoop::processEvents (this=this@entry=0x7fffdc719d90, flags=...) at kernel/qeventloop.cpp:149                                                                                
#5  0x00007ffff3703618 in QEventLoop::exec (this=0x7fffdc719d90, flags=...) at kernel/qeventloop.cpp:204                                                                                                    
#6  0x00007ffff3605410 in QThread::exec (this=<optimized out>) at thread/qthread.cpp:542                                                                                                                    
#7  0x00007ffff36e4edf in QInotifyFileSystemWatcherEngine::run (this=0x94d460) at io/qfilesystemwatcher_inotify.cpp:256                                                                                     
#8  0x00007ffff3607bec in QThreadPrivate::start (arg=0x94d460) at thread/qthread_unix.cpp:338                                                                                                               
#9  0x00007fffef570f8e in start_thread (arg=0x7fffdc71a700) at pthread_create.c:311                                                                                                                         
#10 0x00007ffff786be1d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113                                                                                                                         
                                                                                                                                                                                                            
Thread 1 (Thread 0x7ffff7f917c0 (LWP 8450)):                                                                                                                                                                
#0  0x00007ffff785d445 in __GI___xstat (vers=<optimized out>, name=0x10e8028 "/mnt/videos/Movies/Total Recall (2012).avi", buf=0x7fffffffd2b0) at ../sysdeps/unix/sysv/linux/wordsize-64/xstat.c:37         
#1  0x00007ffff5566a44 in stat (__path=<optimized out>, __statbuf=0x7fffffffd2b0) at /usr/include/x86_64-linux-gnu/sys/stat.h:456                                                                           
#2  stat (buf=0x7fffffffd2b0, path=...) at ../../kdecore/util/kde_file.h:209                                                                                                                                
#3  KDirListerCache::slotFileDirty (this=0x951ae0, path=...) at ../../kio/kio/kdirlister.cpp:1082                                                                                                           
#4  0x00007ffff37190ef in QMetaObject::activate (sender=0x9572e0, m=<optimized out>, local_signal_index=<optimized out>, argv=0x7fffffffd480) at kernel/qobject.cpp:3539                                    
#5  0x00007ffff3b66df2 in KDirWatch::dirty (this=<optimized out>, _t1=...) at ./kdirwatch.moc:113                                                                                                           
#6  0x00007ffff371e5be in QObject::event (this=0x9572e0, e=<optimized out>) at kernel/qobject.cpp:1194                                                                                                      
#7  0x00007ffff40ed8ec in QApplicationPrivate::notify_helper (this=this@entry=0x6543d0, receiver=receiver@entry=0x9572e0, e=e@entry=0x126ba80) at kernel/qapplication.cpp:4567                              
#8  0x00007ffff40f025b in QApplication::notify (this=0x7fffffffdec0, receiver=0x9572e0, e=0x126ba80) at kernel/qapplication.cpp:4428                                                                        
#9  0x00007ffff4dfaaf6 in KApplication::notify (this=0x7fffffffdec0, receiver=0x9572e0, event=0x126ba80) at ../../kdeui/kernel/kapplication.cpp:311                                                         
#10 0x00007ffff370463e in QCoreApplication::notifyInternal (this=0x7fffffffdec0, receiver=receiver@entry=0x9572e0, event=event@entry=0x126ba80) at kernel/qcoreapplication.cpp:946                          
#11 0x00007ffff3708171 in sendEvent (event=0x126ba80, receiver=0x9572e0) at ../../include/QtCore/../../src/corelib/kernel/qcoreapplication.h:231                                                            
#12 QCoreApplicationPrivate::sendPostedEvents (receiver=0x0, event_type=0, data=0x612130) at kernel/qcoreapplication.cpp:1570                                                                               
#13 0x00007ffff3732e83 in sendPostedEvents () at ../../include/QtCore/../../src/corelib/kernel/qcoreapplication.h:236                                                                                       
---Type <return> to continue, or q <return> to quit---                                                                                                                                                      
#14 postEventSourceDispatch (s=0x654720) at kernel/qeventdispatcher_glib.cpp:279                                                                                                                            
#15 0x00007fffef09dfe5 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0                                                                                                            
#16 0x00007fffef09e328 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#17 0x00007fffef09e3e4 in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#18 0x00007ffff3733016 in QEventDispatcherGlib::processEvents (this=0x613a10, flags=...) at kernel/qeventdispatcher_glib.cpp:424
#19 0x00007ffff41931ae in QGuiEventDispatcherGlib::processEvents (this=<optimized out>, flags=...) at kernel/qguieventdispatcher_glib.cpp:204
#20 0x00007ffff370338f in QEventLoop::processEvents (this=this@entry=0x7fffffffdd80, flags=...) at kernel/qeventloop.cpp:149
#21 0x00007ffff3703618 in QEventLoop::exec (this=0x7fffffffdd80, flags=...) at kernel/qeventloop.cpp:204
#22 0x00007ffff3708cf6 in QCoreApplication::exec () at kernel/qcoreapplication.cpp:1218
#23 0x00007ffff7b89697 in kdemain () from /usr/lib/kde4/libkdeinit/libkdeinit4_dolphin.so
#24 0x00007ffff7793ea5 in __libc_start_main (main=0x4006d0, argc=1, ubp_av=0x7fffffffdff8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdfe8)
    at libc-start.c:260
#25 0x0000000000400701 in _start ()
(gdb)
Comment 3 Frank Reininghaus 2013-04-04 07:57:54 UTC
Thanks for the quick reply. Looks like it might be a KIO issue (or at least an issue in code called by KIO).

(In reply to comment #2)
> Thread 1 (Thread 0x7ffff7f917c0 (LWP 8450)):                                
> 
> #0  0x00007ffff785d445 in __GI___xstat (vers=<optimized out>, name=0x10e8028
> "/mnt/videos/Movies/Total Recall (2012).avi", buf=0x7fffffffd2b0) at
> ../sysdeps/unix/sysv/linux/wordsize-64/xstat.c:37         
> #1  0x00007ffff5566a44 in stat (__path=<optimized out>,
> __statbuf=0x7fffffffd2b0) at /usr/include/x86_64-linux-gnu/sys/stat.h:456   
> 
> #2  stat (buf=0x7fffffffd2b0, path=...) at ../../kdecore/util/kde_file.h:209
> 
> #3  KDirListerCache::slotFileDirty (this=0x951ae0, path=...) at
> ../../kio/kio/kdirlister.cpp:1082                                           
> 
> #4  0x00007ffff37190ef in QMetaObject::activate (sender=0x9572e0,
> m=<optimized out>, local_signal_index=<optimized out>, argv=0x7fffffffd480)
> at kernel/qobject.cpp:3539                                    
> #5  0x00007ffff3b66df2 in KDirWatch::dirty (this=<optimized out>, _t1=...)
> at ./kdirwatch.moc:113
Comment 4 Dawit Alemayehu 2013-06-02 14:41:37 UTC
The hang seems to be caused by a a call to stat the file ; so the question I have for you is with what options did you mount the nfs share ? For example, did you use any of the options listed in the links below to improve the response of nfs share ?

http://unix.stackexchange.com/questions/31979/stop-broken-nfs-mounts-from-locking-a-directory
http://www.centos.org/docs/5/html/Deployment_Guide-en-US/s1-nfs-client-config-options.html
Comment 5 Michael Zanetti 2013-06-02 17:16:24 UTC
(In reply to comment #4)
> The hang seems to be caused by a a call to stat the file ; so the question I
> have for you is with what options did you mount the nfs share ? For example,
> did you use any of the options listed in the links below to improve the
> response of nfs share ?
> 
> http://unix.stackexchange.com/questions/31979/stop-broken-nfs-mounts-from-
> locking-a-directory
> http://www.centos.org/docs/5/html/Deployment_Guide-en-US/s1-nfs-client-
> config-options.html

No, I haven't. Just mounting with "mount -t nfs 10.10.10.2:/path/to/share /path/to/mountpoint"

Reading through the options in your links this only seems to change anything if the operations cause a timeout (server goes away). That's not the case here. The server is still available and the copy operation ongoing. Just the frontends hang (dolphin + the plasma notification copy dialog).

The only option that I could think of having any impact is the read/write packet sizes. However, I can reproduce this with servers with different rsize/wsize settings, so it doesn't seem to be the culprit.
Comment 6 Dawit Alemayehu 2013-06-05 04:29:52 UTC
Would you be able to apply a patch to kdelibs and recompile if I supply one ? The patch I have stops doing a stat on a resource everytime it is dirty (read: changed) just to determine whether it is a directory or not. Though this would probably help with your issue here, it still won't explain why stat'ing a resource on a NFS mount several times would cause the issue you experience here.  
I just want to see whether it helps or not.
Comment 7 Michael Zanetti 2013-06-05 07:13:15 UTC
(In reply to comment #6)
> Would you be able to apply a patch to kdelibs and recompile if I supply one
> ?

Yes, if it applies against a stable release (i.e. 4.10.2). I probably won't find the time to compile whole kde git master in the near future.

> The patch I have stops doing a stat on a resource everytime it is dirty
> (read: changed) just to determine whether it is a directory or not. Though
> this would probably help with your issue here, it still won't explain why
> stat'ing a resource on a NFS mount several times would cause the issue you
> experience here.  
> I just want to see whether it helps or not.

Sure, hit me with the patch. Happy to help!
Comment 8 Dawit Alemayehu 2013-06-05 12:34:38 UTC
(In reply to comment #7)
> (In reply to comment #6)
> > Would you be able to apply a patch to kdelibs and recompile if I supply one
> > ?
> 
> Yes, if it applies against a stable release (i.e. 4.10.2). I probably won't
> find the time to compile whole kde git master in the near future.

You only need to compile and install kdelibs. Well technically you would only need to compile and install kdelibs/kio, but the whole kdelibs would be fine as well. And yes it would most certainly work against the stable 4.10.x releases.

> > The patch I have stops doing a stat on a resource everytime it is dirty
> > (read: changed) just to determine whether it is a directory or not. Though
> > this would probably help with your issue here, it still won't explain why
> > stat'ing a resource on a NFS mount several times would cause the issue you
> > experience here.  
> > I just want to see whether it helps or not.
> 
> Sure, hit me with the patch. Happy to help!

You can download the patch from https://git.reviewboard.kde.org/r/110834/. Simply right click on "Download Diff" and select "Save Link As...".
Comment 9 Michael Zanetti 2013-06-06 11:58:14 UTC
Thumbs up dude! Copying files from NFS smoothly now! Thanks a lot!
Comment 10 Michael Zanetti 2013-06-06 12:11:29 UTC
Btw. Tested on KDE 4.10.3. Patch applied without hickups.
Comment 11 Dawit Alemayehu 2013-06-06 12:42:43 UTC
(In reply to comment #9)
> Thumbs up dude! Copying files from NFS smoothly now! Thanks a lot!

You welcome and thank you for testing the patch.
Comment 12 Dawit Alemayehu 2013-06-10 19:57:16 UTC
Git commit bf2bb4861fdafc8228097060552effe4f9c16f6e by Dawit Alemayehu.
Committed on 03/06/2013 at 05:02.
Pushed by adawit into branch 'KDE/4.10'.

Avoid doing a stat on the same resource just to determine whether or not it is a directory.
FIXED-IN: 4.10.5
REVIEW: 110834

M  +15   -5    kio/kio/kdirlister.cpp

http://commits.kde.org/kdelibs/bf2bb4861fdafc8228097060552effe4f9c16f6e
Comment 13 Michael Zanetti 2013-06-12 07:40:21 UTC
*** Bug 317678 has been marked as a duplicate of this bug. ***
Comment 14 Dawit Alemayehu 2014-03-25 11:49:55 UTC
*** Bug 256291 has been marked as a duplicate of this bug. ***