Bug 186182 - dolphin does not copy all files (cp does)
Summary: dolphin does not copy all files (cp does)
Status: RESOLVED WAITINGFORINFO
Alias: None
Product: kio
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: unspecified
Platform: Gentoo Packages Linux
: NOR normal
Target Milestone: ---
Assignee: David Faure
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-03-05 08:24 UTC by Trenton D. Adams
Modified: 2013-06-16 17:30 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
tarball of bug related files (117.89 KB, application/octet-stream)
2009-03-05 08:26 UTC, Trenton D. Adams
Details
kde-libs.debug.gz (4.29 KB, application/x-gzip)
2009-03-19 07:03 UTC, Trenton D. Adams
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Trenton D. Adams 2009-03-05 08:24:42 UTC
Version:            (using KDE 4.2.0)
Compiler:          sys-devel/gcc-4.1.2 
OS:                Linux
Installed from:    Gentoo Packages

Dolphin fails to copy all of my files as requested.  It tells me that some files do not exist anymore, and then fails to copy them, though they do exist.  Using "cp" from the command line works just fine.  I do it as the same user as dolphin.

Source folder      : /timecapsule/backup/pre-linux/Pictures2
Destination folders: ~/photos/Pictures-{good,bad}-copy

The time capsule is mount as a cifs mount.

For example, I have a file at /timecapsule/backup/pre-linux/Pictures2/media/2006/family/2006-11-30-17.42.36.jpg.  Dolphin claims that it cannot copy file /timecapsule/backup/pre-linux/Pictures2/media/2007/family/2006-11-30-17.42.36.jpg because it does not exist.  Well, of course that file doesn't exist, because I don't put 2006 files in the 2007 folder.  The file it claims it is trying to copy is in the 2006 folder, not the 2007 folder.  It does this on 901 files out of 7306..

I have generated three lists of files.  They are
1. The original list from the copy source directory.  it is called pictures-original-copy.list
2. The good copy using "cp" from the commandline.  It is called pictures-good-copy.list
3. The bad copy, using dolphin.  It is called pictures-bad-copy.list

Now see the output below...

trenta@tdamac ~/photos $ wc -l pictures-original-copy.list
7306 pictures-original-copy.list
trenta@tdamac ~/photos $ wc -l pictures-good-copy.list
7306 pictures-good-copy.list
trenta@tdamac ~/photos $ wc -l pictures-bad-copy.list
6405 pictures-bad-copy.list


These lists were generated using "find ./ | sort > ~/photos/pictures-original-copy.list" for easy diffing.  I will attach them when given the option in the wizard, or after the bug is created.  I will also attach a screen shot.
Comment 1 Trenton D. Adams 2009-03-05 08:26:59 UTC
Created attachment 31785 [details]
tarball of bug related files

bug-files.tar.gz
pictures-bad-copy.list
pictures-good-copy.list
pictures-original-copy.list
Comment 2 Trenton D. Adams 2009-03-05 08:29:45 UTC
Oh, one other thing, I picked the "auto skip" option, because it happened 5 or 6 times, so I figured I didn't want to click through all of them (not knowing how many there were).
Comment 3 Trenton D. Adams 2009-03-13 06:44:16 UTC
I'm just curious.  It's been several days, and I don't see anything happening with it.  I attempted this copying several times, and it happened each and every time.  It is totally reproducable.  I have no doubt that if you do it with a directory of a large number of files, that you can reproduce it.

I'm just concerned that this won't be fixed.  I really like the UI for Dolphin, and would love to be able to trust it.

It seems to be a issue of keeping a proper file list in memory.  i.e. the list management is messing up which folders files are actually in.  If you could could tell me where in the source this is located, I may be able to take a quick look as well.  Although I haven't programmed in C/C++ in several years, I may see something obvious, being a new pair of eyes and all.

Is there something else I can do to get more data for this?
Comment 4 David Faure 2009-03-13 11:22:33 UTC
I know it's been days - bugs are piling up like crazy, especially due to regressions in Qt-4.5.

If I had to guess I would suspect the handling of the directory listing in kdelibs/kio/kio/copyjob.cpp, CopyJobPrivate::slotEntries. But that code hasn't changed for quite some time, so it's a bit surprising. Maybe you can check what the debug output says (after compiling kdelibs in debug mode). A reduction to the smallest testcase that shows the bug would be useful too.

I'll come back to this bug soon hopefully...
Comment 5 Trenton D. Adams 2009-03-13 16:46:42 UTC
Thanks for the update David.  I totally understand the bugs piling up thing.  Seems like that's happening to me at work too. lol  I won't worry too much from here on out.

I'm going to see if I can find some time this weekend to take a quick look, try debug, reduce to small test case with a tar of the files I used, etc.

Thanks.
Comment 6 Trenton D. Adams 2009-03-15 19:49:23 UTC
This does not happen, with the same files, if copied from one local folder to another local folder.  It does happen consistently however, when copied from my apple time capsule over CIFS.  I am going to setup samba, and share them out from a local folder using CIFS, and then copy them to another local folder.  We'll see what happens then.

It may also be a latency related bug.

Or a threading issue?  Do you do multiple threads when copying?

I'm going to see if I can duplicate it with the same folder/file structure, but with empty files.  That way my testing will be quicker each time, and we'll know if it's related to the nature of the data in any way.
Comment 7 David Faure 2009-03-16 15:36:13 UTC
No threads involved (but a separate process, the kioslave).

What does "when copied from my apple time capsule over CIFS" mean technically?
A smb:// URL, or smbmount, or something else?

If smb:// URL, then this sounds like a kio_smb bug.
Comment 8 Trenton D. Adams 2009-03-19 03:14:58 UTC
Okay, I have ZERO idea how to get debug output.  I enabled debug mode, by putting the gentoo "debug" use flag on, for dolphin, but I'm not sure where the debug info goes to.  I ran it from the console, and I see no output.
Comment 9 Trenton D. Adams 2009-03-19 03:23:22 UTC
Oh, I didn't compile kdelibs in debug mode, I compiled dolphin in debug mode.  Either way, if there's something special I need to do, I'd like to know.
Comment 10 Trenton D. Adams 2009-03-19 07:03:43 UTC
Created attachment 32253 [details]
kde-libs.debug.gz

Okay, I did debug mode for kdelibs.  It doesn't really offer much useful information, other than what I gave you already...
Comment 11 David Faure 2009-03-31 18:01:45 UTC
Indeed, more debug is needed. Can you uncomment all the //kDebug lines in CopyJobPrivate::slotEntries? (kdelibs/kio/kio/copyjob.cpp between lines 496-590). (Remove the "//", save, and then go to kdelibs/kio and do "make && make install").
Very puzzling bug.
Comment 12 Dawit Alemayehu 2013-06-16 17:30:00 UTC
No reply to comment #11. Feel free to reopen if this is still an issue in the latest stable version of KDE.