Bug 320890 - missing files when copying folders with files
Summary: missing files when copying folders with files
Status: RESOLVED DUPLICATE of bug 162211
Alias: None
Product: kio
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: unspecified
Platform: Arch Linux Linux
: NOR critical
Target Milestone: ---
Assignee: David Faure
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-06-08 11:27 UTC by Alex
Modified: 2014-05-20 11:55 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
diff -rq (94.97 KB, application/zip)
2013-08-24 14:41 UTC, Alex
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alex 2013-06-08 11:27:49 UTC
there is a folder downloaded to a computer via SMB. ~ 11,000 files
copy this folder to the next.  some of the files are not copied. No error message is issued.
Midnight Commander, Total Commander normal copy this folder.
a copy of the made in Total Commander or Midnight Commander, normally copied and Krusader.

the output of commands find, ls --> http://rghost.ru/46600967  
find__0_original.txt     ls__original.txt - source folder.
find__copy_total_commander_ok.txt - After Copying in Total Commander.
others after copying in Krusader.

archlinux 3.9.4-1-ARCH #1 SMP upd: 25 may. FS- ext4 (no errors). Fluxbox.

sorry for my english.

Reproducible: Sometimes

Steps to Reproduce:
repeated with a specific folder
Comment 1 Jan Lepper 2013-06-19 14:22:10 UTC
Can you reproduce this too in Dolphin or Konqueror ?
Comment 2 Alex 2013-06-19 21:27:58 UTC
>>Can you reproduce this too in Dolphin or Konqueror ?
yes.

my tests:
1. reboot OS.
2. run krusader.
3. copy folder. copied - 833,5 MB. bad
4. copy folder. copied - 856,3 MB. bad
5. copy folder. copied - 890,8 MB. bad
6. restart krusader
7. copy folder. copied - 952,4 MB OK!
8. copy folder. copied - 952,4 MB OK!

9. reboot OS.
10. run Dolphin.
11. copy folder. copied - 832,4 MB. bad
12. copy folder. copied - 854,1 MB. bad
13. restart Dolphin.
14. copy folder. copied - 882,7 MB. bad
15. restart Dolphin.
16. copy folder. copied - 914,9 MB. bad
17. run Konqueror.
18. copy folder. copied - 952,4 MB OK!

19. reboot OS.
20. run Konqueror.
21. copy folder. copied - 833,3 MB. bad

after the operating system. when you first start out krusader xterm writes:
$ krusader 
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 14: reading configurations from ~/.fonts.conf is deprecated.
Connecting to deprecated signal QDBusConnectionInterface::serviceOwnerChanged(QString,QString,QString)
QDBusConnection: session D-Bus connection created before QCoreApplication. Application may misbehave.
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 14: reading configurations from ~/.fonts.conf is deprecated.
QDBusConnection: session D-Bus connection created before QCoreApplication. Application may misbehave.
kbuildsycoca4 running...
QDBusConnection: session D-Bus connection created before QCoreApplication. Application may misbehave.
QDBusConnection: session D-Bus connection created before QCoreApplication. Application may misbehave.
Fontconfig warning: "/etc/fonts/conf.d/50-user.conf", line 14: reading configurations from ~/.fonts.conf is deprecated.
QDBusConnection: name 'org.kde.kglobalaccel' had owner '' but we thought it was ':1.5'
QDBusConnection: name 'org.freedesktop.UDisks2' had owner '' but we thought it was ':1.10'

Is this normal?
Comment 3 Jan Lepper 2013-07-21 10:37:33 UTC
I'm sorry for the delay.
Since this also happens with Dolphin and Konqueror it is most likely a bug KIO bug.
Moving to kio.
Comment 4 Dawit Alemayehu 2013-08-17 13:53:19 UTC
Can you use the command line tool diff to show you the differences between the original and copied folders: diff -rq folder1 folder2. This would tell you which files were not being copied and might gives a hint as to where the problem is. If running this command does not show any differences, then there was no problem with the copying and the issue is elsewhere. For example, whatever you were using to check the size of the folders might be showing incorrect information.
Comment 5 Alex 2013-08-24 14:41:20 UTC
Created attachment 81892 [details]
diff -rq

Tested in different ways.

tests 2013-08-24:
krusader launched
1. copy folder using Krusader. copied - 834 MB. bad
2. diff -rq ./telemaster.ru/ ./test11/telemaster.ru/ > test11_krus.txt

3. reboot OS.
4. copy folder using Krusader. copied - 833 MB. bad
5. diff -rq ./telemaster.ru/ ./test12/telemaster.ru/ > test12_krus.txt
6. copy folder using Krusader. copied - 952 MB. OK

7. reboot OS.
8. copy folder using  Midnight Commander. copied - 952 MB. OK
9. copy folder using  Krusader. copied - 952 MB. OK

$ df -h /dev/sda1
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1       1.8T  1.5T  375G  80% /mnt/6__DATA-2
Comment 6 Alex 2013-08-24 14:54:28 UTC
I found two other folders (backups) with the lost files. 
Backups made 1-3 years ago. 
Backups contain more than 8,000 files.
I think the problem only occurs with many files
Comment 7 Dawit Alemayehu 2013-08-25 15:46:46 UTC
(In reply to comment #6)
> I found two other folders (backups) with the lost files. 
> Backups made 1-3 years ago. 
> Backups contain more than 8,000 files.
> I think the problem only occurs with many files

Well, I copied the entire /usr/share/doc/qt/html folder which has over 8000 files in it multiple times using both Dolphin and Konqueror, but I cannot reproduce the problem at all. Did not matter if I restarted the application or rebooted the system. The copying always succeeds. This is really baffling to me and I have no explanation for it because I cannot 

> 7. reboot OS.
> 8. copy folder using  Midnight Commander. copied - 952 MB. OK
> 9. copy folder using  Krusader. copied - 952 MB. OK

Does copying with Midnight Commander always succeed? IOW, you never see the problem when using Midnight Commander ?
 
> $ df -h /dev/sda1
> Filesystem      Size  Used Avail Use% Mounted on
> /dev/sda1       1.8T  1.5T  375G  80% /mnt/6__DATA-2

Hmm... now I am a bit unsure how exactly you were testing the copying so whatever I did to reproduce might not be applicable to your circumstance. The copying test I did above is from one ext4 partition to another. 

To me the partition you listed above seems to be a mount of either a networked parition through nfs/smb(cifs) or it is a mount of another partition on the same machine. So the question I have now is what exactly is the type of the parition you are showing us above? How is it mounted (options used etc)? When you did the copying tests above were you copying from and/or to this partition?
Comment 8 Alex 2013-09-08 23:33:43 UTC
I tested a lot of ...
Again checked the FS. No errors.
Problems with Midnight Commander is not noticed. Always copies quickly and completely.
Files copied from another machine (winxp + total commander).

>>To me the partition you listed above seems to be a mount of either a networked parition through nfs/smb(cifs) or it is a mount of another partition on the same machine. So the question I have now is what exactly is the type of the parition you are showing us above? How is it mounted (options used etc)? When you did the copying tests above were you copying from and/or to this partition?
Do not understand the question ((.

$ cat /etc/fstab | grep DATA-2
UUID=0f4927c7-35b5-4761-8ffa-aa5868c7808f       /mnt/6__DATA-2   ext4   rw,nosuid,nodev,exec,auto,nouser,async,barrier=1,atime,errors=remount-ro   0 2

$ mount | grep DATA-2
/dev/sda1 on /mnt/6__DATA-2 type ext4 (rw,nosuid,nodev,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)

Reproduced the problem on Virtual Box! Image 6GB  http://imsnet.ru/temp/Arch_Linux_x64.zip
And recorded video http://imsnet.ru/temp/bug-video1.mkv
I hope it will help you.
Comment 9 Dawit Alemayehu 2013-09-09 05:17:00 UTC
Hmm... then all I can say is I am unable to reproduce your problem. ArchLinux is the distro I use as well. I tried copying extremely large directories, like the webkit source code directory, from one ext4 partition to another and everything was copied just fine. BTW, what version of KDE or kdelibs are you running?
Comment 10 Alex 2013-09-09 05:51:30 UTC
All software is the latest version from the repository.
I do not use KDE. Only Krusader. + Slim + Fluxbox + Samba
Comment 11 Oleg 2014-02-27 17:20:21 UTC
I am having this issue when copying a few folders from my ext3 home directory into USB flash drive or microSD (both Fat32) with Dolphin. 
I'm using Kubuntu 12.04 (all software up to date).

I saw some forum threads as old as year 2010 about the same problem...
Comment 12 Dawit Alemayehu 2014-05-20 11:55:16 UTC

*** This bug has been marked as a duplicate of bug 162211 ***