Bug 165365 - Dataloss: konquerer cannot move files, so it deletes them
Summary: Dataloss: konquerer cannot move files, so it deletes them
Status: RESOLVED WORKSFORME
Alias: None
Product: konqueror
Classification: Applications
Component: general (show other bugs)
Version: 4.3.4
Platform: Ubuntu Linux
: NOR critical
Target Milestone: ---
Assignee: Konqueror Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-06-29 23:34 UTC by Dotan Cohen
Modified: 2012-09-13 14:15 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dotan Cohen 2008-06-29 23:34:51 UTC
Version:            (using KDE 3.5.9)
Installed from:    Ubuntu Packages

When moving files from local (ext3) hard disk to a USB pen drive, Konqueror decided after moving one file that it cannot move the other 30. However, those 30 files were erased from the local disk, even though they were not written to the pen drive! I have one file on the pen drive (which has over 300 MB of free space, and these were 2MB jpeg files), and the rest of the files are lost. They are not even in the Trash.

The error message that I got when trying to move the files was (translated from Hebrew) "Cannot write /media/disk/Photos/2343.jpg" and I had the option to Skip, Skip all, or Cancel. I clicked Skip all, and then discovered that all the original files are gone (and not in the Trash).

This is a major dataloss issue. I found this similar but unrelated bug from 2001:
http://bugs.kde.org/show_bug.cgi?id=23590
Comment 1 FiNeX 2008-06-30 14:09:58 UTC
I cannot reproduce using KDE3.5.9. I'll explain you what I've done.
- I've used a usb drive with 128Mb.
- I've copied about 120Mb on it
- I've selected from my disk some files which dimension is greater than 8Mb (about 40Mb)
- I've moved (Cut-paste) this files from my disk to the external USB drive

after the first file (which was less than 8Mb) konqueror inform me that the space is exausted, so the process stop, but the other files are not erased.

Did i do something wrong trying to reproduce your problem?

Thanks a lot Dotan!

Comment 2 Dotan Cohen 2008-06-30 15:16:09 UTC
The error you got was due to limited disk space. I do not know what caused the error that I got, as it was neither disk space nor permissions, nor did the message say anything other than "Cannot write /media/disk/Photos/2343.jpg" (well, it said it in Hebrew, but nothing more).

In any case, the deletion of the original files should be done only _after_ the copy is completed and returned a 0.
Comment 3 Dotan Cohen 2008-08-20 13:13:50 UTC
The problem occurred again today.

Local system: KDE 3.5.9 on Kubuntu 8.04
Remote system: KDE 3.5.9 on Kubuntu 8.04

I opened two Konqueror tabs in the same window:
1) local directory
2) remote directory accessed via fish protocol

In the local directory is a directory named "python" which has two sub directories: "screenshots" and "code". Altogether there is about 17 MB in "python". I right-click on "python" and select Cut. I then switch tabs and right-click in an open area in the remote tab. I select Paste. A progress window popped up. At about 35% it gets stuck, and the progress meter stops moving. I press Cancel.

Now, in "python" the "code" directory is missing.
Comment 4 Richard Hartmann 2008-11-02 09:56:32 UTC
This has been filed for 3.5.9. Can anyone check if the _only_ possible way to arrive at the delete function while moving is if _everything_ before went right? In that case, this could probably be closed.

Dohan, Finex: Can you try to reproduce this on 4.1.2?
Comment 5 Dotan Cohen 2008-11-02 16:31:53 UTC
I happen to know that Konqueror deletes each file after each individual transfer. That behaviour is fine in my opinion. However, I do not know what problem led to the dataloss I reported. I have had the issue happen more than once.

This is difficult to triage because I do not yet know what makes the system decide that it suddenly cannot write to the USB drive. Simply having the drive full does not trigger the bug. I have some KDE 4 systems now, so if the issue crops up there I will post whatever details that I can.
Comment 6 Richard Hartmann 2008-11-02 16:57:40 UTC
Would it be acceptable to introduce a hidden config option that will log what goes on in these functions? I know this is extremely hackish, but in this case, it might be worth it.
Comment 7 Dotan Cohen 2008-11-02 17:25:16 UTC
> Would it be acceptable to introduce a hidden config option that
> will log what goes on in these functions?

I think that it would be preferable.
Comment 8 FiNeX 2008-11-02 23:04:35 UTC
Hi, the "usb key" test goes fine to me (kde 4.1.2).
Comment 9 Richard Hartmann 2008-11-03 00:08:59 UTC
What do you mean by "usb key"? If you mean thumb drive/usb stick/usb mass storage, did this problem _always_ occur on KDE 3?
Comment 10 Dotan Cohen 2008-11-03 08:16:44 UTC
The problem is intermittent. Usually I can copy to USB mass storage devices just fine. Sometimes I get an error along the lines of "Cannot write /media/disk/file.txt" and the copy of file.txt that is on the computer is erased, but no copy is written to the USB device.

I think that this happened while copying over SSH once, too, but I am not sure. Note that the problem is _not_ disk space! The devices in question always have enough disk space, and more often than not a few files are copied successfully before a file fails and throws this error.
Comment 11 FiNeX 2008-11-03 10:37:11 UTC
@Richard: sorry, in my native language we evan call "chiave usb" the usb external drive. I've literally tranlated it to "usb key" but it is not right to call in that way :-)

Anyway, both in KDE3 and 4, I've moved some files to an external usb drive which didn't have enough free space. I simply was informed about this with a message box, but no file was erased.

At this point, which are the kernel/hal/dbus/filesystem where the bug has been reproduced?
Comment 12 FiNeX 2009-08-19 00:45:20 UTC
I'm still not able to reproduce this data loss. If someone is able to reproduce it, it could be very useful a detailed step-by-step explanation for reproduce it and:
- KDE version used
- filesystem used
- version of: kernel / dbus / hal

Many thanks
Comment 13 Dotan Cohen 2009-08-20 12:49:29 UTC
I will spend some time next week transferring files and trying to reproduce. As an intermittent issue it is hard to reproduce and I have learned to copy-then-delete-original when using Konqueror, as opposed to cut and paste.
Comment 14 Dotan Cohen 2010-01-12 23:16:13 UTC
It happened again while transferring some 10+ GiB over the LAN.

KDE 4.3.4 on both ends.
Transferring from a JFS local system to a JFS remote file system via fish in Dolphin.

dotancohen@dcl:~$ uname -r
2.6.31-17-generic         
dotancohen@dcl:~$ hald --version
HAL package version: 0.5.13     
dotancohen@dcl:~$ qdbus --version
:1.1                             
 org.kde.klauncher               
:1.10                            
 org.kde.knotify                 
:1.12                            
 org.freedesktop.Notifications   
 org.kde.JobViewServer           
 org.kde.NotificationHost-1503   
 org.kde.plasma-desktop          
:1.127                           
:1.128                           
:1.14                            
 org.kde.lancelot                
:1.185                           
 org.kde.dolphin                 
:1.19                            
 org.kde.kxkb                    
:1.2                             
 org.freedesktop.PowerManagement 
 org.freedesktop.PowerManagement.Inhibit
 org.kde.Kephal                         
 org.kde.NotificationItemWatcher        
 org.kde.kded                           
 org.kde.kpasswdserver                  
 org.kde.powerdevil                     
:1.21                                   
 org.freedesktop.ScreenSaver            
 org.kde.krunner                        
 org.kde.screensaver                    
:1.23                                   
 org.kde.NepomukServer                  
:1.28                                   
 org.kde.kmix                           
:1.30                                   
 org.kde.kbluetooth                     
:1.300                                  
:1.301                                  
:1.302                                  
:1.32                                   
 org.kde.NotificationItem-1536-1        
:1.33                                   
 org.kde.NotificationItem-1534-1        
:1.37                                   
 org.kde.knetworkmanager                
 org.kde.networkmanagement              
:1.40                                   
 org.kde.printer-applet-1553            
:1.44                                   
 net.update-notifier-kde-1550           
:1.45                                   
 org.kde.kwalletd                       
:1.47                                   
 org.kubuntu.restrictedInstall
:1.48
 org.kde.konqueror-1576
:1.49
:1.5
 org.kde.kglobalaccel
:1.50
 org.openobex
:1.51
 org.kde.NotificationItem-1558-1
:1.62
 org.gtk.vfs.Daemon
:1.637
 org.kde.konsole
:1.64
:1.640
:1.65
 org.gtk.Private.GduVolumeMonitor
:1.66
 org.gtk.Private.GPhoto2VolumeMonitor
:1.67
:1.68
 org.gnome.GConf
:1.8
 org.kde.ksmserver
 org.kde.ksmserver-1496
:1.9
 org.kde.kwin
 org.kde.kwin-1498
org.freedesktop.DBus
Comment 15 Dotan Cohen 2010-01-14 14:24:46 UTC
There seems to be many ways to trigger this bug. One way is to move a file with the : character to a vfat-formatted USB disk. The file is not copied to the disk, as the filename is invalid. However, Konqi/Dolphin delete the "moved" file anyway.

There are other ways to trigger this bug, as in my previous comment there was no vfat-formatted disk involved.
Comment 16 Dotan Cohen 2010-10-14 00:28:04 UTC
I found another way to trigger this bug. If the USB device is removed or shut down in the middle of the process, then the files that were not copied over get deleted even though they have not been copied.
Comment 17 FiNeX 2010-10-14 01:50:53 UTC
I don't want to try this at home :-)
Comment 18 Richard Hartmann 2010-10-14 04:02:24 UTC
Setting to critical as per https://bugs.kde.org/page.cgi?id=fields.html#bug_severity
Comment 19 Dotan Cohen 2010-10-26 00:11:44 UTC
> I don't want to try this at home :-)

I have been trying this at home (and loosing files) for two and a half years.

To recap:
The KDE file manager delete a file if it is meant to be moved (cut->paste) to another location and there is an unexpected no-write condition. Two known conditions which trigger the bug:
1) Trying to move a file that contains the ":" character to a VFAT or NTFS filesystem.
2) Removing a USB drive before the write has completed. (also occurs if the device's battery fails)

What does _not_ trigger the bug: filling up the filesystem.
Comment 20 Dotan Cohen 2011-01-25 21:26:57 UTC
This is getting out of hand. I just now tried to move (Cut->Paste) a small (under 100 KiB of text files) directory from a USB flash drive to my /home/user directory. As soon as the transfer started the USB flash drive's icon in Places turned into the "disconnected" icon. I clicked the Places entry to reconnect, and the directory was missing from the USB flash drive. However, it was not written to the disk!

My initial inclination is that the USB flash drive may be defective, however I have had this issue with at least three or four drives and at least three different computers over the years with KDE, and only in KDE. Windows and Gnome do not suffer this. Some of the conditions are reproducible (see previous comment) and some I have yet to decipher.
Comment 21 Dawit Alemayehu 2011-03-24 14:51:42 UTC
(In reply to comment #20)
> This is getting out of hand. I just now tried to move (Cut->Paste) a small
> (under 100 KiB of text files) directory from a USB flash drive to my /home/user
> directory. As soon as the transfer started the USB flash drive's icon in Places
> turned into the "disconnected" icon. I clicked the Places entry to reconnect,
> and the directory was missing from the USB flash drive. However, it was not
> written to the disk!
> 
> My initial inclination is that the USB flash drive may be defective, however I
> have had this issue with at least three or four drives and at least three
> different computers over the years with KDE, and only in KDE. Windows and Gnome
> do not suffer this. Some of the conditions are reproducible (see previous
> comment) and some I have yet to decipher.

Out of curiosity I tried one of the following scenarios to see if I could duplicate your problem:

1.) Create a folder that contains 4 text files with one of them having ':' in its name.
2.) Attempt to move (both by DND and Cut/Paste) the file to a VFAT usb-stick.
3.) Attempt to move the directory that contains the 4 files to a VFAT usb-stick.

In case of step #2, I get a very cryptic error message "could not write <destination-filename>" and the original file is still intact in the source directory.

In case of step #3, the directory and the valid filenames are created on the usb-stick, but the directory with the non-valid filename is kept on the source hard drive.

Though the error messages are rather cryptic and can be improved, I did not see or get any data loss.
Comment 22 Myriam Schweingruber 2012-09-13 13:56:04 UTC
I can't reproduce this at all with KDE 4.9.1 closing for now since there was no more feedback since 2011
Comment 23 Dotan Cohen 2012-09-13 14:15:59 UTC
Thanks, Myriam. I also cannot reproduce.

For future reference, this is how I test reproduction:

1) By filename:
$ echo 'hello' > hello.txt 
$ echo 'little' > li:ttle.txt 
$ echo 'world' > world.txt 

Then, using Dolphin Cut and Paste the files to a USB thumb drive.

2) By disconnecting USB device:
$ dd if=/dev/zero of=hello.txt bs=1000 count=1
$ dd if=/dev/zero of=little.txt bs=1000000000 count=1
$ dd if=/dev/zero of=world.txt bs=1000 count=1

Then, using Dolphin Cut and Paste the files to a USB thumb drive and immediately pull the drive out.