Bug 55804 - Wrong timestamp on files copied to fish
Summary: Wrong timestamp on files copied to fish
Status: CONFIRMED
Alias: None
Product: kio-extras
Classification: Frameworks and Libraries
Component: FISH (show other bugs)
Version: unspecified
Platform: Compiled Sources All
: NOR major
Target Milestone: ---
Assignee: Thorsten Staerk
URL:
Keywords:
: 56213 218133 321010 (view as bug list)
Depends on:
Blocks:
 
Reported: 2003-03-10 18:18 UTC by mi+kde
Modified: 2022-03-11 11:07 UTC (History)
21 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description mi+kde 2003-03-10 18:18:50 UTC
Version:           
Comment 1 groot 2004-01-01 20:55:27 UTC
Not FreeBSD specific.
Comment 2 Nicolas Goutte 2005-09-17 11:00:41 UTC
See also bug #58950 (for tar KIO slave)
Comment 3 Nicolas Goutte 2005-09-18 02:57:30 UTC
See also bug #89377 (for fish KIO slave)
Comment 4 Nicolas Goutte 2005-09-18 03:00:27 UTC
See also bug #24443 (remove/copy in Konqy, probably a duplicate)
Comment 5 Nicolas Goutte 2005-09-18 03:19:04 UTC
See also bug #112375 (media KIO slave)
Comment 6 David Faure 2006-02-27 13:12:00 UTC
#24443 is fixed, i.e. mtime is preserved for local files and directories.
For this bug, let's not forget to preserve mtime with other protocols for both files and dirs (can't be done before KDE4).
Comment 7 Marco Cimmino 2006-11-28 02:13:31 UTC
Any news?
Comment 8 Will Stephenson 2008-03-16 17:31:57 UTC
Ping.  I'm going through the fish:// bugs and http://bugs.kde.org/show_bug.cgi?id=56213#c11 from David references this one.  Can we fix this now or is it deferred til KDE5?
Comment 9 David Faure 2008-03-17 13:40:09 UTC
For all slaves that implement ::copy(), like FTP or FISH, they have to preserve timestamps on their own using something like

            KDE_struct_stat dest_statbuf;
            if (KDE_stat( _dest_orig.data(), &dest_statbuf ) == 0) {
                struct utimbuf utbuf;
                utbuf.actime = dest_statbuf.st_atime; // access time, unchanged
                utbuf.modtime = dt.toTime_t(); // modification time
                utime( _dest_orig.data(), &utbuf );
            }
(as used unconditionally in kio_file, so it compiles on Windows too)

For other slaves it should work already in KDE 4.
Comment 10 Dotan Cohen 2008-11-10 22:26:54 UTC
I can confirm that in Konqueror in KDE 4.1, when files are copied via ssh from a remote host to local, the file's date is the date that the original file was created on the remote host. However, if the file is then copied from local to a different remote host, the new file's date is today.

In instances where file creation times are critical this is a dataloss bug, and therefore very severe.
Comment 11 Dotan Cohen 2008-12-04 22:42:51 UTC
*** Bug 56213 has been marked as a duplicate of this bug. ***
Comment 12 Dotan Cohen 2009-02-08 19:02:08 UTC
*** Bug 56213 has been marked as a duplicate of this bug. ***
Comment 13 Thorsten Staerk 2009-02-08 19:17:53 UTC
I agree that this is severe. If I MOVE (not copy) files from a fish:// location to local, the time data gets lost as well.
Comment 14 Dotan Cohen 2009-02-11 08:09:40 UTC
I just tested Konquerer 3.5.10, Dolphin 1.2 (KDE 4.2), and the system's cp/mv commands to determine behaviour under different copy/move circumstances (local->local, local->remote, remote->local). I found that Dolphin and Konqueror both behaved identically. However, neither of them behaved consistently from circumstance to circumstance, and none of them were consistent with the system's cp/mv behaviour!

I made a monospace table here but it looks terrible, so please see the HTML table at this page to see the results of my test:
http://dotancohen.com/eng/bugs_kde_cut_copy_paste.html

As there is no consistency between Konqi/Dolphin's own behaviour, nor between the behaviour of the cp/mv commands, I feel that this bug should be addressed quickly. File timestamps mean nothing on KDE systems so long as this bug is open.
Comment 15 David Faure 2009-02-11 11:36:52 UTC
The goal of KIO is to emulate cp -a, not cp.
Thanks for the rest of the analysis -- but please be more precise when you say "remote". The implementations of fish, ftp, sftp, and smb are all different and might all need fixing for this (well assuming that they support changing a file's timestamp, which IIRC at least FTP doesn't support).
Comment 16 Dotan Cohen 2009-02-11 11:48:07 UTC
Thank you. By "remote" I mean SSH via Konqueror's fish:// pseudoprotocol.

When I have a chance I will test the other protocols and update the table. I will then report back here. If there is anything else related that you would like me to triage, just ask.
Comment 17 Thorsten Staerk 2009-02-14 14:11:38 UTC
When I move 
fish://root@localhost/blah
to 
/tmp/blah
the timestamp gets changed.

When I move
fish://root@localhost/blah
to
fish://root@localhost/tmp/blah
the timestamp does not get changed.

This indicates that the file:// kioslave is the instance that forgets to set the timestamp in case the file comes from a fish:// location.
Comment 18 Thorsten Staerk 2009-02-28 19:41:07 UTC
There is a comment in the code:

// TODO set modification time on url.path() somehow
// see FileProtocol::put if using utime() to do that.

see kdebase/runtime/kioslave/fish/fish.cpp
Comment 19 Dotan Cohen 2009-04-26 23:10:21 UTC
The sftp put command can have the -P flag used to preserve timestamps and permissions. Furthermore, bug 56213 states that the command MDTM (or MTDM?) is supposed to preserve timestamps as well.

I am updating the title of this bug to better describe the issue. Furthermore, I am changing this from wishlist to bug, because that is what this is.

For those who need convincing that this is a bug, read on. Others stop here.

1) Dataloss. I have voice recordings of my daughter as a baby, and because they were moved with KDE I have lost the dates of those recordings. Some of them are very sensitive to her age, and not knowing her age at the time of recording makes the recording worthless. Any kid will tell you that water is made of hydrogen and oxygen. Hearing a 16 month old say that is precious. Now, I don't remember and don't have the date to be sure any more.

2) See comment #17 and comment #18
Comment 20 Dotan Cohen 2009-12-02 19:31:27 UTC
KDE 4.3.2: Both Konqueror and Dolphin set the timestamp as NOW for files copied from remote to local over sftp:// protocol.
Comment 21 Dotan Cohen 2009-12-02 19:35:44 UTC
Bug 169326 is a possible dupe, with regard to Bluetooth.
Comment 22 Frank Reininghaus 2009-12-10 18:28:04 UTC
*** Bug 218133 has been marked as a duplicate of this bug. ***
Comment 23 David Faure 2010-02-01 14:08:14 UTC
Copying from file to sftp preserves the modification time for me.
libssh-0.4.0, kde 4.4.0.

The code that calls sftp_utimes in kio_sftp::put() comes from r1010049, i.e. from the initial rewrite of kio_sftp to use libssh in August 2009.
Ah, that's after 4.3.0 was released, so 4.4 is indeed the first release with this new kio_sftp.

-> FIXED for kio_sftp.

No idea how to fix for kio_fish, I don't know the protocol.
Comment 24 Dotan Cohen 2010-02-01 14:34:45 UTC
On my 4.3.5 machine, all operations via SFTP result in NOW timestamp for both Modified and Accessed time:

* Copy/Paste
- local->remote: Modified NOW Accessed NOW
- remote->local: Modified NOW Accessed NOW

*Cut/Paste
- local->remote: Modified NOW Accessed NOW
- remote->local: Modified NOW Accessed NOW


I will fire up a 4.4 prerelease version and test there.
Comment 25 Dotan Cohen 2010-02-01 15:45:25 UTC
Tested on Kubuntu 9.10 with KDE SC 4.4 RC2, libssh-4 0.4.0, connected via SFTP in Dolphin and using the mouse:

* Copy/Paste
- local->remote: Modified THEN Accessed NOW
- remote->local: Modified THEN Accessed NOW

* Cut/Paste
- local->remote: Modified NOW Accessed NOW
- remote->local: Modified NOW Accessed NOW

That is the exact _opposite_ of what one would expect to find. Copy/Paste should be NOW as the user is creating a new file. Cut/Paste should be THEN as the user is merely moving a file form one place to another. In fact, the Copy/Paste operation should be optionally THEN as I can think of quite a few use cases for that.
Comment 26 David Faure 2010-02-01 16:56:11 UTC
I agree, moving a file should preserve its mtime, I'll take a look at that. Surprising...

But I disagree that copying a file should change its mtime; this is what this whole report is about, preserving the mtime on copy ;-)
You didn't create a new file from scratch or modify a file, this is a copy of a file that was modified long ago. Well you seem to hint at that too, but I don't see a use case for making it optional. The goal is to always preserve the mtime.
Comment 27 Dotan Cohen 2010-02-01 17:13:31 UTC
> But I disagree that copying a file should change its mtime; this is what
> this whole report is about, preserving the mtime on copy ;-)

No, the report is that the mtime is wrong. As per the common unix tool cp, mtime is NOW. That is in contrst to mv, which uses THEN.

However, I have no problem with the mtime on copied files being THEN. The critical issue is that the mtime on a moved file should never be NOW. If you make both move and copy operations THEN, then I agree that this long-standing dataloss bug can finally be closed! Thanks!
Comment 28 mi+kde 2010-02-02 18:13:05 UTC
As the original reporter, I'd like to opine, that the mtime should always be preserved -- whether the operation is move (rename), or copy (the equivalent of `cp -p').

There are other timestamps maintained by the filesystem, from which the time of copying (or moving) can be discerned, if needed. But, if the two files are identical (one is the copy of the other), the timestamps on them should be identical too -- copying does not modify the file, even if `cp' (without the `-p' flag) thinks, that it does...

BTW, on certain filesystems, such as FreeBSD's default FFS, there are also flags (such as append-only, no-dump, undeletable, etc.), which should be preserved too, whenever possible.
Comment 29 David Faure 2010-02-02 23:47:14 UTC
Dotan: KDE emulates cp -a, not cp, by choice.

Back to the bug: I confirm that kde-mv loses the mtime. Testcase for fast bug reproduction:

touch -r /boot /tmp/myfile ; kde-mv /tmp/myfile sftp://localhost/$HOME/tmp ; ll ~/tmp/myfile

Debugging now.
Comment 30 David Faure 2010-02-02 23:48:17 UTC
SVN commit 1084346 by dfaure:

Preserve modification time when moving files [and when the kioslave's put() supports it, like kio_sftp, but unlike kio_fish]
This was already working for copying, but not for moving (across protocols).
Fixed for: 4.4.0, hopefully
CCBUG: 55804


 M  +4 -0      copyjob.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1084346
Comment 31 David Faure 2010-02-02 23:54:46 UTC
This commit also fixed  kde-mv from fish to local.

Only kde-mv/kde-cp from local to fish is still broken, due to the not implemented use of "modificationtime" in fish's put(). Reassigning.
Comment 32 Dotan Cohen 2010-02-03 00:05:28 UTC
Thank you David! This bug was a showstopper for me, I cannot wait to get this on production systems. Thanks!
Comment 33 Dotan Cohen 2010-03-06 23:24:30 UTC
Has there been any progress for local to fish? I don't mean to push, but I would like to see the issue resolved :)

Thanks!
Comment 34 Dotan Cohen 2010-11-06 18:59:43 UTC
I'm not sure if this is a new issue or not, so I'll ask:
In KDE 4.5.1 it turns out that copying (copy-paste) from remote to local via FTP in Dolphin gives a timestamp of NOW. Is that this same issue, or should I file another issue?

Thanks.
Comment 35 Paul Sommer 2012-09-06 06:34:59 UTC
How comes that a 9 year old bug is not fixed? 
Are the kiddies too busy programming wobbling windows and other nonsense stuff?
It's a shame.
Solution suggest: If ther's no one smart enough to program a working sftp in the KDE team why not just utilise the working scp that comes with any Linux distribution? No need to re invent the wheel.
Comment 36 Christoph Feck 2012-09-06 10:23:23 UTC
Paul, that sounds like a suggestion you could try. Please add the patch to reviewboard.
Comment 37 David Faure 2012-09-06 12:50:59 UTC
Hmm? This bug is already fixed for kio_sftp, see comment #30.
Comment 38 Dotan Cohen 2012-09-06 13:18:05 UTC
Hi David. This bug is not resolved for uploading local files to remote servers via the fish:// pseudo-protocol, see comment 33. This actually just bit me this week and got someone very mad at me! Paul, do you also go by another name and live in Beersheba? :)

As for comment 34 (remote to local via FTP in Dolphin) I'll have to triage a bit later when I have access to an FTP server. I'll also test via SSH to be complete.
Comment 39 Paul Sommer 2012-09-06 16:42:32 UTC
As far as I can say neither upload nor download is fixed (tried sftp with konqueror and krusader under Kubuntu precise). It's a real showstopper as no directory sync is possible that way. One can copy back and forth 'till eternity.
Christoph, if I was able to provide a patch I would have done years before. Unfortunately I'm not in Linux development, just some perl. 
...and no, Dotan. I don't even know what Beersheba is.
Comment 40 Theo 2013-01-07 00:52:30 UTC
Time stamps used to be preserved for me with KDE 4.6 (openSUSE 11.4)  when copying files with Krusader over SFTP from or to a local directory, except when using the Synchronize Directories Tool. After upgrading to KDE 4.8 the time stamps are no longer preserved even when uploading over SFTP from a panel.

(In about two months we can celebrate the tenth birthday of this bug report.)
Comment 41 Paul Sommer 2013-01-07 10:10:58 UTC
(In reply to comment #40)
> (In about two months we can celebrate the tenth birthday of this bug report.)

Hey, great! 10th anniversary.
Maybe we sould take the chance to rename KDE into BUG.  
This is not the only bug that hasn't been fixed for years. (I only say Amarok)
Comment 42 Dawit Alemayehu 2013-06-10 15:17:09 UTC
(In reply to comment #41)
> (In reply to comment #40)
> > (In about two months we can celebrate the tenth birthday of this bug report.)
> 
> Hey, great! 10th anniversary.
> Maybe we sould take the chance to rename KDE into BUG.  
> This is not the only bug that hasn't been fixed for years. (I only say
> Amarok)

If you do not have anything constructive to say, please keep it to yourself. Adding noise in a bug report will not get it fixed! In case you are not aware, this is a volunteer project.
Comment 43 Dawit Alemayehu 2013-06-10 15:41:56 UTC
I will reiterate what David stated already. Each of the kioslaves are their own independent modules maintained by different people. So if there is a problem in any of them it would have to be fixed in that protocol's ioslave itself. 

As such if you find issues with a specific protocol, it would be wise to open a bug report against that specific protocol's ioslave, so the issue might get handled in a timely fashion. Please note that  most developers are only subscribed to get bug reports against things they maintain ; so a generic bug report like this one might never reach them at all.

Anyhow, if the kio_sftp ioslave is broken again, I am to blame since I added an optimization that made it a bit faster to copy files so long as either the source or the destination is a local file. This was done for the KDE v4.8 release. I will check and fix that.

I will also see what can be done about the ftp ioslave.
Comment 44 Dawit Alemayehu 2013-06-11 12:35:38 UTC
Git commit a3b8bc5ec9ecffa45b2d64789a4d571b7af79cc7 by Dawit Alemayehu.
Committed on 10/06/2013 at 21:51.
Pushed by adawit into branch 'KDE/4.10'.

Set the "modified" (aka mtime) property for direct move jobs, i.e. where the
ioslave has implemented copyFromFile/copyToFile.
REVIEW: 110944

M  +6    -0    kio/kio/job.cpp

http://commits.kde.org/kdelibs/a3b8bc5ec9ecffa45b2d64789a4d571b7af79cc7
Comment 45 Dawit Alemayehu 2013-06-11 12:37:02 UTC
Git commit f0187df8cbcca0f613a23431018255813b129ad1 by Dawit Alemayehu.
Committed on 10/06/2013 at 23:41.
Pushed by adawit into branch 'KDE/4.10'.

Set the proper mtime when moving a file from sftp -> file.
REVIEW: 110947

M  +10   -0    kioslave/sftp/kio_sftp.cpp

http://commits.kde.org/kde-runtime/f0187df8cbcca0f613a23431018255813b129ad1
Comment 46 Andreas Schneider 2013-06-11 12:52:18 UTC
*** Bug 321010 has been marked as a duplicate of this bug. ***
Comment 47 Dotan Cohen 2013-06-11 18:23:57 UTC
Thank you for looking into this, Dawit!
Comment 48 Dawit Alemayehu 2013-06-14 06:28:37 UTC
(In reply to comment #47)
> Thank you for looking into this, Dawit!

No problem, but please note that my fix is only going to address kio_sftp and kio_ftp. And even then for kio_ftp, I can only address the issue for moving files from the ftp server to the local file system. To see why I was not able to do the same for the other case, checkout http://git.reviewboard.kde.org/r/111016/.

After the code listed above is committed, the timestamp set on copied/moved files through

kio_file will be:
============
* Copy/Paste
- local->remote: Modified THEN Accessed NOW
- remote->local: Modified THEN Accessed NOW

*Cut/Paste
- local->remote: Modified THEN Accessed NOW
- remote->local: Modified THEN Accessed NOW

kio_sftp will be:
=============
* Copy/Paste
- local->remote: Modified NOW Accessed NOW
- remote->local: Modified NOW Accessed NOW

*Cut/Paste
- local->remote: Modified THEN Accessed NOW
- remote->local: Modified THEN Accessed NOW

kio_ftp will be:
=============
* Copy/Paste
- local->remote: Modified NOW Accessed NOW
- remote->local: Modified NOW Accessed NOW

*Cut/Paste
- local->remote: Modified NOW Accessed NOW   [ requires new implementation ]
- remote->local: Modified THEN Accessed NOW
Comment 49 Dawit Alemayehu 2013-06-15 20:34:15 UTC
Git commit 4a75f8b6dbc36a858b5a05e4f65d39d07b0dc98c by Dawit Alemayehu.
Committed on 14/06/2013 at 06:22.
Pushed by adawit into branch 'KDE/4.10'.

Set the modified timestamp on ftp -> file moves.
REVIEW: 111016

M  +15   -0    kioslave/ftp/ftp.cpp

http://commits.kde.org/kdelibs/4a75f8b6dbc36a858b5a05e4f65d39d07b0dc98c
Comment 50 Dawit Alemayehu 2013-09-05 13:29:11 UTC
Git commit ef3de4a99c3af0557f51afe61e30addb592f2d4a by Dawit Alemayehu.
Committed on 04/09/2013 at 05:00.
Pushed by adawit into branch 'KDE/4.11'.

Set modified meta-data for all copy jobs, not just move operations.
REVIEW: 112528

M  +2    -2    kio/kio/job.cpp

http://commits.kde.org/kdelibs/ef3de4a99c3af0557f51afe61e30addb592f2d4a
Comment 51 Mike 2015-11-09 14:46:17 UTC
For me it seems the bug ist still open ....because I have this in Plasma 5. It's a really serious bug for digikam users after realizing that copied files from SD card loose their create-date info..; :-(
Comment 52 Vadim 2016-07-03 01:14:40 UTC
Ubuntu 14.04.4 LTS
Qt: 4.8.6
KDE: 4.14.13

Krusader
Version 2.4.0-beta3 "Single Step"
Using KDE Development Platform 4.14.13

When I synchronize directories to smb timestamp changed to new one.
Bug is still open.
Comment 53 szecsodimlaszlo 2022-03-11 11:07:09 UTC
Happy 19th anniversary.

Arch Linux
KDE 5.24.3
Qt 5.15.3 & 6.2.3
kio 5.91.0
kio-extras 21.12.3
krusader 2.7.2 & 2.8.0.dev