Bug 409360

Cannot reproduce when dragging a test file from one folder to another, and choosing 'Move Here'. Did you move it differently?

Tested this again today. To me it happens using 'Move here' as you did. But today I have noticed that it happens only on certain folders.

To give you a context:
If I connect to the server (a cluster) via ssh I use the address myaddress.com, when I need to transfer files via sftp I need to use the address transfer.myaddress.com.

When I move file in my home (and subdirectories) no problems appear. When I move file in another folder called /work1 the files gets converted to links after moving.

The folder /work1 is likely a network drive mounted on the remote server, indeed `mount` returns this entry:

beegfs_nodev on /work1 type beegfs (rw,_netdev,cfgFile=/etc/beegfs/beegfs-client2.conf,)

How can I provide more useful debug information?

Sorry for the delay especially on a bug with alleged data loss.

Can I clarify some things.


You visit your server via sftp:// in the address bar in dolphin
That server has a network mount using beegfs at /work1

Moving files from your remote machine to the remote machine where the destination is /work1 fails


Using /work1 as a source in a remote -> remote move is fine
Uploading from local to /work1 is fine

Here are all the combinations that I have tested, I hope this makes the thing more clear

/work1 means the network mounted beegfs location on the remote machine or any subfolder of it
~ means the home of remote machine or any subfolder of it
Local is my local machine

Ok - Local > Remote:~
Ok - Remote:~ > Local
Ok - Rename on Remote:~
Ok - Remote:~ > Remote:~
Ok - Local > Remote:/work1/
Ok - Remote:/work1/ > Local
Ok - Rename on Remote:/work1/
Fails / data loss - Remote:/work1/ > Remote:/work1/
Ok - Remote:/work1/ > Remote:~ (Dolphin split view)
Ok - Remote:~ > Remote:/work1/  (Dolphin split view)

The failure with data loss happens only when moving from /work1 (or a subfolder) to a different folder also in /work1. 


## New error I found

Fails / Internal error - Remote:/work1/ > Remote:/work1/home (Remote:/work1/home is a symlink to Remote:~)
Fails / Internal error - Remote:~ > Remote:~/work1/  (Remote:~/work1 is a symlink to Remote:/work1)

When moving across the remote home and /work1:
- it works if I use dolphin split view (basically there are two connections to the server)
- it fails with Dolphin reporting a message saying that there was an interal error when I try to move the file within the same Dolphin view (one sftp connection) but across devices. To do this I used symlinks on the remote server as explained above.

That's a really good breakdown, thank you

One more question as you have such a setup ready


Do you see the same thing when using fish://myServer instead of sftp://myServer

I am not sure what we can do about this. This isn't really a bug on our end.

When beegfs mounts are configured with sysCreateHardlinksAsSymlinks [1] they prevent hardlinks from working and automatically convert them to symlinks (ugh). And most sftp daemons implement the sftp 'rename' operation as a hardlink followed by a deletion of the old file name. Essentially the beegfs flag breaks sftp. Simply as that.
The hardlink trick is specifically to work around the fact that the protocol wants rename to fail when the file exists but posix rename operations would overwrite them by default. That is only true up to sftp protocol version 3 or so. 4+ introduced flags to explicitly allow the server to overwrite, of course that requires the server to actually implement protocol version 4, which openssh for example does not do AFAIK. [2]
So, your sysadmins really should not combine that beegfs flag from above while also allowing sftp access over protocol versions <4. The two are simply not compatible, certainly not for servers that implement rename using the hardlink trick.

libssh could possibly implement the posix-rename@openssh.com extension which would, I guess, solve this problem for us. CCing Andreas for his point of view on the matter.

[1] https://github.com/conda/conda/pull/2355
[2] https://tools.ietf.org/html/draft-ietf-secsh-filexfer-13#section-8.3

Thanks for the explanation, I have notified this problem to my sysadmins.

With fish:

Ok - Remote:/work1/ > Local
Ok - Rename on Remote:/work1/
Ok - Remote:/work1/ > Remote:/work1/

Fails / Could not rename file - Remote:/work1/ > Remote:/work1/home (Remote:/work1/home is a symlink to Remote:~)
Fails / Could not rename file - Remote:~ > Remote:~/work1/  (Remote:~/work1 is a symlink to Remote:/work1)

So with fish the original problem seems not present.

I think to fix this, we need https://bugs.libssh.org/T98

Closing as this needs dealing with in libssh upstream.