Bug 458001 - KDE / KIO / Dolphin needs to support "copy_file_range" for server-side copies over a network (such as NFS)
Summary: KDE / KIO / Dolphin needs to support "copy_file_range" for server-side copies...
Status: RESOLVED FIXED
Alias: None
Product: frameworks-kio
Classification: Frameworks and Libraries
Component: general (other bugs)
Version First Reported In: unspecified
Platform: unspecified Linux
: NOR wishlist
Target Milestone: ---
Assignee: KIO Bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-08-17 17:05 UTC by flan_suse
Modified: 2022-08-27 07:26 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed/Implemented In: 5.98
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description flan_suse 2022-08-17 17:05:56 UTC
When copying a large file on an NFS mount with Dolphin, it takes a very long time since it does a round trip over the network.

NFS 4.2 has already supported server side copies, yet Dolphin does not take advantage of this with "copy_file_range".

This means someone has to resort to the terminal with "cp" in order to copy files without sending the data on a round trip over the network.

I thought this was included in KDE Frameworks 5.88? But apparently this is not the case?

Has anything been done with this: https://invent.kde.org/frameworks/kio/-/merge_requests/602
Comment 1 Nicolas Fella 2022-08-17 20:35:14 UTC
Is this about browsing NFS directly in Dolphin (i.e. a nfs:/ URL) or a NFS mounted to the local filesystem and accessed via Dolphin as if it was a regular file?
Comment 2 flan_suse 2022-08-18 02:28:40 UTC
(In reply to Nicolas Fella from comment #1)
> Is this about browsing NFS directly in Dolphin (i.e. a nfs:/ URL) or a NFS
> mounted to the local filesystem and accessed via Dolphin as if it was a
> regular file?

Mounted normally with "mount".

For example:

mount -t nfs 192.168.0.100:/remote/path/of/share /mnt/remote

nfsstat -m reveals the client and server are both using NFS v4.2. Copying a file with cp is very fast and uses "copy_fele_range" as expected. The copy operation does *not* make a round trip over the gigabit network. It's done on the server (which is one of the major features specifically introduced with NFS v4.2)

Output of nfsstat -m:

Flags: rw,relatime,vers=4.2,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.2,local_lock=none,addr=192.168.0.100

Copying a large 2 GB file with cp takes only a few seconds. (Server side copy!) :)

Copying a large 2 GB file with Dolphin takes over half a minute! (Round trip over the network.) :'(

The 2 GB file is filled with random data.

I expected this feature was already used by Dolphin since KDE Frameworks 5.88+, but it doesn't seem to be the case?

Is there any activity or update on this?
Comment 3 Nicolas Fella 2022-08-18 10:41:14 UTC
Okay, so this is about the file worker, not the nfs worker.
Comment 4 flan_suse 2022-08-18 14:51:36 UTC
(In reply to Nicolas Fella from comment #3)
> Okay, so this is about the file worker, not the nfs worker.

My main confusion / question:

Has this been addressed by the KDE developers and *should* be working as expected?

Or was the feature "accepted" but never finalized?

Being a layperson end-user, I cannot tell if it was actually included in the final code or not.

https://invent.kde.org/frameworks/kio/-/merge_requests/602

That's why I was unsure to submit this ticket as a "bug" or a "wishlist". Because if it *was* implemented, then it's not working as intended. However, if it was not implemented, then yes, this is a *dearly* needed feature of Dolphin/KIO.

Network shares (even NFS) and NAS appliances are not rare for Linux users. Besides, everything else is in place; it's just that the client (i.e, Dolphin) needs to use it.
Comment 5 Bug Janitor Service 2022-08-20 08:46:35 UTC
A possibly relevant merge request was started @ https://invent.kde.org/frameworks/kio/-/merge_requests/957
Comment 6 Méven Car 2022-08-20 09:23:56 UTC
(In reply to flan_suse from comment #4)
> (In reply to Nicolas Fella from comment #3)
> > Okay, so this is about the file worker, not the nfs worker.
> 
> My main confusion / question:
> 
> Has this been addressed by the KDE developers and *should* be working as
> expected?
> 
> Or was the feature "accepted" but never finalized?
> 
> Being a layperson end-user, I cannot tell if it was actually included in the
> final code or not.
> 
> https://invent.kde.org/frameworks/kio/-/merge_requests/602
> 
> That's why I was unsure to submit this ticket as a "bug" or a "wishlist".
> Because if it *was* implemented, then it's not working as intended. 

Despite it was implemented and improved things, we still had inefficiencies that my new MR addresses mostly.
Your bug report is quite legitimate, thanks for pointing this out.
Comment 7 Méven Car 2022-08-26 16:35:15 UTC
Git commit fa8edc7099820d72179b29ccd15535b5398c5952 by Méven Car, on behalf of Méven Car.
Committed on 26/08/2022 at 16:35.
Pushed by meven into branch 'master'.

file_unix: optimize copy

 * avoid to stat the source file to get its size in the loop

M  +8    -7    src/ioslaves/file/file_unix.cpp

https://invent.kde.org/frameworks/kio/commit/fa8edc7099820d72179b29ccd15535b5398c5952
Comment 8 Méven Car 2022-08-27 07:26:05 UTC
Please note we still have a margin of improvements, still this MR (https://invent.kde.org/frameworks/kio/-/merge_requests/602) should be significant especially for NFS to NFS transferts.

Feel free to reopen this bug if you feel it has not reached what you expected.

The next speed improvement we can get is in our radar (https://invent.kde.org/frameworks/kio/-/merge_requests/957#note_509744) but will necessitate a lot more work unfortunately.