Bug 427644 - Dolphin reads incomplete file list from samba network share
Summary: Dolphin reads incomplete file list from samba network share
Status: RESOLVED FIXED
Alias: None
Product: kio-extras
Classification: Frameworks and Libraries
Component: Samba (show other bugs)
Version: 20.08
Platform: Neon Linux
: NOR normal
Target Milestone: ---
Assignee: Plasma Bugs List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-10-13 17:10 UTC by piotr
Modified: 2020-10-14 17:18 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In: 20.08.3


Attachments
The properties dialog. The same directory, note the diffrerent size and number of files. (39.37 KB, image/png)
2020-10-13 17:10 UTC, piotr
Details
The properties dialog. The same directory, note the diffrerent size and number of files. (34.26 KB, image/png)
2020-10-13 17:32 UTC, piotr
Details

Note You need to log in before you can comment on or make changes to this bug.
Description piotr 2020-10-13 17:10:32 UTC
Created attachment 132326 [details]
The properties dialog. The same directory, note the diffrerent size and number of files.

SUMMARY

When accessing samba (cifs) network share with large number of big files (in this particular case 3578 jpg files totaling 9GB of data) Dolphin shows only partial directory contents, displaying random number of files (e.g. showing first that the directory contains 3175 files, after refreshing the view showing that it contains 3239 files etc.) *without notifying user that file listing might be incomplete*. Similarly, computing the directory size with properties dialog yields different values with every click on the "compute" button. 

Above behavior has been observed over relatively slow - and possibly unreliable - network connection (wifi extender coupled to powerline adapter), which might - or might not - be a factor here. Note however that in any case  no error information was given by Dolphin.

THIS MIGHT RESULT IN DATA LOSS E.G. WHEN COPYING FILES!!!   


STEPS TO REPRODUCE
1. Open samba network share with large number of large files with Dolphin (i.e. smb:// location)
2. Wait for Dolphin to complete loading file list, note number of files in the directory
3. Repeat above procedure several times

OBSERVED RESULT
Number of files in the directory is different every time, some files present in the network share are not shown in the directory by Dolphin!

EXPECTED RESULT
Dolphin should show all the files present in the network share (and of course the number of files displayed should be the same with every refresh) or alternatively user should be notified that the loading of the directory list was interrupted (and thus the folder view is corrupted).

SOFTWARE/OS VERSIONS
Operating System: KDE neon 5.20
KDE Plasma Version: 5.20.0
KDE Frameworks Version: 5.75.0
Qt Version: 5.15.0
Kernel Version: 5.4.0-48-generic
OS Type: 64-bit
Processors: 4 × Intel® Core™ i5-2410M CPU @ 2.30GHz
Memory: 7.6 GiB of RAM
Graphics Processor: Mesa DRI Intel® HD Graphics 3000

ADDITIONAL INFORMATION
In this case the SMB server supports only 1.0 version of the protocol, so appropriate definitions have been made in smb.conf. However mounting the share in the shell via mount command (and accessing it - copying files via cp and Midnight Commander) works correctly.
Comment 1 piotr 2020-10-13 17:32:38 UTC
Created attachment 132327 [details]
The properties dialog. The same directory, note the diffrerent size and number of files.

Compare with previous one
Comment 2 Harald Sitter 2020-10-14 12:17:38 UTC
You are only showing the properties dialog having incorrect numbers. That has nothing to do with what is actually shown in dolphin or what will be copied. What makes you think that files are actually missing in either of these scenarios?

Are the files in the directory indeed only jpg files, or are there possibly sym/hardlinks involved as well?
Comment 3 Harald Sitter 2020-10-14 14:13:04 UTC
Nevermind. I already see what's going on.

Since the smbc discovery refactor there's a hard cap on how long listing may take, this can easily get exhausted for smbc content listing.
Comment 4 Bug Janitor Service 2020-10-14 14:36:49 UTC
A possibly relevant merge request was started @ https://invent.kde.org/network/kio-extras/-/merge_requests/36
Comment 5 Harald Sitter 2020-10-14 15:49:22 UTC
Git commit c58fdc63aeab10db329cd5a6167254543684c1a2 by Harald Sitter.
Committed on 14/10/2020 at 14:35.
Pushed by sitter into branch 'release/20.08'.

smb: no discovery cap on file listing

File and share listing can take a very long time as they are IO bound on
the remote, so if one is listing a large directory on a HDD it may well
take minutes. Equally sending huge listings over the wire may take a
while if not enough throughput is available. So don't cap the execution
of these discoveries at all.

Server discovery retains the timeout, mostly because I don't want to
remove it in 20.08 though an argument could be made that we should trust
in the reliability of the discovery stacks to not get stuck on random
unforseen misbehavior of servers.
FIXED-IN: 20.08.3

M  +8    -1    smb/kio_smb_browse.cpp

https://invent.kde.org/network/kio-extras/commit/c58fdc63aeab10db329cd5a6167254543684c1a2