Bug 353813 - File System Browser freezes Kate when opening a file in a folder with thousands of files
Summary: File System Browser freezes Kate when opening a file in a folder with thousan...
Status: RESOLVED UPSTREAM
Alias: None
Product: frameworks-kio
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: unspecified
Platform: Kubuntu Linux
: NOR crash
Target Milestone: ---
Assignee: David Faure
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-10-12 02:51 UTC by Lê Hoàng Phương
Modified: 2021-01-13 18:44 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Lê Hoàng Phương 2015-10-12 02:51:25 UTC
The current file system browser doesn't handle a folder with lots of files very well. I'm currently working with a folder with thousands (around 5k) of .svg files and opening one file will lead to Kate consuming 100% CPU and being frozen for very long time.

Reproducible: Always

Steps to Reproduce:
1. Generate a folders with 5k of text files.
2. Open one of them with Kate

Actual Results:  
Kate consumes 100% CPU and freezes

Expected Results:  
Kate should not freeze, and should not use 100% CPU just for listing files.
Comment 1 Christoph Cullmann 2016-01-31 20:58:24 UTC
This is then some KDirOperator issue.
Comment 2 Méven Car 2018-10-28 11:04:35 UTC
I reproduced on a SSD drive with 25000 0 byte files.
Dolphin can even crashes.

python code to reproduce creating 25000 files:
for i in range(0, 25000):
    open("file_%d" % i, "w")

When testing with kdeconnect (introducing slower IO) on a mounted phone in a directory containing thousands of files, the issue is more severe.
Comment 3 Méven Car 2018-10-28 12:42:25 UTC
I believe this can be explained indeed by kdiroperator.cpp

In

void KDirOperator::setUrl(const QUrl &_newurl, bool clearforward)

[...]
        KIO::StatJob *job = KIO::stat(newurl);
        KJobWidgets::setWindow(job, this);
        bool res = job->exec(); // <- The job is executed synchronously, main thread is paused
[...]
Comment 4 Méven Car 2018-10-28 15:33:00 UTC
The problem is more likely related to the lines in

void KDirOperator::setUrl(const QUrl &_newurl, bool clearforward)

    d->openUrl(newurl);

And

bool KDirOperator::Private::openUrl(const QUrl &url, KDirLister::OpenUrlFlags flags)
{
    const bool result = KProtocolManager::supportsListing(url)
                        && isSchemeSupported(url.scheme())
                        && dirLister->openUrl(url, flags);


They call dirLister->openUrl which calls in the end KCoreDirListerCache::listDir, which is a synchronous call, hence the freeze of the UI during the IO.

About the crash, it happens under wayland (I haven't tried with X):

# dolphin // opens the dir with 25000 files
Using Wayland-EGL
qt.accessibility.core: Cannot create accessible child interface for object:  PlacesView(0x55d684e6f100)  index:  15
Using the 'xdg-shell-v6' shell integration
The Wayland connection broke. Did the Wayland compositor die?
Comment 5 Patrick Silva 2019-03-29 14:43:48 UTC
This bug persists :(

Operating System: Arch Linux 
KDE Plasma Version: 5.15.3
KDE Frameworks Version: 5.56.0
Qt Version: 5.12.2
Comment 6 Nate Graham 2021-01-13 18:44:56 UTC
#6  QMessageLogger::fatal (this=this@entry=0x7fffd70c5ba0, msg=msg@entry=0x7ff994ac00b8 "The Wayland connection broke. Did the Wayland compositor die?") at global/qlogging.cpp:893

This means that the compositor crashed. Due to a Qt issue, when this happens, the app using it will crash too. KDE developers submitted a fix, but sadly it was not merged. See https://codereview.qt-project.org/c/qt/qtwayland/+/308984.

Until we get better handling of this in Qt, the best we can do is debug why the compositor crashed in the first place. So can you please get a backtrace of the crash in kwin_wayland and then file a new bug report with it on kwin | wayland-generic? Thanks!

You may be able to use the `coredumpctl` utility to retrieve the backtrace. See https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports#Retrieving_a_backtrace_using_coredumpctl