Bug 372999 - Compress and Extract plugins freezing application for long time if many files selected
Summary: Compress and Extract plugins freezing application for long time if many files...
Status: RESOLVED FIXED
Alias: None
Product: ark
Classification: Applications
Component: general (show other bugs)
Version: 16.08.2
Platform: Arch Linux Linux
: NOR major
Target Milestone: ---
Assignee: Elvis Angelaccio
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-11-27 17:01 UTC by Alex Bikadorov
Modified: 2016-11-30 15:29 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 16.11.90


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alex Bikadorov 2016-11-27 17:01:04 UTC
Hi,

the "compressfileitemaction" and "extractfileitemaction" are blocking the parent application (Dolphin and Krusader) for a long time if many files are selected and the right-click popupmenu is about to be shown.

Tested with 1000 (empty) files - it freezes for about 5 seconds before the menu is shown. And think its linear: for 10.000 files its 50 seconds.
During that time console prints:
> Mimetype for filename extension ( "text/plain" ) did not match mimetype for content ( "application/x-zerosize" ). Using content-based mimetype."

For Krusader i will disable the plugins for >1000 selected files until this is fixed.
Comment 1 Alex Bikadorov 2016-11-27 17:10:02 UTC
Git commit 823ee73b5eaac87bd7664db0ba264b4370e07071 by Alexander Bikadorov.
Committed on 27/11/2016 at 17:09.
Pushed by abikadorov into branch 'master'.

Panel: workaround for long freeze on popupmenu creation with many files selected

M  +4    -2    krusader/Panel/krpopupmenu.cpp

https://commits.kde.org/krusader/823ee73b5eaac87bd7664db0ba264b4370e07071
Comment 2 Elvis Angelaccio 2016-11-27 17:32:03 UTC
Thanks for the report, I will investigate.
Comment 3 Elvis Angelaccio 2016-11-27 18:05:56 UTC
The bottle-neck is in extractfileitemaction.cpp:

    foreach (const QUrl &url, fileItemInfos.urlList()) {
        const QMimeType mimeType = determineMimeType(url.fileName());
        if (m_pluginManager->preferredPluginsFor(mimeType).isEmpty()) {
            continue;
        }
        ...
    }

For some reason, determineMimeType() is slow (query to the mime DB?)
Comment 4 Elvis Angelaccio 2016-11-27 18:32:08 UTC
Actually, it's 

    m_pluginManager->preferredPluginsFor()

that is slow.
Comment 5 Elvis Angelaccio 2016-11-28 11:26:02 UTC
Git commit 1b817f61f187eda4dc14a3227e0c0430c30d808e by Elvis Angelaccio.
Committed on 28/11/2016 at 11:23.
Pushed by elvisangelaccio into branch 'Applications/16.12'.

Sort mimetypes by comment only when necessary

The list of supported mimetypes is quite long and sorting them by
comment is a non-negligible operation.
Now we do it only when the mimetypes are shown in user-visible dialogs.

M  +1    -1    app/mainwindow.cpp
M  +1    -1    kerfuffle/createdialog.cpp
M  +12   -4    kerfuffle/pluginmanager.cpp
M  +14   -4    kerfuffle/pluginmanager.h

https://commits.kde.org/ark/1b817f61f187eda4dc14a3227e0c0430c30d808e
Comment 6 Elvis Angelaccio 2016-11-29 18:44:40 UTC
Git commit 7618ea17108877e308d6c7b4e767fedda9534f9f by Elvis Angelaccio.
Committed on 29/11/2016 at 18:39.
Pushed by elvisangelaccio into branch 'Applications/16.12'.

pluginmanager: cache list of preferred plugins

PluginManager::preferredPluginsFor() is an hot path as it can be called
thousands of times from the extractfileitemaction plugin.
We can save the list of preferred plugins in a cache, instead of
re-computing it over and over again.

Tested in Dolphin by right-clicking:
* 1K files: no delay
* 10K files: ~1 sec. of delay
* 50K files: ~4 sec. of delay

The (empty) test files were created with:
    printf '%s ' {1..1000} | xargs touch
FIXED-IN: 16.11.90

Differential Revision: D3527

M  +10   -3    kerfuffle/pluginmanager.cpp
M  +4    -3    kerfuffle/pluginmanager.h

https://commits.kde.org/ark/7618ea17108877e308d6c7b4e767fedda9534f9f
Comment 7 Alex Bikadorov 2016-11-30 15:14:43 UTC
Thanks for the quick fix! Unfortunately this is not what I experienced with bugs in frameworks-*.

I guess I have to keep the workaround in Krusader until Applications/16.12 (or higher) is common everywhere. (If you know of a best practice solution please enlighten me).
Comment 8 Elvis Angelaccio 2016-11-30 15:22:30 UTC
(In reply to Alex Bikadorov from comment #7)
> Thanks for the quick fix! Unfortunately this is not what I experienced with
> bugs in frameworks-*.
> 
> I guess I have to keep the workaround in Krusader until Applications/16.12
> (or higher) is common everywhere. (If you know of a best practice solution
> please enlighten me).

I could try to backport the fix on 16.08 branch and ask distros to rebuild their ark 16.08.3 packages, but that doesn't guarantee everyone will get the fix unfortunately.
Comment 9 Alex Bikadorov 2016-11-30 15:29:25 UTC
(In reply to Elvis Angelaccio from comment #8)
> I could try to backport the fix on 16.08 branch and ask distros to rebuild
> their ark 16.08.3 packages, but that doesn't guarantee everyone will get the
> fix unfortunately.

Hmm, that feels like too much hassle for a non-major bug. I think everybody can live with the workaround for a while. Anyway, thanks again.