Bug 457408

Summary: Discover is slow to load when a lot of backends fail at the same time
Product: [Applications] Discover Reporter: Jonathan Wakely <zilla>
Component: KNewStuff BackendAssignee: Plasma Bugs List <plasma-bugs>
Status: RESOLVED FIXED    
Severity: normal CC: aleixpol, leinir, nate, rdieter, yvan
Priority: NOR    
Version: 5.25.3   
Target Milestone: ---   
Platform: Fedora RPMs   
OS: Linux   
Latest Commit: Version Fixed In: 5.26.1
Attachments: Output of plasma-discover --backends kns-backend --mode Update
Output of plasma-discover --backends flatpak-backend,kns-backend --mode Update

Description Jonathan Wakely 2022-08-02 11:53:23 UTC
There's still something wrong with the Discover UX, despite Bug 443555 fixing the worst source of slowness.

If I get a systray notification that there are updates, and I click on it, it takes 30 seconds to stop showing the "fetching updates" progress bar. Why? Is it refreshing  something? It doesn't seem to be, because it still tells me I have system RPMs to update, even though I've already updated them on the command line with dnf. So it's not refreshing the packagekit backend, otherwise it would know that the updates have been installed.

The available backends are:

$ plasma-discover --listbackends 
Available backends:
 * kns-backend
 * packagekit-backend
 * fwupd-backend
 * flatpak-backend


If I run 'plasma-discover --backends flatpak-backend,packagekit-backend,fwupd-backend' then the window loads instantly, and takes less than a second to finish "Fetching updates" and show "Up to date" at the bottom left corner.

If I add kns-backend to that list, the window opens but shows "Fetching updates" and is completely unresponsive for 30 seconds. Is that the same problem as when I click on the "Updates available" notification?

What even is KNewStuff, and what do I miss out on if I disable it?
Comment 1 Jonathan Wakely 2022-08-02 12:11:06 UTC
Is it even possible to disable that backend? I don't see anything under Settings in the discover app, or any discoverrc config setting that would disable the kns-backend (or any other backend).
Comment 2 Nate Graham 2022-08-02 13:39:20 UTC
> If I run 'plasma-discover --backends flatpak-backend,packagekit-backend,fwupd-backend' then
> the window loads instantly, and takes less than a second to finish "Fetching updates" and show
> "Up to date" at the bottom left corner.
That's excellent news.

> Is that the same problem as when I click on the "Updates available" notification?

Yes, when you open Discover from the notifier, it loads all backends, so that'll be the cause there too.

So we need to figure out why the KNS backend is so slow for you. Can you paste the output from your terminal window when you run `plasma-discover --backends kns-backend --mode Update`? Thanks!
Comment 3 Jonathan Wakely 2022-08-02 13:54:36 UTC
Created attachment 151072 [details]
Output of plasma-discover --backends kns-backend --mode Update

This is the output of that command. However, when I run this, the UI finishes loading and is responsive almost immediately.

So although this output shows lots of errors (which I'm told are already fixed after 5.25.3) they don't seem to be the cause of the slowness.
Comment 4 Jonathan Wakely 2022-08-02 14:04:31 UTC
Created attachment 151074 [details]
Output of plasma-discover --backends flatpak-backend,kns-backend --mode Update

This is the output of the same command but using flatpak-backend,kns-backend

Now, the UI takes 16 seconds to go from "Fetching updates" to "Up to date", and then takes even longer to actually be responsive (maybe the flatpak backend is still updating in a background thread?). It takes another 20+ seconds to actually be responsive and let me click on anything (it won't even close the window during that time, clicking the close button brings up the 'Application "discover" is not responding' dialog to kill it).

If I use --backends fwupd-backend,kns-backend then it loads immediately. Similarly, if I use just flatpak-backend or just packagekit-backend, it loads immediately.

But it's very slow when kns-backend is combined with either flatpak-backend or packagekit-backend. And of course all of them are enabled by default when you click the update notifier.
Comment 5 Nate Graham 2022-08-02 14:21:16 UTC
Very interesting observation, thanks for that.
Comment 6 Nate Graham 2022-08-02 14:21:59 UTC
I can reproduce this, too.
Comment 7 Jonathan Wakely 2022-08-02 14:28:52 UTC
(In reply to Jonathan Wakely from comment #4)
> If I use --backends fwupd-backend,kns-backend then it loads immediately.
> Similarly, if I use just flatpak-backend or just packagekit-backend, it
> loads immediately.

This suggests that there is caching happening, and that for some combinations of backends, the already-cached set of updates is reused and quickly loaded.

But for some "bad" combinations, the cache isn't used, and so some of the updates get re-fetched every time, making it slow. And the default (all backends enabled) happens to be one of those bad combinations.

Just a guess though.
Comment 8 Aleix Pol 2022-10-13 15:38:53 UTC
This should be addressed by https://invent.kde.org/plasma/discover/-/merge_requests/383
Comment 9 Aleix Pol 2022-10-13 15:45:10 UTC
Git commit b704933b2d8cf92c8b6bf552f1ee4c1f817850d9 by Aleix Pol.
Committed on 13/10/2022 at 15:40.
Pushed by apol into branch 'master'.

Delay subCategoriesChanged signals when disabling

They can trigger a lot of JS code that induce lockups
Related: bug 401666

M  +17   -2    libdiscover/Category/Category.cpp
M  +2    -0    libdiscover/Category/Category.h

https://invent.kde.org/plasma/discover/commit/b704933b2d8cf92c8b6bf552f1ee4c1f817850d9
Comment 10 Aleix Pol 2022-10-13 15:47:55 UTC
Git commit a3dd72f42c92c9a7077164697e5e9b0badf518b4 by Aleix Pol Gonzalez, on behalf of Aleix Pol.
Committed on 13/10/2022 at 15:47.
Pushed by apol into branch 'Plasma/5.26'.

Delay subCategoriesChanged signals when disabling

They can trigger a lot of JS code that induce lockups
Related: bug 401666


(cherry picked from commit b704933b2d8cf92c8b6bf552f1ee4c1f817850d9)

M  +17   -2    libdiscover/Category/Category.cpp
M  +2    -0    libdiscover/Category/Category.h

https://invent.kde.org/plasma/discover/commit/a3dd72f42c92c9a7077164697e5e9b0badf518b4
Comment 11 Jonathan Wakely 2023-01-15 11:05:27 UTC
It's still very very slow for "Fetching updates..." to finish with 5.26.5

Far worse, in fact.

Here I run plasma-discover and when it opens (and everything is up to date) I hit "refresh", and it takes FOUR MINUTES:
tmp$ time plasma-discover  --mode Update
QObject::startTimer: Timers cannot have negative intervals
qrc:/qml/DiscoverPage.qml:17:13: QML InlineMessage: Cannot anchor to an item that isn't a parent or sibling.
adding empty sources model QStandardItemModel(0x55d643689e60)
qrc:/qml/DiscoverPage.qml:17:13: QML InlineMessage: Cannot anchor to an item that isn't a parent or sibling.
file:///usr/lib64/qt5/qml/org/kde/kirigami.2/private/globaltoolbar/ToolBarPageHeader.qml:12:1: QML ToolBarPageHeader: Binding loop detected for property "implicitWidth"
qrc:/qml/DiscoverPage.qml:17:13: QML InlineMessage: Cannot anchor to an item that isn't a parent or sibling.
file:///usr/lib64/qt5/qml/org/kde/kirigami.2/private/PrivateActionToolButton.qml:76:5: QML Binding: Binding loop detected for property "value"
kf.newstuff.core: Could not find category "Calligra Flow Stencil"
KNS error in "Calligra_stencils" : KNSCore::ConfigFileError "All categories are missing" QVariant(Invalid)
invalid kns backend! "/etc/xdg/calligra_stencils.knsrc" because: "Invalid Calligra_stencils backend, contact your distributor."
org.kde.plasma.libdiscover: Discarding invalid backend "calligra_stencils.knsrc"
kns error "/etc/xdg/calligra_stencils.knsrc" "Invalid Calligra_stencils backend, contact your distributor."

** (process:477766): WARNING **: 10:54:32.945: Found icon of unknown type 'unknown' in 'system/package/os/org.kde.latte-dock.desktop/*', skipping it.

** (process:477766): WARNING **: 10:54:32.965: Found icon of unknown type 'unknown' in 'system/package/os/org.kde.discover.snap/*', skipping it.

** (process:477766): WARNING **: 10:54:32.980: Found icon of unknown type 'unknown' in 'system/package/os/io.github.syllo.nvtop/*', skipping it.

** (process:477766): WARNING **: 10:54:33.122: Found icon of unknown type 'unknown' in 'system/package/os/com.github.maliit.keyboard/*', skipping it.

** (process:477766): WARNING **: 10:54:33.204: Found icon of unknown type 'unknown' in 'system/package/os/org.kde.kiten.desktop/*', skipping it.

real    3m57.966s
user    0m6.412s
sys     0m0.498s
tmp$ time plasma-discover  --mode Update
QObject::startTimer: Timers cannot have negative intervals
qrc:/qml/DiscoverPage.qml:17:13: QML InlineMessage: Cannot anchor to an item that isn't a parent or sibling.
adding empty sources model QStandardItemModel(0x55b55a561640)
qrc:/qml/DiscoverPage.qml:17:13: QML InlineMessage: Cannot anchor to an item that isn't a parent or sibling.
file:///usr/lib64/qt5/qml/org/kde/kirigami.2/private/globaltoolbar/ToolBarPageHeader.qml:12:1: QML ToolBarPageHeader: Binding loop detected for property "implicitWidth"
qrc:/qml/DiscoverPage.qml:17:13: QML InlineMessage: Cannot anchor to an item that isn't a parent or sibling.
file:///usr/lib64/qt5/qml/org/kde/kirigami.2/private/PrivateActionToolButton.qml:76:5: QML Binding: Binding loop detected for property "value"
kf.newstuff.core: Could not find category "Calligra Flow Stencil"
KNS error in "Calligra_stencils" : KNSCore::ConfigFileError "All categories are missing" QVariant(Invalid)
invalid kns backend! "/etc/xdg/calligra_stencils.knsrc" because: "Invalid Calligra_stencils backend, contact your distributor."
org.kde.plasma.libdiscover: Discarding invalid backend "calligra_stencils.knsrc"
kns error "/etc/xdg/calligra_stencils.knsrc" "Invalid Calligra_stencils backend, contact your distributor."

** (process:479295): WARNING **: 10:58:42.636: Found icon of unknown type 'unknown' in 'system/package/os/org.kde.latte-dock.desktop/*', skipping it.

** (process:479295): WARNING **: 10:58:42.655: Found icon of unknown type 'unknown' in 'system/package/os/org.kde.discover.snap/*', skipping it.

** (process:479295): WARNING **: 10:58:42.670: Found icon of unknown type 'unknown' in 'system/package/os/io.github.syllo.nvtop/*', skipping it.

** (process:479295): WARNING **: 10:58:42.815: Found icon of unknown type 'unknown' in 'system/package/os/com.github.maliit.keyboard/*', skipping it.

** (process:479295): WARNING **: 10:58:42.895: Found icon of unknown type 'unknown' in 'system/package/os/org.kde.kiten.desktop/*', skipping it.

real    0m3.756s
user    0m4.036s
sys     0m0.376s
tmp$ time plasma-discover  --mode Update
QObject::startTimer: Timers cannot have negative intervals
qrc:/qml/DiscoverPage.qml:17:13: QML InlineMessage: Cannot anchor to an item that isn't a parent or sibling.
adding empty sources model QStandardItemModel(0x55cb41a30c10)
qrc:/qml/DiscoverPage.qml:17:13: QML InlineMessage: Cannot anchor to an item that isn't a parent or sibling.
file:///usr/lib64/qt5/qml/org/kde/kirigami.2/private/globaltoolbar/ToolBarPageHeader.qml:12:1: QML ToolBarPageHeader: Binding loop detected for property "implicitWidth"
qrc:/qml/DiscoverPage.qml:17:13: QML InlineMessage: Cannot anchor to an item that isn't a parent or sibling.
file:///usr/lib64/qt5/qml/org/kde/kirigami.2/private/PrivateActionToolButton.qml:76:5: QML Binding: Binding loop detected for property "value"
kf.newstuff.core: Could not find category "Calligra Flow Stencil"
KNS error in "Calligra_stencils" : KNSCore::ConfigFileError "All categories are missing" QVariant(Invalid)
invalid kns backend! "/etc/xdg/calligra_stencils.knsrc" because: "Invalid Calligra_stencils backend, contact your distributor."
org.kde.plasma.libdiscover: Discarding invalid backend "calligra_stencils.knsrc"
kns error "/etc/xdg/calligra_stencils.knsrc" "Invalid Calligra_stencils backend, contact your distributor."

real    0m49.288s
user    0m3.655s
sys     0m0.535s
tmp$ time plasma-discover  --mode Update
QObject::startTimer: Timers cannot have negative intervals
qrc:/qml/DiscoverPage.qml:17:13: QML InlineMessage: Cannot anchor to an item that isn't a parent or sibling.
adding empty sources model QStandardItemModel(0x558f87d3c760)
qrc:/qml/DiscoverPage.qml:17:13: QML InlineMessage: Cannot anchor to an item that isn't a parent or sibling.
file:///usr/lib64/qt5/qml/org/kde/kirigami.2/private/globaltoolbar/ToolBarPageHeader.qml:12:1: QML ToolBarPageHeader: Binding loop detected for property "implicitWidth"
qrc:/qml/DiscoverPage.qml:17:13: QML InlineMessage: Cannot anchor to an item that isn't a parent or sibling.
file:///usr/lib64/qt5/qml/org/kde/kirigami.2/private/PrivateActionToolButton.qml:76:5: QML Binding: Binding loop detected for property "value"
kf.newstuff.core: Could not find category "Calligra Flow Stencil"
KNS error in "Calligra_stencils" : KNSCore::ConfigFileError "All categories are missing" QVariant(Invalid)
invalid kns backend! "/etc/xdg/calligra_stencils.knsrc" because: "Invalid Calligra_stencils backend, contact your distributor."
org.kde.plasma.libdiscover: Discarding invalid backend "calligra_stencils.knsrc"
kns error "/etc/xdg/calligra_stencils.knsrc" "Invalid Calligra_stencils backend, contact your distributor."

** (process:480155): WARNING **: 10:59:59.997: Found icon of unknown type 'unknown' in 'system/package/os/org.kde.latte-dock.desktop/*', skipping it.

** (process:480155): WARNING **: 11:00:00.015: Found icon of unknown type 'unknown' in 'system/package/os/org.kde.discover.snap/*', skipping it.

** (process:480155): WARNING **: 11:00:00.028: Found icon of unknown type 'unknown' in 'system/package/os/io.github.syllo.nvtop/*', skipping it.

** (process:480155): WARNING **: 11:00:00.178: Found icon of unknown type 'unknown' in 'system/package/os/com.github.maliit.keyboard/*', skipping it.

** (process:480155): WARNING **: 11:00:00.259: Found icon of unknown type 'unknown' in 'system/package/os/org.kde.kiten.desktop/*', skipping it.

real    0m41.332s
user    0m5.073s
sys     0m0.668s


Then the next two times I do the same thing, it takes nearly 40s. But everything is already up to date, nothing needs to be fetched, just connect to the servers, check we're up to date, and finish.

Why does it take 40s to confirm everything is up to date?

I can run all of dnf update, pkcon update, flatpak update and fwupmgr get-updates in 4s, why does it take discover ten times as long?

$ date ; sudo dnf --quiet  update ; fwupdmgr get-updates ; pkcon update ; flatpak update;  date
Sun 15 Jan 11:04:37 GMT 2023
Devices with no available firmware updates: 
 • MZVPV128HDGM-00000
 • SSD 860 EVO 2TB
 • SSD 860 EVO 2TB
 • SSD 860 QVO 2TB
 • System Firmware
 • WDC WD10EZEX-00BN5A0
 • WDC WD10EZEX-00WN4A0
Devices with the latest available firmware version:
 • UEFI dbx
No updates available
Getting updates               [=========================]         
Finished                      [=========================]         
No packages require updating to newer versions.
Looking for updates…

Nothing to do.
Sun 15 Jan 11:04:41 GMT 2023
Comment 12 Jonathan Wakely 2023-01-15 11:22:05 UTC
N.B. it's specifically when hitting the "Refresh" button that it now takes 30-40s