Bug 464517 - Discover crashes on start unless .cache/discover/ deleted
Summary: Discover crashes on start unless .cache/discover/ deleted
Status: RESOLVED FIXED
Alias: None
Product: Discover
Classification: Applications
Component: discover (show other bugs)
Version: 5.26.90
Platform: openSUSE Linux
: VHI crash
Target Milestone: ---
Assignee: Plasma Bugs List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-01-19 19:03 UTC by Fabian Vogt
Modified: 2023-10-24 14:04 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In: 5.27.9


Attachments
Terminal output when starting Discover (18.90 KB, text/plain)
2023-01-21 17:55 UTC, P. Förster
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Fabian Vogt 2023-01-19 19:03:54 UTC
The first time I started discover it worked fine. I closed it, but after that then it always crashes on start.
After deleting .cache/discover it works fine again. This is 100% reproducible, discover only starts with empty cache.

linux@localhost:~> plasma-discover
fetch ratings! false
adding empty sources model QStandardItemModel(0x56351c74a350)
no component found for "org.opensuse.opensuse-tumbleweed"
parseList():: XML Error:  "Premature end of document." 
In xml name "" with text "" at offset:
 0 
In XML:
 ""
KNS error in "Plasma Widgets" : KNSCore::OcsError "Unknown Open Collaboration Service API error. (0)" QVariant(int, 0)
invalid kns backend! "/usr/share/knsrcfiles/plasmoids.knsrc" because: "Invalid Plasma Widgets backend, contact your distributor."
org.kde.plasma.libdiscover: Discarding invalid backend "plasmoids.knsrc"
kns error "/usr/share/knsrcfiles/plasmoids.knsrc" "Invalid Plasma Widgets backend, contact your distributor."
KNS error in "Plasma Styles" : KNSCore::OcsError "Unknown Open Collaboration Service API error. (0)" QVariant(int, 0)
invalid kns backend! "/usr/share/knsrcfiles/plasma-themes.knsrc" because: "Invalid Plasma Styles backend, contact your distributor."
org.kde.plasma.libdiscover: Discarding invalid backend "plasma-themes.knsrc"
org.kde.plasma.libdiscover: last stream isn't over yet Filters(category: Category(0x56351c7d9dd0, name = "Games"),) ResourcesProxyModel(0x56351d2639b0)
org.kde.plasma.libdiscover: last stream isn't over yet Filters(category: Category(0x56351c7d39d0, name = "Developer Tools"),) ResourcesProxyModel(0x56351d2622e0)
kns error "/usr/share/knsrcfiles/plasma-themes.knsrc" "Invalid Plasma Styles backend, contact your distributor."
KNS error in "System monitor Sensor Display Styles" : KNSCore::OcsError "Unknown Open Collaboration Service API error. (0)" QVariant(int, 0)
invalid kns backend! "/usr/share/knsrcfiles/systemmonitor-faces.knsrc" because: "Invalid System monitor Sensor Display Styles backend, contact your distributor."
org.kde.plasma.libdiscover: Discarding invalid backend "systemmonitor-faces.knsrc"
org.kde.plasma.libdiscover: last stream isn't over yet Filters(category: Category(0x56351c7d9dd0, name = "Games"),) ResourcesProxyModel(0x56351d2639b0)
org.kde.plasma.libdiscover: last stream isn't over yet Filters(category: Category(0x56351c7d39d0, name = "Developer Tools"),) ResourcesProxyModel(0x56351d2622e0)
kns error "/usr/share/knsrcfiles/systemmonitor-faces.knsrc" "Invalid System monitor Sensor Display Styles backend, contact your distributor."
(continues for all knsrcs)
[Thread 0x7fffc6ffd6c0 (LWP 5568) exited]

Thread 1 "plasma-discover" received signal SIGSEGV, Segmentation fault.
0x00007fffcc6c31eb in PackageKitBackend::resolvePackages(QStringList const&) () from /usr/lib64/qt5/plugins/discover/packagekit-backend.so
(gdb) bt
#0  0x00007fffcc6c31eb in PackageKitBackend::resolvePackages(QStringList const&) () from /usr/lib64/qt5/plugins/discover/packagekit-backend.so
#1  0x00007fffcc6cae88 in ?? () from /usr/lib64/qt5/plugins/discover/packagekit-backend.so
#2  0x00007ffff5f7ec50 in QObject::event(QEvent*) () from /lib64/libQt5Core.so.5
#3  0x00007ffff78db44e in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /lib64/libQt5Widgets.so.5
#4  0x00007ffff5f53138 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /lib64/libQt5Core.so.5
#5  0x00007ffff5f560d1 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /lib64/libQt5Core.so.5
#6  0x00007ffff5fab363 in ?? () from /lib64/libQt5Core.so.5
#7  0x00007ffff45e2a90 in g_main_context_dispatch () from /lib64/libglib-2.0.so.0
#8  0x00007ffff45e2e48 in ?? () from /lib64/libglib-2.0.so.0
#9  0x00007ffff45e2edc in g_main_context_iteration () from /lib64/libglib-2.0.so.0
#10 0x00007ffff5faab66 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /lib64/libQt5Core.so.5
#11 0x00007ffff5f51bab in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /lib64/libQt5Core.so.5
#12 0x00007ffff5f59d16 in QCoreApplication::exec() () from /lib64/libQt5Core.so.5
#13 0x000055555556d2a0 in ?? ()
#14 0x00007ffff58725b0 in __libc_start_call_main () from /lib64/libc.so.6
#15 0x00007ffff5872679 in __libc_start_main_impl () from /lib64/libc.so.6
#16 0x000055555556dc75 in ?? ()
(gdb) q

For reference, a working start of discover:

linux@localhost:~> rm -rf .cache/discover/
linux@localhost:~> plasma-discover
fetch ratings! true
adding empty sources model QStandardItemModel(0x560fc9cae630)
org.kde.plasma.discover: couldn't open file "/home/linux/.cache/discover/featured-5.9.json" "No such file or directory"
no component found for "org.opensuse.opensuse-tumbleweed"
qrc:/qml/Feedback.qml:44: ReferenceError: UserFeedbackSettings is not defined
file:///usr/lib64/qt5/qml/org/kde/kirigami.2/ScrollablePage.qml:200:9: QML MouseArea: Binding loop detected for property "implicitHeight"
file:///usr/lib64/qt5/qml/org/kde/kirigami.2/ScrollablePage.qml:200:9: QML MouseArea: Binding loop detected for property "implicitHeight"
file:///usr/lib64/qt5/qml/org/kde/kirigami.2/ScrollablePage.qml:200:9: QML MouseArea: Binding loop detected for property "implicitHeight"
file:///usr/lib64/qt5/qml/org/kde/kirigami.2/ScrollablePage.qml:200:9: QML MouseArea: Binding loop detected for property "implicitHeight"
Comment 1 Fabian Vogt 2023-01-21 13:14:16 UTC
The cause is that the cache for https://api.kde-look.org/ocs/v1/content/categories contains only metadata, but not any actual data:

.cache/discover/attica/data8/2:
total 4
-rw------- 1 linux linux 718 Jan 21 07:56 9hw8n262.d

This is how it should like with a full GET request:

/tmp/qtnamcache/data8/2:
insgesamt 28
-rw------- 1 fabian users 26575 21. Jan 14:13 9hw8n262.d

The crash is not directly related, appears to be some use-after-free:

==4859== Invalid read of size 8
==4859==    at 0x68E1BF0: QObjectPrivate::maybeSignalConnected(unsigned int) const (in /usr/lib64/libQt5Core.so.5.15.8)
==4859==    by 0x68F004D: ??? (in /usr/lib64/libQt5Core.so.5.15.8)
==4859==    by 0x49363F1: ResultsStream::resourcesFound(QVector<AbstractResource*> const&) (moc_AbstractResourcesBackend.cpp:159)
==4859==    by 0x28443DE8: UnknownInlinedFun (PackageKitBackend.cpp:532)
==4859==    by 0x28443DE8: UnknownInlinedFun (PackageKitBackend.cpp:521)
==4859==    by 0x28443DE8: UnknownInlinedFun (PackageKitBackend.cpp:626)
==4859==    by 0x28443DE8: UnknownInlinedFun (invoke.h:61)
==4859==    by 0x28443DE8: UnknownInlinedFun (invoke.h:111)
==4859==    by 0x28443DE8: std::_Function_handler<void (), PackageKitBackend::search(AbstractResourcesBackend::Filters const&)::{lambda()#9}>::_M_invoke(std::_Any_data const&) [clone .lto_priv.0] (std_function.h:290)
==4859==    by 0x68E4C4F: QObject::event(QEvent*) (in /usr/lib64/libQt5Core.so.5.15.8)
==4859==    by 0x4B9344D: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.15.8)
==4859==    by 0x68B9137: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib64/libQt5Core.so.5.15.8)
==4859==    by 0x68BC0D0: QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) (in /usr/lib64/libQt5Core.so.5.15.8)
==4859==    by 0x6911362: ??? (in /usr/lib64/libQt5Core.so.5.15.8)
==4859==    by 0x81A6A8F: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.7400.4)
==4859==    by 0x81A6E47: ??? (in /usr/lib64/libglib-2.0.so.0.7400.4)
==4859==    by 0x81A6EDB: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.7400.4)
==4859==  Address 0xd00b420 is 64 bytes inside a block of size 85 free'd
==4859==    at 0x484717B: free (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==4859==    by 0x8F93524: xmlFreeNodeList (in /usr/lib64/libxml2.so.2.10.3)
==4859==    by 0x8F93634: xmlFreeNode (in /usr/lib64/libxml2.so.2.10.3)
==4859==    by 0x82CE5BA: ??? (in /usr/lib64/libappstream.so.0.15.6)
==4859==    by 0x82CEB07: ??? (in /usr/lib64/libappstream.so.0.15.6)
==4859==    by 0x82F6EFB: ??? (in /usr/lib64/libappstream.so.0.15.6)
==4859==    by 0x82CB1C4: as_cache_get_components_by_categories (in /usr/lib64/libappstream.so.0.15.6)
==4859==    by 0x82EDBAF: as_pool_get_components_by_categories (in /usr/lib64/libappstream.so.0.15.6)
==4859==    by 0x7199E1B: AppStream::Pool::componentsByCategories(QStringList const&) const (in /usr/lib64/libAppStreamQt.so.0.15.6)
==4859==    by 0x4974E13: AppStreamUtils::componentsByCategories(AppStream::Pool*, Category*, AppStream::Bundle::Kind) (AppStreamUtils.cpp:259)
==4859==    by 0x28443A7A: UnknownInlinedFun (PackageKitBackend.cpp:618)
==4859==    by 0x28443A7A: UnknownInlinedFun (invoke.h:61)
==4859==    by 0x28443A7A: UnknownInlinedFun (invoke.h:111)
==4859==    by 0x28443A7A: std::_Function_handler<void (), PackageKitBackend::search(AbstractResourcesBackend::Filters const&)::{lambda()#9}>::_M_invoke(std::_Any_data const&) [clone .lto_priv.0] (std_function.h:290)
==4859==    by 0x68E4C4F: QObject::event(QEvent*) (in /usr/lib64/libQt5Core.so.5.15.8)
==4859==  Block was alloc'd at
==4859==    at 0x48447B4: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==4859==    by 0x8FE3552: xmlStrndup (in /usr/lib64/libxml2.so.2.10.3)
==4859==    by 0x8F8CC5B: xmlNewTextLen (in /usr/lib64/libxml2.so.2.10.3)
==4859==    by 0x8F8CCCD: xmlNewDocTextLen (in /usr/lib64/libxml2.so.2.10.3)
==4859==    by 0x8F98A7F: xmlNodeAddContentLen (in /usr/lib64/libxml2.so.2.10.3)
==4859==    by 0x82C8C52: ??? (in /usr/lib64/libappstream.so.0.15.6)
==4859==    by 0x82C8D58: ??? (in /usr/lib64/libappstream.so.0.15.6)
==4859==    by 0x82C8D58: ??? (in /usr/lib64/libappstream.so.0.15.6)
==4859==    by 0x82C8D58: ??? (in /usr/lib64/libappstream.so.0.15.6)
==4859==    by 0x82CE58F: ??? (in /usr/lib64/libappstream.so.0.15.6)
==4859==    by 0x82CEB07: ??? (in /usr/lib64/libappstream.so.0.15.6)
==4859==    by 0x82F6EFB: ??? (in /usr/lib64/libappstream.so.0.15.6)
==4859== 
==4859== Invalid read of size 8
==4859==    at 0x68E1C00: QObjectPrivate::maybeSignalConnected(unsigned int) const (in /usr/lib64/libQt5Core.so.5.15.8)
==4859==    by 0x68F004D: ??? (in /usr/lib64/libQt5Core.so.5.15.8)
==4859==    by 0x49363F1: ResultsStream::resourcesFound(QVector<AbstractResource*> const&) (moc_AbstractResourcesBackend.cpp:159)
==4859==    by 0x28443DE8: UnknownInlinedFun (PackageKitBackend.cpp:532)
==4859==    by 0x28443DE8: UnknownInlinedFun (PackageKitBackend.cpp:521)
==4859==    by 0x28443DE8: UnknownInlinedFun (PackageKitBackend.cpp:626)
==4859==    by 0x28443DE8: UnknownInlinedFun (invoke.h:61)
==4859==    by 0x28443DE8: UnknownInlinedFun (invoke.h:111)
==4859==    by 0x28443DE8: std::_Function_handler<void (), PackageKitBackend::search(AbstractResourcesBackend::Filters const&)::{lambda()#9}>::_M_invoke(std::_Any_data const&) [clone .lto_priv.0] (std_function.h:290)
==4859==    by 0x68E4C4F: QObject::event(QEvent*) (in /usr/lib64/libQt5Core.so.5.15.8)
==4859==    by 0x4B9344D: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.15.8)
==4859==    by 0x68B9137: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib64/libQt5Core.so.5.15.8)
==4859==    by 0x68BC0D0: QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) (in /usr/lib64/libQt5Core.so.5.15.8)
==4859==    by 0x6911362: ??? (in /usr/lib64/libQt5Core.so.5.15.8)
==4859==    by 0x81A6A8F: g_main_context_dispatch (in /usr/lib64/libglib-2.0.so.0.7400.4)
==4859==    by 0x81A6E47: ??? (in /usr/lib64/libglib-2.0.so.0.7400.4)
==4859==    by 0x81A6EDB: g_main_context_iteration (in /usr/lib64/libglib-2.0.so.0.7400.4)
==4859==  Address 0x6e69687469772075 is not stack'd, malloc'd or (recently) free'd
Comment 2 P. Förster 2023-01-21 17:55:48 UTC
Created attachment 155478 [details]
Terminal output when starting Discover

I can reproduce this on a fresh install of KDE neon Testing.

SYSTEM INFORMATION
Operating System: KDE neon Testing Edition
KDE Plasma Version: 5.26.90
KDE Frameworks Version: 5.102.0
Qt Version: 5.15.8
Kernel Version: 5.15.0-58-generic (64-bit)
Graphics Platform: X11
Processors: 16 × AMD Ryzen 7 5700U with Radeon Graphics
Memory: 15.0 GiB of RAM
Graphics Processor: AMD Radeon Graphics
Manufacturer: LENOVO
Product Name: 82LM
System Version: IdeaPad 5 14ALC05
Comment 3 P. Förster 2023-01-21 18:00:48 UTC
I can also confirm that after deleting .cache/discover Discover starts as expected again.
Comment 4 Fabian Vogt 2023-01-21 20:27:55 UTC
> The crash is not directly related, appears to be some use-after-free:

Indeed. The bug with categories caching causes ResourcesProxyModel::invalidateFilter() to be called for all resources/streams which calls "delete" on them. This happens while the backends are still busy fetching stuff for those streams, so they end up calling ResultsStream::resourcesFound on dangling pointers.
Comment 5 Fabian Vogt 2023-01-21 22:35:41 UTC
It's a bug in plasma-desktop's attica-kde plugin triggering a bug in QNetworkDiskCache:

     const QString cacheDir = QStandardPaths::writableLocation(QStandardPaths::CacheLocation) + QStringLiteral("/attica");
     QNetworkDiskCache *cache = new QNetworkDiskCache(m_accessManager);
     QStorageInfo storageInfo(cacheDir);
     cache->setCacheDirectory(cacheDir);
     cache->setMaximumCacheSize(storageInfo.bytesTotal() / 1000);

QStorageInfo returns -1 there if the cache directory didn't exist yet. QNetworkDiskCache happily accepts 0 as max size and eagerly deletes everything it can on every occasion.

This happens in the worst possible spot, QNetworkDiskCachePrivate::storeItem:

void QNetworkDiskCachePrivate::storeItem(QCacheItem *cacheItem)
{
...
    currentCacheSize = q->expire();
    if (!cacheItem->file) {
        QString templateName = tmpCacheFileName();
        cacheItem->file = new QTemporaryFile(templateName, &cacheItem->data);
        if (cacheItem->file->open()) {
            cacheItem->writeHeader(cacheItem->file);
            cacheItem->writeCompressedData(cacheItem->file);
        }
    }

    if (cacheItem->file
        && cacheItem->file->isOpen()
        && cacheItem->file->error() == QFile::NoError) {
        cacheItem->file->setAutoRemove(false);
        // ### use atomic rename rather then remove & rename
        if (cacheItem->file->rename(fileName))
            currentCacheSize += cacheItem->file->size();
        else
            cacheItem->file->setAutoRemove(true);
    }

cacheItem->file is the temporary file with the cached contents. q->expire() however notices that the cache is too big and deletes it, setting cacheItem->file to nullptr. This unfortunately triggers the if block below, which is only meant for compressed caches, which are backed by cacheItem->data instead of cacheItem->file. The categories cache will not be compressed, which means that cacheItem->data is empty and it writes no actual data into the new file.
Comment 6 Fabian Vogt 2023-01-21 22:40:00 UTC
The in hindsight ridiculously simple fix for the attica-kde bug: https://invent.kde.org/plasma/plasma-desktop/-/merge_requests/1343
Comment 7 Fushan Wen 2023-01-22 09:27:52 UTC
Git commit 63788e6520a64a0846e4ee5ff0f2693730604986 by Fushan Wen, on behalf of Fabian Vogt.
Committed on 22/01/2023 at 09:22.
Pushed by fusionfuture into branch 'master'.

attica-kde: Use the correct location for determining the max cache size

QStorageInfo returns -1 there if the cache directory didn't exist yet.
QNetworkDiskCache happily accepts 0 as max size and eagerly deletes
everything it can on every occasion, triggering cache corruption.

M  +1    -1    attica-kde/kdeplugin/kdeplatformdependent.cpp

https://invent.kde.org/plasma/plasma-desktop/commit/63788e6520a64a0846e4ee5ff0f2693730604986
Comment 8 Fushan Wen 2023-01-22 09:28:18 UTC
Git commit 884e1988a064ec132b351d4f9b1a272d3627829c by Fushan Wen, on behalf of Fabian Vogt.
Committed on 22/01/2023 at 09:28.
Pushed by fusionfuture into branch 'cherry-pick-63788e65'.

attica-kde: Use the correct location for determining the max cache size

QStorageInfo returns -1 there if the cache directory didn't exist yet.
QNetworkDiskCache happily accepts 0 as max size and eagerly deletes
everything it can on every occasion, triggering cache corruption.


(cherry picked from commit 63788e6520a64a0846e4ee5ff0f2693730604986)

M  +1    -1    attica-kde/kdeplugin/kdeplatformdependent.cpp

https://invent.kde.org/plasma/plasma-desktop/commit/884e1988a064ec132b351d4f9b1a272d3627829c
Comment 9 Fushan Wen 2023-01-22 09:34:23 UTC
Git commit ec5cc12b4c4119672acd95680e897ac96e573ee8 by Fushan Wen, on behalf of Fabian Vogt.
Committed on 22/01/2023 at 09:34.
Pushed by fusionfuture into branch 'cherry-pick-63788e65-2'.

attica-kde: Use the correct location for determining the max cache size

QStorageInfo returns -1 there if the cache directory didn't exist yet.
QNetworkDiskCache happily accepts 0 as max size and eagerly deletes
everything it can on every occasion, triggering cache corruption.


(cherry picked from commit 63788e6520a64a0846e4ee5ff0f2693730604986)

M  +1    -1    attica-kde/kdeplugin/kdeplatformdependent.cpp

https://invent.kde.org/plasma/plasma-desktop/commit/ec5cc12b4c4119672acd95680e897ac96e573ee8
Comment 10 Fabian Vogt 2023-01-22 09:41:37 UTC
I'll report the QNetworkDiskCache bug to Qt soon.

https://bugs.kde.org/show_bug.cgi?id=463864 has the same backtrace as the triggered UAF, so I think that can particular bug can continue there.
Comment 11 Bug Janitor Service 2023-10-23 12:42:29 UTC
A possibly relevant merge request was started @ https://invent.kde.org/plasma/discover/-/merge_requests/671
Comment 12 Aleix Pol 2023-10-24 00:32:26 UTC
Git commit 90e271b96974adb009e27570855d7040d1b3cea7 by Aleix Pol Gonzalez, on behalf of David Edmundson.
Committed on 24/10/2023 at 02:29.
Pushed by apol into branch 'master'.

Fix network cache size for first run

QNetworkDiskCache will take care of creating a directory if it doesn't
exist. QStorageInfo on a directory that doesn't exist will return that
it has 0 bytesTotal.

As well as being potentially inefficient for the second run, it has been
reported that a QNetworkDiskCache of 0 causes out of bounds memory
writes.

M  +1    -1    libdiscover/CachedNetworkAccessManager.cpp

https://invent.kde.org/plasma/discover/-/commit/90e271b96974adb009e27570855d7040d1b3cea7
Comment 13 Bug Janitor Service 2023-10-24 08:07:20 UTC
A possibly relevant merge request was started @ https://invent.kde.org/plasma/discover/-/merge_requests/675
Comment 14 David Edmundson 2023-10-24 08:08:51 UTC
Git commit b98e9209b84473fcaa158d6a2d0710a51e3e43c3 by David Edmundson.
Committed on 24/10/2023 at 10:06.
Pushed by davidedmundson into branch 'Plasma/5.27'.

Fix network cache size for first run

QNetworkDiskCache will take care of creating a directory if it doesn't
exist. QStorageInfo on a directory that doesn't exist will return that
it has 0 bytesTotal.

As well as being potentially inefficient for the second run, it has been
reported that a QNetworkDiskCache of 0 causes out of bounds memory
writes.


(cherry picked from commit 90e271b96974adb009e27570855d7040d1b3cea7)

M  +1    -1    libdiscover/CachedNetworkAccessManager.cpp

https://invent.kde.org/plasma/discover/-/commit/b98e9209b84473fcaa158d6a2d0710a51e3e43c3