STEPS TO REPRODUCE 1. Have a Flatpak installation with ~50 pending updates 2. Open Discover, select "Update" and click "Update all" OBSERVED RESULT The application crashes. The exact stack trace and journal messages are different each time, but they are all variations of "Too many open files": plasma-discover[3235]: Creating pipes for GWakeup: Too many open files plasma-discover[4090]: Error spawning revokefs-fuse: Too many open files plasma-discover[3518]: Cannot create repo on revokefs mountpoint /var/tmp/flatpak-cache-T1LSA2/org.gnome.Platform.Locale-SQUKA2: opening repo: While checking parent repository '/var/lib/flatpak/repo': opening repo: opendir(tmp): Too many open files There is also a large number of /usr/lib/revokefs-fuse processes running. EXPECTED RESULT All refs are updated correctly. SOFTWARE/OS VERSIONS KDE Plasma Version: 5.27.5 KDE Frameworks Version: 5.107.0 Qt Version: 5.19.5 ADDITIONAL INFORMATION Having a look at the discover Flatpak backend I see that FlatpakBackend::installApplication() is called in a loop for every application that needs to be updated: https://invent.kde.org/plasma/discover/-/blob/v5.27.5/libdiscover/resources/StandardBackendUpdater.cpp?ref_type=tags#L65 Each one of these creates a new FlatpakJobTransaction, each with its own FlatpakTransactionThread which in turn has its own FlatpakTransaction object: https://invent.kde.org/plasma/discover/-/blob/v5.27.5/libdiscover/backends/FlatpakBackend/FlatpakTransactionThread.cpp?ref_type=tags#L101 In other words, instead of creating one FlatpakTransaction and putting all refs in it it creates one transaction per ref and (as far as I can see) runs them all in parallel. I haven't confirmed it yet but I suspect that the problem is related to this.
A possibly relevant merge request was started @ https://invent.kde.org/plasma/discover/-/merge_requests/630
Git commit db0ebc855517f189f64c1602a5d27e185cf02833 by Harald Sitter. Committed on 13/09/2023 at 09:13. Pushed by sitter into branch 'master'. flatpak: make FlatpakTransactionThread a qrunnable instead we can't just have an unlimited number of threads for flatpak transactions. it'd eventually cause excessive load on both CPU and network to the point where things will start misbehaving. we also run risk of exhausting other software limited resources such as file descriptors. to resolve this problem we now treat the transactionthread as runnable and put it in a limited threadpool for concurrent execution. the new runnable has a finished signal that is emitted on every return from run() to match the QThread API. concurrency is limited to no more than 4 runnables at a time. that should still be plenty concurrent while generally unexpected to exhaust the default 1024 file descriptor limit - an install transaction appears to weigh between 60 and 100 fds other backends don't necessarily have this problem since they have daemons that do the work for us so we have way fewer open fds for them. M +30 -5 libdiscover/backends/FlatpakBackend/FlatpakJobTransaction.cpp M +6 -2 libdiscover/backends/FlatpakBackend/FlatpakTransactionThread.cpp M +4 -1 libdiscover/backends/FlatpakBackend/FlatpakTransactionThread.h https://invent.kde.org/plasma/discover/-/commit/db0ebc855517f189f64c1602a5d27e185cf02833
Git commit 46d14515c3105e4e318d28db41057d9f1df3ce4d by Harald Sitter. Committed on 13/09/2023 at 09:19. Pushed by sitter into branch 'Plasma/5.27'. flatpak: make FlatpakTransactionThread a qrunnable instead we can't just have an unlimited number of threads for flatpak transactions. it'd eventually cause excessive load on both CPU and network to the point where things will start misbehaving. we also run risk of exhausting other software limited resources such as file descriptors. to resolve this problem we now treat the transactionthread as runnable and put it in a limited threadpool for concurrent execution. the new runnable has a finished signal that is emitted on every return from run() to match the QThread API. concurrency is limited to no more than 4 runnables at a time. that should still be plenty concurrent while generally unexpected to exhaust the default 1024 file descriptor limit - an install transaction appears to weigh between 60 and 100 fds other backends don't necessarily have this problem since they have daemons that do the work for us so we have way fewer open fds for them. (cherry picked from commit db0ebc855517f189f64c1602a5d27e185cf02833) M +29 -5 libdiscover/backends/FlatpakBackend/FlatpakJobTransaction.cpp M +7 -2 libdiscover/backends/FlatpakBackend/FlatpakTransactionThread.cpp M +4 -1 libdiscover/backends/FlatpakBackend/FlatpakTransactionThread.h https://invent.kde.org/plasma/discover/-/commit/46d14515c3105e4e318d28db41057d9f1df3ce4d