Bug 423928 - kdirwatch refcounting (stop exhausting fs.inotify.max_user_instances)
Summary: kdirwatch refcounting (stop exhausting fs.inotify.max_user_instances)
Status: RESOLVED FIXED
Alias: None
Product: frameworks-kcoreaddons
Classification: Unclassified
Component: general (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Michael Pyne
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-07-06 14:01 UTC by Harald Sitter
Modified: 2020-09-11 10:57 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In: 5.75


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Harald Sitter 2020-07-06 14:01:35 UTC
If I'm reading the code correctly kdirwatch shares its private instances and puts them into thread-local storage. This really ought to have refcounting in place so the private instance is thrown away when all "fronting" objects have been deleted. Otherwise threads that create temporary KDirWatch objects will create inotify instances that ultimately end up being unused 99% of the time but still counting against the per-user cap on instances. This can somewhat easily exhaust the instance cap.

For example see comment 4 on bug #423818 which I suspect may be at least partially caused by kdirwatch. plasmashell and krunner have a decent amount of instances (entries in the table) but almost all have no active watches (first column) because they are inactive.
Some gdb breaking suggests this is caused by the krunner framework which runs runner processes in threadweaver managed threads. These runners may use KService and thus cause a temporary KDirWatch on the ksycoca database. Since the thread assignment out of the thread pool is random the runner will eventually have run on all possible threads and consequently left behind unused kdirwatchprivate instances on all of them.

KDirWatchPrivate should have refcounting. Once all its associated KDirWatch objects are gone the Private object should delete itself as well (maybe with a second or so delay just in case an application is being silly?).
Comment 1 David Faure 2020-09-07 18:22:25 UTC
Makes sense :-)
Comment 2 Bug Janitor Service 2020-09-10 12:12:31 UTC
A possibly relevant merge request was started @ https://invent.kde.org/frameworks/kcoreaddons/-/merge_requests/25
Comment 3 Harald Sitter 2020-09-11 10:57:29 UTC
Git commit ac616bd02a1c99fc706e244455ed92a4b5e9e320 by Harald Sitter.
Committed on 11/09/2020 at 10:40.
Pushed by dfaure into branch 'master'.

refcount and delete KDirWatchPrivate instances

this adds reference counting for frontend users of KDirWatchPrivate
instances. it allows us to clean up the thread-local private object when
no more frontend objects are left.

this enables heavily threaded applications to release inotify instances
for threads that aren't deleted but instead returned to a pool.
FIXED-IN: 5.75

M  +3    -1    autotests/CMakeLists.txt
M  +20   -0    autotests/kdirwatch_unittest.cpp
M  +22   -1    src/lib/io/kdirwatch.cpp
M  +8    -0    src/lib/io/kdirwatch_p.h

https://invent.kde.org/frameworks/kcoreaddons/commit/ac616bd02a1c99fc706e244455ed92a4b5e9e320