Bug 449355 - High memory usage when idle
Summary: High memory usage when idle
Status: RESOLVED WORKSFORME
Alias: None
Product: frameworks-baloo
Classification: Frameworks and Libraries
Component: Baloo File Daemon (show other bugs)
Version: 5.90.0
Platform: Manjaro Linux
: NOR normal
Target Milestone: ---
Assignee: baloo-bugs-null
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-01-30 07:51 UTC by Filip Bengtsson
Modified: 2024-07-31 03:46 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
Screenshots (52.50 KB, application/gzip)
2022-01-30 07:51 UTC, Filip Bengtsson
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Filip Bengtsson 2022-01-30 07:51:47 UTC
Created attachment 146065 [details]
Screenshots

SUMMARY
The process baloo_file is sitting on 6.4 Gb of memory, while ksysguard reports the confusing "Status: idle, 75% done" (and has done so for ~20 hours).


STEPS TO REPRODUCE
Not sure if this will reproduce my problem, bit this is what I've done recently:
1. My Windows computer gave up the ghost so I dumped all files from that drive into my downloads folder, and baloo started indexing it. That's 277.3 GB of all kinds of files.
2. After four days it was still indexing and maxing out HDD bandwidth, so I blacklisted the downloads folder (using ksysguard).
3. ksysguard reported that baloo did some index cleanup and now says idle. I let the computer run overnight to see if baloo would release the memory, it hadn't.
4. I tried rebooting, but baloo always ramps up to the same memory usage in a few minutes after startup.

OBSERVED RESULT
Above.

EXPECTED RESULT
I know baloo uses a lot of resources, but I expect at least a message in ksysguard that is more helpful in explaining what's going on; it is clearly doing something and not idling.

SOFTWARE/OS VERSIONS
Linux/KDE Plasma: 5.16.2-1-MANJARO (64 bt)
KDE Plasma Version: 5.23.5
KDE Frameworks Version: 5.90.0
Qt Version: 5.15.2

ADDITIONAL INFORMATION
Baloo 5.90.0
Relevant screenshots attached.
Comment 1 tagwerk19 2022-01-30 14:21:54 UTC
Sooo...

Baloo has "seen" a load of files. That is baloo_file has picked up that you've copied loads to your Downloads folder, that doesn't take long in normal cases, and has queued them for content indexing

You can watch to see what "content indexing" is happening if you run "balooctl monitor". You'll see files being indexed in batches of 40. It's possible that the baloo_file_extractor was choking on one particular file (think a multi-gigabyte mailbox file - but surprises come in all shapes, sizes and forms). During this work it'll be baloo_file_extractor taking the resources.

if you've subsequently disabled indexing of ~/Downloads, baloo_file would see that it has to clean up all those records it had initially created (and the "content" for the 75% of the files you'd managed to index). That's hard work and can mean your index baloons in size, see Bug 437754.

Can be the best solution, as opposed to waiting for 10's of thousands of records to be cleaned from the index, to purge and start again.

I've seen that people in this situation, who have an SSD and a HDD, make sure that their .local/share/baloo folder is on the SSD to avoid all the "seek" work...
Comment 2 tagwerk19 2024-07-01 08:39:23 UTC
There's been a fix about a year ago to limit (through systemd) the amount of RAM Baloo can use:
     https://invent.kde.org/frameworks/baloo/-/merge_requests/121
You might find Baloo is slower, as it tries to fit into the RAM its given, but it will not grab so much that the rest of the system is affected.

There was a following patch:
     https://invent.kde.org/frameworks/baloo/-/merge_requests/148
to make sure the initial scan for changes works within the restricted RAM.

These don't change anything when deleting a large number of files but it should mean that Baloo's "catching up" in the background is not so noticeable.

Are you still haivng problems or is it OK to close this call?
Comment 3 Bug Janitor Service 2024-07-16 03:46:29 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 4 Bug Janitor Service 2024-07-31 03:46:15 UTC
This bug has been in NEEDSINFO status with no change for at least
30 days. The bug is now closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

Thank you for helping us make KDE software even better for everyone!