Bug 386513

I had some problems attaching the output as the log file was too large. I am pasting some errors which were repeated a lot in the output.

==1626== 387,712 bytes in 8 blocks are possibly lost in loss record 37,323 of 37,327
==1626==    at 0x4C2F286: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1626==    by 0x4C2F3C2: posix_memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1626==    by 0x1F5775BF: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F5E68DF: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F631F7E: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F620AF9: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F61F4CF: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F82F7C2: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F7F3BC7: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1E1BB561: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626==    by 0x1E18A6C8: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626==    by 0x1E1BC3B7: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626== 
==1626== 434,720 bytes in 1,672 blocks are possibly lost in loss record 37,324 of 37,327
==1626==    at 0x4C2CE5F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1626==    by 0x1F7373ED: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F7374B8: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F73854F: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F9BF685: brw_vec4_alloc_reg_set (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F9158AB: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F85D8A9: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F7F400F: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1E1BBA0E: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626==    by 0x1E18F7B5: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626==    by 0x1E18B462: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626==    by 0x1E18C751: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626== 
==1626== 562,976 bytes in 1,928 blocks are possibly lost in loss record 37,325 of 37,327
==1626==    at 0x4C2CE5F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1626==    by 0x1F7373ED: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F7374B8: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F73854F: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F965D22: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F9667AD: brw_fs_alloc_reg_sets (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F9158A2: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F85D8A9: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F7F400F: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1E1BBA0E: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626==    by 0x1E18F7B5: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626==    by 0x1E18B462: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626== 
==1626== 3,060,192 bytes in 1 blocks are possibly lost in loss record 37,326 of 37,327
==1626==    at 0x4C2F286: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1626==    by 0x4C2F3C2: posix_memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1626==    by 0x1F577613: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F633164: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F67049A: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F82F7DC: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F7F3BC7: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1E1BB561: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626==    by 0x1E18A6C8: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626==    by 0x1E1BC3B7: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626==    by 0x1B8D1118: QGLXContext::init(QXcbScreen*, QPlatformOpenGLContext*) (qglxintegration.cpp:280)
==1626==    by 0x1B8CF31E: QXcbGlxIntegration::createPlatformOpenGLContext(QOpenGLContext*) const (qxcbglxintegration.cpp:190)
==1626== 
==1626== 17,580,048 bytes in 1 blocks are possibly lost in loss record 37,327 of 37,327
==1626==    at 0x4C2CE5F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1626==    by 0x1F641DBA: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F82F7A6: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1F7F3BC7: ??? (in /usr/lib/xorg/modules/dri/i965_dri.so)
==1626==    by 0x1E1BB561: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626==    by 0x1E18A6C8: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626==    by 0x1E1BC3B7: ??? (in /usr/lib/libGLX_mesa.so.0.0.0)
==1626==    by 0x1B8D1118: QGLXContext::init(QXcbScreen*, QPlatformOpenGLContext*) (qglxintegration.cpp:280)
==1626==    by 0x1B8CF31E: QXcbGlxIntegration::createPlatformOpenGLContext(QOpenGLContext*) const (qxcbglxintegration.cpp:190)
==1626==    by 0x18093602: QXcbIntegration::createPlatformOpenGLContext(QOpenGLContext*) const (qxcbintegration.cpp:244)
==1626==    by 0x81DC35F: QOpenGLContext::create() (qopenglcontext.cpp:612)
==1626==    by 0x5AFBAAB: ??? (in /usr/lib/libQt5Quick.so.5.9.2)
==1626== 
==1626== LEAK SUMMARY:
==1626==    definitely lost: 776 bytes in 77 blocks
==1626==    indirectly lost: 0 bytes in 0 blocks
==1626==      possibly lost: 27,960,050 bytes in 35,480 blocks
==1626==    still reachable: 7,007,244 bytes in 64,908 blocks
==1626==                       of which reachable via heuristic:
==1626==                         newarray           : 325,168 bytes in 257 blocks
==1626==                         multipleinheritance: 17,976 bytes in 66 blocks
==1626==         suppressed: 0 bytes in 0 blocks
==1626== Reachable blocks (those to which a pointer was found) are not shown.
==1626== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==1626== 
==1626== For counts of detected and suppressed errors, rerun with: -v
==1626== ERROR SUMMARY: 30181 errors from 24883 contexts (suppressed: 0 from 0)

Happens to me too. A simple calculation like log(6)/log(4) make a giant spike in CPU (+80% for a few seconds) usage and memory usage also increases to > 500 MiB after this.

So I ran some tests on this. The leak does not happen (or is very slow) when no runners are enabled. However, I am able to trigger it with either the 'applications' (launch applications) runner, or the 'windows' (find open windows) runner alone enabled. I noticed that the speed of the memory usage increase seemes to increase with the number of results returned by krunner. For example, if I had lots of windows open, and I typed a query that would result in a large number of results being returned (like typing 'kons' with 20 or so Konsole windows open), the memory usage would jump by roughly or more 5 MB each time.

This is with krunner 5.12.4, and Qt 5.10.1.

So I ran some tests on this. The leak does not happen (or is very slow) when no runners are enabled. However, I am able to trigger it with either the 'applications' (launch applications) runner, or the 'windows' (find open windows) runner alone enabled. I noticed that the speed of the memory usage increase seemes to increase with the number of results returned by krunner. For example, if I had lots of windows open, and I typed a query that would result in a large number of results being returned (like typing 'kons' with 20 or so Konsole windows open), the memory usage would jump by roughly or more 5 MB each time.

This is with krunner 5.12.4, and Qt 5.10.1.

Sorry for the double comment. I double clicked the 'save changes' button, and I got some error about detecting a 'mid air collision', and I pressed the 'submit my new comment' button, assuming it would send only the new comment.

Unfortunately still an issue :/

I write a script which opens krunner 500 times and just types krun and the memory usage increased from 55MB to 80MB.

I will look into this further, but I also have a lot of other bugfixes/feature requests currently going on.

And considering that these plugins are also used in the Kicker/Kickoff/Dashboard launchers this most likely affects the plasmashell process too.

*** Bug 410126 has been marked as a duplicate of this bug. ***

*** Bug 397164 has been marked as a duplicate of this bug. ***

heaptrack sees a rather large leak in qsql/sqlite as used by kactivites and bookmarks. I'm not quite sure who's at fault. With those plugins disabled per-query leaking becomes rather small and I haven't bothered finding what leaks in the kilobyte range.

Can you upload your script @alex?

At least the bookmarks runner does eventually cap out (for repeated krun queries anyway). What appears to happen is that every thread gets its own sql connection and they each have about a 1-2 MiB heap. Terribly inefficient and particularly troublesome with the huge amount of runner threads we can have these days. These appear to also not get shut down again. So mem consumption doesn't go down once krunner is idle again :S

>What appears to happen is that every thread gets its own sql connection and they each have about a 1-2 MiB heap.
I had the same suspicion. The thread usage is always problem when having a bit more complicated runners. A possible solution could be to use a D-Bus runner. This way we are on the safe side and could also use better caching and thus avoid making SQL queries when not needed.

Especially since the runner could be loaded in the kickoff/kicker launcher and KRunner at the same time we could do a more agressive caching without keeping the same data in memory.

Better caching could also be beneficial to reduce disk write operations(this has been extreme because of the combination of two bugs that have been fixed).

And as you(@sitter) know we have big plans for the D-Bus runners, but the current state is already more than sufficient to port this runner.

@davidedmundson What do you think about this?

There's still some kind of memory leaking in 5.23.2. The memory usage climbed up to gigabytes over time. I haven't been tracking what might cause it; normally it seems to not do it all too much after all.

>heaptrack sees a rather large leak in qsql/sqlite as used by kactivites and bookmarks

Activities was taken care of with https://invent.kde.org/plasma/kactivitymanagerd/-/merge_requests/11, the bookmarks runner will need to get ported too. I will do that in the next weeks.

This leak nowadays its very huge. Krunner grows to hundreds and thousands MB. How can I recollect useful evidence?

Regards.

I don't know if this can be useful.

https://youtu.be/YG9Zo1B8bPc