Bug 379551

this sounds odd - do you have many tabs open?

in general, how do you track memory consumption? can you try a heap profiler like heaptrack to see what happens and what consumes memory?

does anything happen when you do this once RSS memory consumption is reported as high:

gdb --pid $(pidof kdevelop)
...
(gdb) call KDevelop::DUChain::self()->storeToDisk()

Created attachment 105746 [details]
Heaptrack results

I'm really sorry for not replying sooner, but the defense of my Master's thesis is soon and I just do not have the time to experiment with KDevelop extensively.

First of all - tabs. I usually have 6-8 tabs open. Sometimes more, sometimes less, but it does not seem to influence memory consumption all that much.

Next, I ran KDevelop with heaptrack in terminal, using:
heaptrack ./KDevelop.AppImage

I'm attaching the results file. Unfortunately, I'm not sure if it's complete or not because I had to kill KDevelop. The moment I noticed the system slowing down, I hit the close button on the KDevelop window. It closed and the memory use stopped increasing. However, the kdevelop process remained alive, maxing out a single CPU core. I waited for around 20 minutes, hoping for it to finish, but it did not and I had to kill it. That's when heaptrack finished as well and produced the heaptrack.KDevelop.AppImage.5390.gz that I'm attaching.


Admittedly, I forgot to do this:
gdb --pid $(pidof kdevelop)
...
(gdb) call KDevelop::DUChain::self()->storeToDisk()

I'll do it once I have more free time on my hands.

I've also noticed that the terminal that I used to start KDevelop was fillied with hundreds of these messages:

kdevplatform.shell: Broken text-document:  QUrl("file:///tmp/kdevelop_Pa3204.patch")

Plenty of these messages were in the terminal as well:

kdevelop.plugins.clang: Something went wrong during 'clang_codeCompleteAt' for file "ONE_OF_MY_CURRENTTLY_OPEN_PROJECT_FILES"
libclang: crash detected in code completion

The way AppImage works makes it impossible to track kdevelop with heaptrack currently. So you'll either have to runtime attach heaptrack later on (heaptrack --pid $(pidof kdevelop), assuming an AppImage client application can be found that way).

And don't worry with the delay, we too have lots of stuff to do. So take your time and concentrate on your thesis. Good luck!

Created attachment 110818 [details]
htop

htop screenshot for appimage high memory use

Created attachment 110819 [details]
heaptrack(2018-02-20)

heaptrack.kdevelop.17748.gz

Hi, I have the same issue.

My system info is

No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux 9.3 (stretch)
Release:	9.3
Codename:	stretch

Linux ddzt 4.9.0-5-amd64 #1 SMP Debian 4.9.65-3+deb9u2 (2018-01-04) x86_64 GNU/Linux

I am using KDevelop-5.2.1-x86_64.AppImage

I did this "heaptrack --pid $(pidof kdevelop)", I killed the kdevelop manually after a few mins.

https://bugs.kde.org/attachment.cgi?id=110818
https://bugs.kde.org/attachment.cgi?id=110819

that heaptrack trace is pretty meaningless - it only shows stuff in deflate and not a lot of memory consumption in total. Are you sure the pid it attached to is the one you saw in htop? Also, just to ensure, please build heaptrack from sources, there was a bug in the attach mode which may explain the lack of data: https://phabricator.kde.org/R45:779dceb8cbe77a2d653d5244126a80e0d5d5346f

Hi, I am sorry, the htop screenshot and the heaptrack not create at the same time.

I am keep running heaptrack with kdevelop today, but seem not happened today.
(Maybe I cleaned the cache, when I open kdevelop there have a dialog)
The final value are VIRT 9122M, RES 2857M, SHR 444M.

(Before, the VIRT can reach to 15G+)

I will keep tracking. If I have a good track file, I will put it here.

Thank you very much!

Also, I build heaptrack from this commit
https://github.com/KDE/heaptrack/commit/6e31841708b351b9f53e59d384e94df014d927ed

It's happened today, I just open the kdevelop and left it there.
Several hours later, I come back my desk, try to do something.
I feel my computer have a little freeze (cursor will stuck a few second)
Then I check the htop, I saw   VIRT 14G+, RES 10G+ (around 1min, RES reduce to 8G+, then I feel the computer is come back)

I click the kdevelop's close button, but still running. So I pressed "ctrl+c" 

I will put the screenshot and heaptrack here. Hope it's can help you.

THX

Created attachment 110870 [details]
heaptrack(2018-02-21)

Created attachment 110871 [details]
htop(2018-02-21)

Created attachment 110872 [details]
after_click_close_btn

Sorry, The size of heaptrack.kdevelop.22591.gz file is 7.8 MB, So I compress it to 7z file.

That file is equally uninteresting and also only shows deflate stuff. Heaptrack only saw a memory peak of 2.7 *mega*byte, so I guess attaching heaptrack to the AppImage really doesn't work at all...

One more thing you could try:

- unpack the appimage with --appimage-extract
- go into the squashfs-root folder, edit the AppRun script to run kdevelop through heaptrack
- launch the AppRun script

maybe we get more information that way?

I will try it. Thanks.

Hey there,

I was just notified about this:

https://codereview.qt-project.org/#/c/218906/

So maybe that's the issue you are facing. Do you see messages about crashes on the command line output?

Hi,

I don't see any "crash" keywords in terminal.

Also, I found if I did the "clear cache" when launch kdevelop.

Even run 10+ hours, the max memory usage VIRT 8922M,  RES 3493M, SHR 32440

At least, my computer don't freeze.


I can put the log file here.

Hi,

I don't see any "crash" keywords in terminal.

Also, I found if I did the "clear cache" when launch kdevelop.

Even run 10+ hours, the max memory usage VIRT 8922M,  RES 3493M, SHR 32440

At least, my computer don't freeze.


I can put the log file here.

Created attachment 111039 [details]
kdevelop log

Created attachment 112098 [details]
Heaptrack file (PID 12200)

Created attachment 112099 [details]
KDevelop log (PID 12200)

First and last 1000 lines (full log is 45 MB)

I have been testing the KDevelop 5.2.1 AppImage on Debian 7.8 (wheezy) for the last few weeks (our office is still using KDevelop 4.7.4, which runs great, albeit a bit slow with large projects, and it would be nice to move to the latest). For the most part, KDevelop 5.2.1 is runs along with no problems (sometimes for one to two days at a time), but occasionally it starts eating away at memory rapidly, to the point where my desktop starts freezing up and I have to type Ctrl-Alt-F1 to switch to console mode and kill the kdevelop process. My workstation has 32 GB RAM, and after I kill the process, usually 20 GB or more becomes available again.

The last few days I have been launching KDevelop with the latest heaptrack (built from master). Rather than launching the AppRun script, I simply set the environment variables from my shell (csh), and launch KDevelop with:

heaptrack kdevelop >& kdevelop_log.txt

This afternoon I encountered the memory leak once again. Managed to select Session > Quit from KDevelop, and though the window disappeared, the kdevelop process (PID 12200) kept running and consuming additional memory. Eventually I sent two SIGTERM signals to kill it. I have attached the heaptrack file and the first and last 1000 lines of the log file. Hopefully there is some helpful information here.

Martin, how did you run heaptrack? The track file only sees 27.7MB and it only recorded for 9.7 seconds. Did it really only run for such a short time?

Strange, heaptrack was running the kdevelop session for around eight hours (and kdevelop was running smoothly for most of this time, using a stable amount of memory when I checked a couple times during the first few hours). A few additional details I forgot to mention yesterday:
* I unpacked the AppImage and ran kdevelop from the usr/bin/ directory within.
* The default Debian 7 gcc (v4.7.2) is too old to build heaptrack, so I used a custom install of the gcc v4.8.5 compiler and runtime for the heaptrack session (kdevelop exhibits the memory leak whether it uses the gcc v4.8.5 runtime or the default v4.7.2 runtime).
* When the kdevelop process refused to quit (and continued eating up memory), I sent SIGTERM to the kdevelop process (not the heaptrack process). 

Here are all the commands I ran (from csh):

set DIR="/scratch/home/mfuhrer/sw/db7/src/kdevelop-5.2.1"
setenv LD_PRELOAD $DIR/exec_wrapper.so
setenv APPIMAGE_ORIGINAL_QML2_IMPORT_PATH ""
setenv APPIMAGE_ORIGINAL_LD_LIBRARY_PATH $LD_LIBRARY_PATH
setenv APPIMAGE_ORIGINAL_QT_PLUGIN_PATH $QT_PLUGIN_PATH
setenv APPIMAGE_ORIGINAL_XDG_DATA_DIRS $XDG_DATA_DIRS
setenv APPIMAGE_ORIGINAL_PATH $PATH
setenv APPIMAGE_ORIGINAL_PYTHONHOME ""
setenv QML2_IMPORT_PATH $DIR/usr/lib/qml
setenv LD_LIBRARY_PATH $DIR/usr/lib/:${LD_LIBRARY_PATH}
setenv QT_PLUGIN_PATH $DIR/usr/lib/qt5/plugins/
setenv XDG_DATA_DIRS $DIR/usr/share/:${XDG_DATA_DIRS}
setenv PATH $DIR/usr/bin:${PATH}
setenv KDE_FORK_SLAVES 1
setenv PYTHONHOME $DIR/usr/
setenv APPIMAGE_STARTUP_QML2_IMPORT_PATH $QML2_IMPORT_PATH
setenv APPIMAGE_STARTUP_LD_LIBRARY_PATH $LD_LIBRARY_PATH
setenv APPIMAGE_STARTUP_QT_PLUGIN_PATH $QT_PLUGIN_PATH
setenv APPIMAGE_STARTUP_XDG_DATA_DIRS $XDG_DATA_DIRS
setenv APPIMAGE_STARTUP_PATH $PATH
setenv APPIMAGE_STARTUP_PYTHONHOME $PYTHONHOME
setenv KDEV_DISABLE_PLUGINS KDevWelcomePage
setenv LD_LIBRARY_PATH /vega/xp/ThirdParty/gcc/db7/xprel/lib64:${LD_LIBRARY_PATH}
cd $DIR/usr/bin
heaptrack kdevelop >& kdevelop_log.txt

The last line was executing for about eight hours (until I killed kdevelop). Could the track file be truncated because I killed the kdevelop process? Is there some way to monitor the track file during the heaptrack session? Unfortunately I don't have KDE 5, so can't build the heaptrack GUI to verify the track file.

You can try `heaptrack_print heaptrack.*.gz |& less` - it's CLI only and doesn't depend on any KF5 libs.

The commands look harmless, I wonder what the issue is here...

Just tried with the steps you provided, and I can reproduce the heaptrack bug which is now covered there: https://bugs.kde.org/show_bug.cgi?id=393387

Let's continue the heaptrack investigation there and then once that is fixed we can get back to the KDevelop issue here.

ok Martin, heaptrack should now work properly! please try again. and thanks a lot for the report, this was quite a nasty issue!

Created attachment 112317 [details]
KDevelop log (PID 16665)

KDevelop messages printed to the terminal.

Created attachment 112318 [details]
KDevelop backtrace (PID 16665)

Backtrace generated when I tried to quit KDevelop during memory leak.

Hi Milian, thanks for the heaptrack fix. It appears to be working correctly now. I encountered the memory leak again this morning (workstation started freezing up), and tried to quit KDevelop. I waited several minutes until KDevelop crashed and attached the log and backtrace. The heaptrack file is very large so I've posted it for download from a third party site:
https://mega.nz/#!sdMXxZBT!Pxv1UGEqS2ef73Q39IgtELcneWzwXHpgszxYeyBAmaw

Reviewing the log it appears that there is some issue while libclang is parsing a source file. I am curious if this is what triggers the leak. I will try some more tests on my end...

It could potentially be the same reason as the one explained here:

https://codereview.qt-project.org/#/c/218906/

i.e. for some reason clang crashes, then we essentially leak memory...

The clang issue in Milian's previous comment appears to have been resolved. I noticed that the KDevelop 5.3.0 AppImage is distributed with libclang.so.6 (KDevelop 5.2.x was using libclang.so.5).

Good news is, I have been trying out KDevelop 5.3.0 for a week and haven't encountered the memory leak anymore.