Summary: | preprocessor can run oom on certain constructs [rpp::Stream::operator<<] | ||
---|---|---|---|
Product: | [Applications] kdevelop | Reporter: | Shriram <shriram.uc> |
Component: | Language Support: CPP (old) | Assignee: | kdevelop-bugs-null |
Status: | RESOLVED FIXED | ||
Severity: | crash | CC: | afief.h, aspotashev, david.nolden.kde, dmcennis, job, kfunk, klemensbaum, mail, manveru+bugs.kde, oswald.luc.91, shriram.uc, tim |
Priority: | NOR | ||
Version: | unspecified | ||
Target Milestone: | --- | ||
Platform: | Unlisted Binaries | ||
OS: | Linux | ||
Latest Commit: | Version Fixed In: | ||
Attachments: |
massif output
more massif output New crash information added by DrKonqi massif data (partial) massif file when parsing delay.c via duchainify Backtrace when importing LLVM |
Description
Shriram
2009-11-12 19:51:55 UTC
Is this project available somewhere? Also 14G of memory is (IMHO) too much even for such a project, so this rather looks like there's a leak somewhere. Unfortunately the project is not open-source. I'll could try repro'ing it with other projects. Will report back soon. Hmm, too bad. Maybe I can find an similarly large project within KDE. BTW: Please leave the Component entry as I set it, thanks. There's probably a semi-leak somewhere (Which means: The memory stays referenced, but is not really required). The tool 'massif' helps finding such issues, the problem is that KDevelop runs slowly in it so it's hard to get meaningful results. Can you go into your ~/.kdevduchain/0, type "du", and post the result? (velvety) /home/shriram/.kdevduchain/0 (shriram) :
> du -sch *
2.7M Code Model
0 Code Model_dynamic
5.2M Comment Repository
0 Comment Repository_dynamic
4.0K Counters
0 crash_counter
2.1M Definition Map
0 Definition Map_dynamic
2.6M Environment Information
0 Environment Information_dynamic
2.1M Environment Lists
0 Environment Lists_dynamic
2.2M file modification repository
0 file modification repository_dynamic
2.6M file modification sets
0 file modification sets_dynamic
3.5M Identifier Repository
0 Identifier Repository_dynamic
2.1M Importer Map
0 Importer Map_dynamic
2.1M include path repository
0 include path repository_dynamic
2.3M Instantiation Information Repository
0 Instantiation Information Repository_dynamic
0 lock
5.2M macro repository
4.0K macro repository_dynamic
3.2M macro sets
0 macro sets_dynamic
2.3M Persistent Context Table
0 Persistent Context Table_dynamic
3.0M Persistent Declaration Table
0 Persistent Declaration Table_dynamic
4.7M Qualified Identifier Repository
0 Qualified Identifier Repository_dynamic
3.0M Recursive Imports
0 Recursive Imports_dynamic
4.0M String Index
0 String Index_dynamic
3.4M string sets
0 string sets_dynamic
26M topcontexts
5.4M Type Repository
0 Type Repository_dynamic
2.1M Use Map
0 Use Map_dynamic
0 version_61
91M total
There's also a ~/.kdevduchain/1/...
Here's the du-
(velvety) /home/shriram/.kdevduchain/1 (shriram) :
> du -sch *
5.2M Code Model
4.0K Code Model_dynamic
13M Comment Repository
0 Comment Repository_dynamic
4.0K Counters
0 crash_counter
2.7M Definition Map
0 Definition Map_dynamic
4.1M Environment Information
0 Environment Information_dynamic
2.3M Environment Lists
0 Environment Lists_dynamic
2.2M file modification repository
0 file modification repository_dynamic
5.6M file modification sets
0 file modification sets_dynamic
8.1M Identifier Repository
0 Identifier Repository_dynamic
2.4M Importer Map
0 Importer Map_dynamic
2.1M include path repository
0 include path repository_dynamic
2.7M Instantiation Information Repository
0 Instantiation Information Repository_dynamic
0 lock
7.3M macro repository
4.0K macro repository_dynamic
5.6M macro sets
0 macro sets_dynamic
2.6M Persistent Context Table
0 Persistent Context Table_dynamic
6.7M Persistent Declaration Table
4.0K Persistent Declaration Table_dynamic
14M Qualified Identifier Repository
0 Qualified Identifier Repository_dynamic
6.3M Recursive Imports
4.0K Recursive Imports_dynamic
15M String Index
0 String Index_dynamic
6.9M string sets
0 string sets_dynamic
109M topcontexts
14M Type Repository
0 Type Repository_dynamic
2.2M Use Map
0 Use Map_dynamic
0 version_61
236M total
Here's another crash that I keep running into. Not sure if its related. Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fde3b5fe910 (LWP 29451)] QVector<unsigned int>::realloc (this=0x7fde3b5f8c50, asize=579827905, aalloc=-493913916) at /usr/include/QtCore/qvector.h:471 471 x.d->sharable = true; (gdb) bt 20 #0 QVector<unsigned int>::realloc (this=0x7fde3b5f8c50, asize=579827905, aalloc=-493913916) at /usr/include/QtCore/qvector.h:471 #1 0x00007fde5464c344 in QVector<unsigned int>::append (this=0x7fde3b5f8c50, t=<value optimized out>) at /usr/include/QtCore/qvector.h:525 #2 0x00007fde5464ba74 in rpp::Stream::operator<< (this=0x7fde3b5f86d0, c=@0xffffffff8a3ff000) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-stream.cpp:216 #3 0x00007fde5464d730 in rpp::Stream::operator<< (c=<value optimized out>, this=<value optimized out>) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-stream.h:160 #4 rpp::pp_macro_expander::operator() (c=<value optimized out>, this=<value optimized out>) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-macro-expander.cpp:338 #5 0x00007fde5464e6f0 in rpp::pp_macro_expander::operator() (this=0x7fde3b5f5ca0, input=@0x7fde3b5f5b40, output=@0x7fde3b5f86d0) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-macro-expander.cpp:245 #6 0x00007fde5464f952 in rpp::pp_macro_expander::operator() (this=0x7fde3b5f6790, input=@0x7fde3b5f6610, output=@0x7fde3b5f86d0) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-macro-expander.cpp:546 #7 0x00007fde5464e6f0 in rpp::pp_macro_expander::operator() (this=0x7fde3b5f7240, input=@0x7fde3b5f70e0, output=@0x7fde3b5f86d0) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-macro-expander.cpp:245 #8 0x00007fde5464f952 in rpp::pp_macro_expander::operator() (this=0x7fde3b5f7d10, input=@0x7fde3b5f7bb0, output=@0x7fde3b5f86d0) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-macro-expander.cpp:546 #9 0x00007fde5464f952 in rpp::pp_macro_expander::operator() (this=0x7fde3b5f8800, input=@0x7fde3b5f8680, output=@0x7fde3b5f86d0) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-macro-expander.cpp:546 #10 0x00007fde5464f0cb in rpp::pp_macro_expander::operator() (this=0x7fde3b5f92b0, input=@0x7fde3b5f9150, output=@0x7fde3b5fc870) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-macro-expander.cpp:455 #11 0x00007fde5464f952 in rpp::pp_macro_expander::operator() (this=0x7fde3b5f9d80, input=@0x7fde3b5f9c20, output=@0x7fde3b5fc870) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-macro-expander.cpp:546 #12 0x00007fde5464f952 in rpp::pp_macro_expander::operator() (this=0x7fde3b5fa850, input=@0x7fde3b5fa6f0, output=@0x7fde3b5fc870) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-macro-expander.cpp:546 #13 0x00007fde5464f952 in rpp::pp_macro_expander::operator() (this=0x7fde3b5fb320, input=@0x7fde3b5fb1c0, output=@0x7fde3b5fc870) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-macro-expander.cpp:546 #14 0x00007fde5464f952 in rpp::pp_macro_expander::operator() (this=0x7fde3b5fbdf0, input=@0x7fde3b5fbc90, output=@0x7fde3b5fc870) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-macro-expander.cpp:546 #15 0x00007fde5464f952 in rpp::pp_macro_expander::operator() (this=0x7fde3b5fc9d8, input=@0x7fde3b5fc8c0, output=@0x7fde3b5fc870) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-macro-expander.cpp:546 #16 0x00007fde5465b4b7 in rpp::pp::operator() (this=0x7fde3b5fc9d0, input=@0x7fde3b5fc8c0, output=@0x7fde3b5fc870) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-engine.cpp:260 #17 0x00007fde5465b8dd in rpp::pp::processFileInternal (this=0x7fde3b5fc9d0, fileName=<value optimized out>, fileContents=<value optimized out>, result=@0x7fde3b5fdbd0) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-engine.cpp:97 #18 0x00007fde5465b965 in rpp::pp::processFile (this=0x0, fileName=@0xffffffff8a3ff000, data=@0xffffffffffffffff) at /home/shriram/kdevelop/kdevelop/languages/cpp/parser/rpp/pp-engine.cpp:84 #19 0x00007fde548b1432 in PreprocessJob::run (this=0x30809e0) at /home/shriram/kdevelop/kdevelop/languages/cpp/preprocessjob.cpp:238 (More stack frames follow...) Created attachment 38311 [details]
massif output
I ran kdevelop thru massif. It ran out of space after a while and froze my machine. Had to do a hard reset. But I managed to get couple of profile output files. Noe sure if they're useful. Attaching them here.
Created attachment 38312 [details]
more massif output
Do you use the snippet-plugin? The last massif output indicates a problem in that plugin. Try disabling the snippet-plugin through the settings, and see whether it's better without it. Or better yet update kdevplatform (the snippets plugin), enable it's debug area in "kdebugdialog" and than reproduce the issue. Save the CLI output to a file and send it to us/me so we could try to track down the issue in the snippets plugin. Just by looking at the code I could not find anything obvious :( SVN commit 1049648 by zwabel: Greatly simplify the management of the macros owned by preprocess-environments. This should also close some memory-leaks. CCBUG: 214298 M +0 -1 cppduchain/cppduchain.cpp M +1 -1 cppduchain/cpppreprocessenvironment.cpp M +0 -2 parser/parsesession.cpp M +0 -1 parser/parsesession.h M +1 -12 parser/rpp/pp-engine.cpp M +9 -176 parser/rpp/pp-environment.cpp M +4 -41 parser/rpp/pp-environment.h M +0 -4 parser/rpp/tests/main.cpp M +0 -5 preprocessjob.cpp M +0 -8 tests/test_cppcodecompletion.cpp WebSVN link: http://websvn.kde.org/?view=rev&revision=1049648 Awesome! That sounds promising. I'll give it a whirl. On Sun, Nov 15, 2009 at 8:22 AM, David Nolden < david.nolden.kde@art-master.de> wrote: > https://bugs.kde.org/show_bug.cgi?id=214298 > > > > > > --- Comment #12 from David Nolden <david nolden kde art-master de> > 2009-11-15 17:22:36 --- > SVN commit 1049648 by zwabel: > > Greatly simplify the management of the macros owned by > preprocess-environments. > This should also close some memory-leaks. > > CCBUG: 214298 > > M +0 -1 cppduchain/cppduchain.cpp > M +1 -1 cppduchain/cpppreprocessenvironment.cpp > M +0 -2 parser/parsesession.cpp > M +0 -1 parser/parsesession.h > M +1 -12 parser/rpp/pp-engine.cpp > M +9 -176 parser/rpp/pp-environment.cpp > M +4 -41 parser/rpp/pp-environment.h > M +0 -4 parser/rpp/tests/main.cpp > M +0 -5 preprocessjob.cpp > M +0 -8 tests/test_cppcodecompletion.cpp > > > WebSVN link: http://websvn.kde.org/?view=rev&revision=1049648 > > -- > Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email > ------- You are receiving this mail because: ------- > You are on the CC list for the bug. > You reported the bug. > Yep try it out. But it only adresses the smaller issue visible in the first massif outputs. The main problem though seems to be the snippet support, which was not fixed yet. See the suggestions before. Yeah. I'll try disabling the snippets plugin. Thanks! -Shriram On Sun, Nov 15, 2009 at 9:55 AM, David Nolden < david.nolden.kde@art-master.de> wrote: > https://bugs.kde.org/show_bug.cgi?id=214298 > > > > > > --- Comment #14 from David Nolden <david nolden kde art-master de> > 2009-11-15 18:55:10 --- > Yep try it out. But it only adresses the smaller issue visible in the first > massif outputs. The main problem though seems to be the snippet support, > which > was not fixed yet. See the suggestions before. > > -- > Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email > ------- You are receiving this mail because: ------- > You are on the CC list for the bug. > You reported the bug. > Shriram - disabling the plugin won't fix the bug. And since we cannot reproduce it so far, some more answers to the questions I posted above would help us a lot. Created attachment 67243 [details]
New crash information added by DrKonqi
kdevelop (4.2.60) on KDE Platform 4.7.4 (4.7.4) using Qt 4.7.4
- What I was doing when the application crashed:
kdevelop consistently crashes when parsing the full source tree of the boost c++ libraries from trunk
-- Backtrace (Reduced):
#7 0x00007f4b830f46fb in QVector<unsigned int>::realloc (this=0x7f4b31d24c80, asize=310377811, aalloc=536870911) at /usr/include/qt4/QtCore/qvector.h:490
#8 0x00007f4b830f4912 in QVector<unsigned int>::append (this=0x7f4b31d24c80, t=<optimized out>) at /usr/include/qt4/QtCore/qvector.h:549
#9 0x00007f4b830f40bb in rpp::Stream::operator<< (this=0x7f4b31d24450, c=<optimized out>) at /home/tim/hd2t2/sources/kdevelop/languages/cpp/parser/rpp/pp-stream.cpp:211
#10 0x00007f4b830f4b1c in rpp::Stream::operator<< (this=<optimized out>, c=<optimized out>) at /home/tim/hd2t2/sources/kdevelop/languages/cpp/parser/rpp/pp-stream.h:164
#11 0x00007f4b830f65c9 in rpp::pp_macro_expander::operator() (this=0x7f4b31d24690, input=..., output=..., substitute=true, table=0x0) at /home/tim/hd2t2/sources/kdevelop/languages/cpp/parser/rpp/pp-macro-expander.cpp:344
sources are: boost-trunk from svn, r76171 https://svn.boost.org/svn/boost/trunk Created attachment 67249 [details]
massif data
data obtained by running massif for about an hour
kdevelop dies when parsing http://www.boost.org/doc/libs/1_36_0/libs/preprocessor/doc/examples/delay.c which is the reason for tim's issue. I'll try to massif that file and see what happens. still valid Created attachment 67254 [details]
(partial) massif file when parsing delay.c via duchainify
I'm getting a very similar crash when importing the LLVM sources. KDevelop crashes with a similar backtrace after eating all my available memory (8GB). I'll attach the backtrace. Created attachment 74623 [details]
Backtrace when importing LLVM
*** Bug 314198 has been marked as a duplicate of this bug. *** *** Bug 329699 has been marked as a duplicate of this bug. *** *** Bug 336732 has been marked as a duplicate of this bug. *** *** Bug 336963 has been marked as a duplicate of this bug. *** *** Bug 339920 has been marked as a duplicate of this bug. *** Hello! We are working on a new clang-based C/C++ language plugin for KDevelop 5 which supersedes the old C++ plugin in KDevelop 4. See e.g.: https://www.kdevelop.org/news/first-beta-release-kdevelop-500-available Due to a lack of manpower, we cannot fix bugs in the old C++ plugin. We rather want to supply a good Clang based C++ experience for KDevelop 5 than wasting our time on the legacy C++ support for KDevelop 4. With the new clang-based C/C++ language plugin, the bug presented here does not occur. In my testing. For these reasons, I'll close this bug. Please stay tuned for KDevelop 5. If you think this bug is applicable to Clang/KDevelop 5, please reopen the report and add new information on how to reproduce the bug there. |