Bug 73671

A few things:

1. How much RAM does your box have?

2. Are you sure the parsing is the problem?

3. Make sure you run "kdevelop" and not "gideon" anymore. If you are using the latter you are running remnants of a pre-beta1 install.

I did some testing on this. I used a P4/1800/512 box and the 2.6.0 linux sources.

The first parsing is very slow. It took about 1,5 hours on my box.

Starting up the second time, when the project PCS is used, took about 3,5 minutes. 

Once the sources are parsed, interacting with the classview is no slower than normal, it feels quite snappy.

Memory consumption is always hard to measure, but VmRss was reported as 350+ Mb. I think the memory consumption as such is a lesser problem than the very slow initial parsing. I guess parsing should indeed be made optional somehow.

I made kdevelop parse Linux 2.4.22 Kernel Sources. At about 80% i attached a gdb to it. This results somewhere doing a heap sort down in QListView called from ProblemReporter.
Letting it run on and giving it a SIGINT some time later, results in the same. I did sent it several SIGINTs and always ended somewhere in that heapsort.

Looking at cppsupportpart.cpp lines 121 to 130, CppDriver::parsedFile. The problems of the recently parsed file are removed from ProblemReporter and reinserted there. #ifdef 0ing those 9 lines, makes kdevelop parse linux kernel at about 20% of the time it took with those lines.

Still: Even when loading the parseroutput from a pcs file, it seems that it gets slower the more it has read.
Roberto, is there something other than the ProblemReporter ListView which gets sorted on the fly?

I did the same test as Sascha and came to about the same results. On my box parsing of the 2.6.0 sources shrank from 1,5h to about 15 min.

We both think it makes sense to not update the problem reporter during the initial import of a new project. This trivial patch fixes that:

RCS file: /home/kde/kdevelop/languages/cpp/cppsupportpart.h,v
retrieving revision 1.77
diff -u -r1.77 cppsupportpart.h
--- cppsupportpart.h	14 Nov 2003 16:21:25 -0000	1.77
+++ cppsupportpart.h	29 Jan 2004 01:51:48 -0000
@@ -62,7 +62,7 @@
     QString specialHeaderName( bool local=false ) const;
     void updateParserConfiguration();
 
-    ProblemReporter* problemReporter() { return m_problemReporter; }
+    ProblemReporter* problemReporter() { return isValid() ? m_problemReporter : 0; }
     BackgroundParser* backgroundParser() { return m_backgroundParser; }
     CppCodeCompletion* codeCompletion() { return m_pCompletion; }
     CppCodeCompletionConfig* codeCompletionConfig() { return m_pCompletionConfig; }

What do you think, Roberto? :)

this patch saved my life, on huuudge project import (over 500MB of sources, about 9000 files) it takes about 10h to do 25% of parsing with orginall version, patch makes it 2 minutes for 25%. And it does it well.
I don't know what is stopping you guys from commiting that kinda improvments!
it still eats memory thou, but If you want to have access to all that data instantly, there is no way around it.
I.e. VC 6 and .net one is parsing only current file, which is in editor, and does class parsing, but only brief. Still using loads of ram thou ;)
finnal words : please committ this change, even if it is not a nicest one.

Roberto can you comment on it?

I doubt there is any point in keeping this report open. The major problem was the loooong time it took to do the initial parsing of a project, which the patch solved. 

The problem with the parser taking loads of RAM will not be solved in kdev3, but hopefully in kdev4 (with a new parser).

Closing.