User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.34 (KHTML, like Gecko) rekonq Safari/534.34 Build Identifier: I'm currently working on a large c++ project. Kdevelop interface freezes during linking process, nothing happens until linking has ended. Reproducible: Always On my sistem, for each message line produced by compiler, ProcessLineMaker::slotReadyReadStdout() is triggered multiple times, bringing to gui freeze when messages are longer than usual, as it happens in my linking step, where a large number of object files are involved.
Update: the freeze happens when MakeOutputModel::addLines() is called passing it a large output string (~16.000 symbols, on my use case). makeoutputmodel.cpp:222 is called a time for each of the available errorFormats. Regular expression evaluation, which is known to be quite slow in current qt implementation, is called several times, thus causing interface freeze.
I have some ideas on how to resolve that, but I'm laking a test case... do you have an idea how to test this? maybe with a hand-written make file that uses e.g. echo with many arguments? if you could provide such a test project it would help me tremendously!
Git commit 12a0e9327119533a07dbbe1d6af833acd4de0ad4 by Milian Wolff. Committed on 29/03/2012 at 22:28. Pushed by mwolff into branch '4.3'. add unit test to benchmark makeoutputmodel M +2 -1 projectbuilders/makebuilder/CMakeLists.txt M +9 -13 projectbuilders/makebuilder/makeoutputmodel.cpp M +11 -1 projectbuilders/makebuilder/makeoutputmodel.h A +4 -0 projectbuilders/makebuilder/tests/CMakeLists.txt A +50 -0 projectbuilders/makebuilder/tests/test_makeoutputmodel.cpp [License: LGPL (v2+)] A +37 -0 projectbuilders/makebuilder/tests/test_makeoutputmodel.h [License: LGPL (v2+)] http://commits.kde.org/kdevelop/12a0e9327119533a07dbbe1d6af833acd4de0ad4
Git commit 0a916158c33a5bd2d79c9a4cfaf2521850852af4 by Milian Wolff. Committed on 30/03/2012 at 02:19. Pushed by mwolff into branch '4.3'. prevent UI-lockup when adding lots of stdout lines to make output model we now add batches of 50 lines and then return to the event loop, continuing with the rest of the lines via a QueuedConnection the unit test shows that (on my machine) the event loop is roughly returning every 80ms. the test ensures that this time does not cross the threshold of 200ms which is imo too much. on a further note, it is interesting to see that optimizing the actual adding of lines is not easily possible, the tons of regexp matching is simply required... the only possible way would be to make the error patterns configurable such that e.g. those for unused compilers or unused toolchains are not queried. M +19 -3 projectbuilders/makebuilder/makeoutputmodel.cpp M +6 -0 projectbuilders/makebuilder/makeoutputmodel.h M +22 -0 projectbuilders/makebuilder/tests/test_makeoutputmodel.cpp http://commits.kde.org/kdevelop/0a916158c33a5bd2d79c9a4cfaf2521850852af4
Created attachment 70031 [details] a linking string
Nice, but it doesn't fix my particular use case. test_makeoutputmodel.cpp, benchAddLines(), insert into lines just one string, the one in the attachment, try to run the test. UI lockup jumps FAR beyond 200 ms! I'm trying to fix this by running ErrorFormat and ActionFormat regexps evaluation in a different thread.
I found out that, in my particular use case, most of the time is spent in the evaluation of the first two ErrorFormat::errorFormats regular expressions. If I comment them out, time spent inside MakeOutputModel::addLineBatch() falls from 400 to 0.01 msecs. Since the patch you committed some days ago seems quite useless in my use case, I'd reopen this issue and try to optimize those regular expressions!
yes thanks for reopening and giving me some useful data. I'll try to take a look at it again as well now.
Git commit bed56dbc23ef04fc86d7a7d3054ffbf69b080152 by Milian Wolff. Committed on 04/04/2012 at 18:25. Pushed by mwolff into branch 'master'. add benchmark that tests performance of makeoutputmodel for really long linker lines it fails currently M +44 -0 projectbuilders/makebuilder/tests/test_makeoutputmodel.cpp M +1 -0 projectbuilders/makebuilder/tests/test_makeoutputmodel.h http://commits.kde.org/kdevelop/bed56dbc23ef04fc86d7a7d3054ffbf69b080152
an easy fix: add a ^ to the start of the two slow regexps. I'm thinking of commiting that to solve the issue. afaiks, gcc outputs one error per line and the errors always start at the beginning of the line
Git commit 8c736770ae3ddd173dbe0cce4479c277f75df7ff by Milian Wolff. Committed on 04/04/2012 at 19:00. Pushed by mwolff into branch 'master'. greatly improve performance of our compiler error-detection regexps errors start at the beginning of a line, by adding this information to the regexps (^ at the beginning) they can perform much better for very long lines which can occur during linking of large projects also adds a unit test with some error messages I found on the net to "proof" it's still working as intended M +6 -6 projectbuilders/makebuilder/outputfilters.cpp M +53 -0 projectbuilders/makebuilder/tests/test_makeoutputmodel.cpp M +3 -0 projectbuilders/makebuilder/tests/test_makeoutputmodel.h http://commits.kde.org/kdevelop/8c736770ae3ddd173dbe0cce4479c277f75df7ff
Why don't fix this issue inside 4.3 branch, too?