Bug 384876 - project import inefficiency
Summary: project import inefficiency
Status: RESOLVED WORKSFORME
Alias: None
Product: kdevelop
Classification: Applications
Component: general (other bugs)
Version First Reported In: unspecified
Platform: Compiled Sources All
: NOR normal
Target Milestone: ---
Assignee: kdevelop-bugs-null
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-09-20 11:26 UTC by RJVB
Modified: 2022-12-10 05:14 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed/Implemented In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description RJVB 2017-09-20 11:26:47 UTC
These are observations by Sven Brauch taken from a phabricator review. Not taking credit, nor taking the challenge to address the reported issue just yet. I may in due time if no one more qualified beats me to it.

The observations probably pertain to the cmake project plugin, in "CMake Server" mode (which is roughly 4x slower than the json-based importer in my own benchmarks).

>>>>>>

It indeed takes ages to import the project, and I couldn't figure out why ... it is certainly not CPU load. I looked in perf, the CPU time spent in importing the project is quite low.
In fact, the code model generation (which seems to be the last step in the import AFAIU) finishes relatively quickly here (after something like 5 seconds), but then it still takes like 30 seconds for the import to finish. I don't know where that time is spent.

This is a bug, something is waiting too long for something else. Let's find the bug instead of introducing checkboxes to work around it.

Here's the two things that are broken and need fixing:

1. FileManagerListJob has a queue of directories to list. It processes those sequentially, and after listing each, it goes through the event loop (async slot invocation) to list the next. This takes aaaaaages for lots of directories to list, which are mostly spent waiting and not in actual CPU time. When doing this synchronously as a test, it takes maybe half a second instead of 30 seconds.
2. The reason this is noticeable now is that for some reason, it lists all the build dirs. This is probably a bug in the new cmake importer, they should not be listed.

Fixing the second point would solve the problem, but fixing the first one somehow (just making it sync isn't a solution, but there is certainly a better way) would also be very nice since it will further speed up the import.
The first point is IMO the much worse bug, since that is what will cause large project imports (think clang) be very slow in either case. For completely no reason whatsoever.
Comment 1 Justin Zobel 2022-11-10 08:52:15 UTC
Thank you for reporting this issue in KDE software. As it has been a while since this issue was reported, can we please ask you to see if you can reproduce the issue with a recent software version?

If you can reproduce the issue, please change the status to "REPORTED" when replying. Thank you!
Comment 2 Bug Janitor Service 2022-11-25 05:17:44 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 3 Bug Janitor Service 2022-12-10 05:14:50 UTC
This bug has been in NEEDSINFO status with no change for at least
30 days. The bug is now closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

Thank you for helping us make KDE software even better for everyone!