Bug 358563

Summary:	Possible infinite loop in UnsureType::toString()
Product:	[Developer tools] kdev-python	Reporter:	Alex Richardson <arichardson.kde>
Component:	general	Assignee:	Sven Brauch <mail>
Status:	RESOLVED FIXED
Severity:	crash	CC:	mail, p.v.niekerk
Priority:	NOR
Version First Reported In:	git master
Target Milestone:	1.7.3
Platform:	Other
OS:	Linux
Latest Commit:	http://commits.kde.org/kdev-python/bb98418eb0da83257f3adb91baf8630ee808ddec	Version Fixed/Implemented In:
Sentry Crash Report:
Attachments:	potential fix for this bug better fix debug output

Description Alex Richardson 2016-01-26 00:43:00 UTC

When I open subprocess.py kdevelop freezes. The process is stuck in UnsureType::toString() (stack traces points to outline view trying to call toString() on the return type of subprocess.check_call().
This only happens after editing python code kdevelop for a while. Possibly every time a new UnsureType is created this is distinct from all others (even though the the types should be the same) so that the is_new_type() check returns false. It might not be an infinite loop, but I decided to kill KDevelop after about 2 minutes.

When I attached a debugger the types list had over 500 entries, it seems to me like that should not happen.

Reproducible: Sometimes

Comment 1 Sven Brauch 2016-01-26 10:57:19 UTC

Good find, that is certainly the same as bug 355094. I'll look into it now that I have more specific information.

Comment 2 Sven Brauch 2016-01-26 10:57:35 UTC

*** Bug 355094 has been marked as a duplicate of this bug. ***

Comment 3 Sven Brauch 2016-01-27 22:08:28 UTC

Hmm. Can you post the backtrace? Also do you have example code which you edit to trigger this bug?

Comment 4 Alex Richardson 2016-01-28 16:07:29 UTC

I was editing the script here: https://github.com/RichardsonAlex/cheri-scripts

Hasn't happended the last two days, but I have only done minor edits, so I guess the number of UnsureType instances haven't grown enough. I'll post a backtrace next time the freeze happens.
Any suggestions where I should add some qDebug() calls to make this easier to debug?

Comment 5 Sven Brauch 2016-01-28 16:46:20 UTC

Thanks, I'll try that. You could try unsuretype.cpp in kdevplatform, in addType, to see which types are added ... but it will probably be too spammy no matter where you put it.

Comment 6 Sven Brauch 2016-01-30 15:53:15 UTC

I found a possibly related issue, could you maybe apply the patch below and see if you can still reproduce your problem?

Comment 7 Sven Brauch 2016-01-30 15:53:52 UTC

Created attachment 96925 [details]
potential fix for this bug

Comment 8 Sven Brauch 2016-01-30 17:16:39 UTC

Created attachment 96929 [details]
better fix

Actually, try this instead, the other patch causes some new bugs.

Comment 9 Alex Richardson 2016-02-09 17:25:06 UTC

Seems to be working now (at least I haven't had a freeze in the last two days)

I'll reopen in case I run into the problem again.

Comment 10 Sven Brauch 2016-02-09 18:23:21 UTC

Very good, thanks for testing. This is in 5.0, if I don't see it causing other issues I'll try to backport it to 4.7.x as well ...

Comment 11 Sven Brauch 2016-02-09 19:32:32 UTC

apparently not resolved after all, reopening this

Comment 12 Alex Richardson 2016-02-09 20:15:47 UTC

Created attachment 97119 [details]
debug output

Still happening, here is some debug output

Comment 13 Sven Brauch 2016-02-09 20:23:32 UTC

Thanks for the output. I think the reason is that KDevPlatform compares only the index of the type, but that is not at all reliable for unsure types. Instead, the types themselves have to be checked for duplicates.

Comment 14 Alex Richardson 2016-02-10 15:22:09 UTC

I added some debug output into UnsureType::addType() and get the rather confusing:

"unsure (str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str)" 
+
 "int"  
->  "unsure (str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int, str, int)"

UnsureType::addType() checks if a type as already contained, so the int should not be added.

Similarly I also get this:
"unsure (Path, str, Path, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path)" + "str"  ->  "unsure (Path, str, Path, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str, Path, str)"


Possibly the Python types are not being indexed correctly, so that a new distinct IndexedType is being created each time a variable has type str, int, pathlib.Path, etc.

Comment 15 Alex Richardson 2016-02-10 16:45:38 UTC

I am also getting some of these messages:

kdevplatform.language: invalid item for index 35 1478 0

In case it helps: This is the huge unsure type that is being created (it is now so large that outputting the debug info is starting to cause noticeable lags):

"unsure (str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str, (unknown direct declaration)), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str, (unknown direct declaration)), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str, function void (CheriConfig)), list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str, function void (CheriConfig)), list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str, function mixed (CheriConfig)), list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), tuple of ( unsure (str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure ((unknown direct declaration), Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str, (unknown direct declaration)), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str, (unknown direct declaration)), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str, function void (CheriConfig)), list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str, function void (CheriConfig)), list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str, function mixed (CheriConfig)), list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str), str, list of unsure (str, int), unsure (list of void, list), list of str, list of unsure (Path, str)), str, unsure (unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str, unsure (), str), unsure (Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str, Project, str) ), list of mixed)"

Comment 16 Sven Brauch 2016-02-10 20:35:27 UTC

Interesting. Can you maybe print type->indexed()->index() for each type in the list (and also the one which is added)?

Comment 17 Sven Brauch 2016-02-10 23:27:24 UTC

Git commit bb98418eb0da83257f3adb91baf8630ee808ddec by Sven Brauch.
Committed on 10/02/2016 at 23:27.
Pushed by brauch into branch '5.0'.

fix hinted type appearing repeatedly in unsure type list

The check looking whether a type is already in the list of possible
types for an unsure does not work properly for HintedTypes, because
they create a new IndexedType each (due to different modification
revisions).
When a function declaration was imported from another file, this would
leed to arbitrarily large type lists, since on each parse pass of the
importer an entry would be added but the list was never cleared.

M  +1    -1    duchain/declarationbuilder.cpp
M  +5    -72   duchain/helpers.cpp
M  +1    -1    duchain/helpers.h
M  +1    -7    duchain/types/hintedtype.cpp
M  +8    -10   duchain/types/hintedtype.h
M  +19   -0    duchain/types/unsuretype.cpp
M  +18   -14   duchain/types/unsuretype.h

http://commits.kde.org/kdev-python/bb98418eb0da83257f3adb91baf8630ee808ddec

Comment 18 Alex Richardson 2016-02-11 10:52:13 UTC

Thank you for fixing this! However there are still a lot of types being added that in my opinion don't need to exist:

found potential duplicates when adding N6Python10HintedTypeE int(index = 774764) ->
    67278 - int of type N8KDevelop13StructureTypeE
    102814 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    139700 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    140778 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    191576 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    206348 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    346692 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    351636 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    362174 (aka 351724: int of type N6Python10UnsureTypeE) - int of type N6Python10HintedTypeE
    386886 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    408898 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    533652 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    540306 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    573586 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    614644 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    618548 (aka 66042: bool of type N8KDevelop12IntegralTypeE) - bool of type N6Python10HintedTypeE
    648374 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    665250 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    709684 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    750432 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    89504 (aka 66548: str of type N8KDevelop13StructureTypeE) - str of type N6Python10HintedTypeE
    765024 (aka 90024: str of type N6Python10UnsureTypeE) - str of type N6Python10HintedTypeE
    774764 (aka 403436: int of type N6Python10UnsureTypeE) - int of type N6Python10HintedTypeE



I'll submit a patch shortly.

Comment 19 Alex Richardson 2016-02-12 12:49:32 UTC

Git commit bba0f034c4bdfdf9375b7532d134ef05214ce51b by Alex Richardson.
Committed on 12/02/2016 at 12:48.
Pushed by arichardson into branch '5.0'.

Don't add HintedType instances to UnsureType if the target type exists

But only do this if the HintedType also has the same context, otherwise
they might be invalidated at separate times and removing one of them
removes the still valid other HintedType.

If we don't deduplicate here we still get UnsureTypes which have lots
of entries like this:

found potential duplicates when adding N6Python10HintedTypeE int(index = 774764) ->
    67278 - int of type N8KDevelop13StructureTypeE
    102814 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    139700 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    140778 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    191576 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    206348 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    346692 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    351636 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    362174 (aka 351724: int of type N6Python10UnsureTypeE) - int of type N6Python10HintedTypeE
    386886 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    408898 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    533652 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    540306 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    573586 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    614644 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    618548 (aka 66042: bool of type N8KDevelop12IntegralTypeE) - bool of type N6Python10HintedTypeE
    648374 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    665250 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    709684 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    750432 (aka 67278: int of type N8KDevelop13StructureTypeE) - int of type N6Python10HintedTypeE
    89504 (aka 66548: str of type N8KDevelop13StructureTypeE) - str of type N6Python10HintedTypeE
    765024 (aka 90024: str of type N6Python10UnsureTypeE) - str of type N6Python10HintedTypeE
    774764 (aka 403436: int of type N6Python10UnsureTypeE) - int of type N6Python10HintedTypeE

REVIEW: 127039

M  +6    -0    duchain/types/hintedtype.cpp
M  +2    -0    duchain/types/hintedtype.h
M  +38   -8    duchain/types/unsuretype.cpp

http://commits.kde.org/kdev-python/bba0f034c4bdfdf9375b7532d134ef05214ce51b

Comment 20 Sven Brauch 2016-02-12 22:07:05 UTC

Hmm. Running duchainify /usr/lib/python3.5, I still see some very long hangs somewhere around this code (notably processing site-packages/twisted/trial/test/test_deferred.py). Maybe this needs to be investigated even further :(