Bug 374097 - Kdevelop 5 crashes when editing python file
Summary: Kdevelop 5 crashes when editing python file
Status: RESOLVED FIXED
Alias: None
Product: kdev-python
Classification: Developer tools
Component: Language support (show other bugs)
Version: 5.0.3
Platform: Appimage Linux
: NOR crash
Target Milestone: ---
Assignee: Sven Brauch
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-12-23 19:04 UTC by Jonathan Verner
Modified: 2017-01-19 21:06 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:


Attachments
The file leading to the crash. (1.62 KB, text/x-python)
2016-12-23 19:04 UTC, Jonathan Verner
Details
Modified AppRun script to run kdevelop in gdb (2.89 KB, application/x-shellscript)
2016-12-27 16:33 UTC, Jonathan Verner
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Verner 2016-12-23 19:04:57 UTC
Created attachment 102962 [details]
The file leading to the crash.

When editing the attached file and entering `c` into the `cls.add_bundle` call
on line `63` kdevelop immediately crashes on me. Sometimes, it prints the following
lines on the console:

> doing pep8 checking
> invalid pep8 error line: ""

In general, editing this file leads to frequent crashes. Unfortunately, I am not sure how to get a backtrace with the appimage, but if someone helps me, I can provide it.
Comment 1 Sven Brauch 2016-12-23 19:12:26 UTC
Thanks for the precise report. I can't reproduce it though ... no crash here. I just notice it's very laggy with code style checking enabled (really need to fix that).

Can you reproduce the issue you describe every time? Can you still reproduce it after closing kdevelop and removing ~/.cache/kdevduchain?
Comment 2 Jonathan Verner 2016-12-26 20:29:29 UTC
> Can you reproduce the issue you describe every time? 
> Can you still reproduce it > after closing kdevelop and removing 
> ~/.cache/kdevduchain?

That is weird. It crashed on me reliably every time. But now it does not. I don't remember, but I might have restarted my computer in the meantime which would lead to clearing the ~/.cache directory...

It probably makes sense to close the bug now, since I can't reproduce it either :-)
Comment 3 Sven Brauch 2016-12-26 20:30:25 UTC
Rebooting doesn't clear .cache, but kdevelop does it at some point after a crash. I'll close this, please reopen if it happens again.
Comment 4 Jonathan Verner 2016-12-26 20:36:20 UTC
> Rebooting doesn't clear .cache

In my setup it does :-)

> but kdevelop does it at some point after a crash. 

That might have happened too :-)

> I'll close this, please reopen if it happens again.

I just hit a similar crash (also with the doing pep8 checking and invalid pep8 error line message) editing a different file. Opening the file again lead to a crash, but after deleting the cache, the crash went away...
and starting kdevelop again fixed it.
Comment 5 Jonathan Verner 2016-12-26 20:38:04 UTC
(In reply to Jonathan Verner from comment #4)

> ...
> and starting kdevelop again fixed it.

ignore that last line, it doesn't make grammatical sense with the previous lines and I forgot to delete it.
Comment 6 Sven Brauch 2016-12-26 21:12:00 UTC
Clearing .cache is a major performance impact with KDevelop, you'll have to wait for ages until it rebuilds its code representation cache ... just so you know ;)
Comment 7 Jonathan Verner 2016-12-27 14:17:43 UTC
Hmm, the crash seems to happen quite regularly for different files and projects. It is always preceded by the `invalid pep8 error line: ""` message. It also only seems to happen when I have the file opened in a project. I didn't (yet) encounter the crash when I open just the given file. Is there a way to get a backtrace from the appimage? I'd be happy to investigate this further... 

I am reopening the bug for now, but if you think it is hopeless, feel free to close it again :-)
Comment 8 Sven Brauch 2016-12-27 14:19:02 UTC
You can rename the AppImage to .iso, then unpack it with ark. In the "AppRun" script, in the last line, put "gdb kdevelop" instead of "kdevelop", then run that script; then it will run in gdb and you should be able to get a backtrace.
Comment 9 Jonathan Verner 2016-12-27 16:31:51 UTC
(In reply to Sven Brauch from comment #8)
> You can rename the AppImage to .iso, then unpack it with ark. In the
> "AppRun" script, in the last line, put "gdb kdevelop" instead of "kdevelop",
> then run that script; then it will run in gdb and you should be able to get
> a backtrace.

Nice. I needed to play with it a bit (gdb needs to run in a different
environment from the one setup by the AppRun script for kdevelop, otherwise it just crashes when trying to load kdevelop's python which is different from the one it is linked against). I am attaching a modified AppRun in case it is useful for someone. If the script is passed "--run_in_gdb" as its first argument,
it runs kdevelop using gdb; otherwise it just runs kdevelop normally.

I was able to get the following backtrace:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff99814700 (LWP 26149)]
0x00007fff99858f4f in Python::Helper::declarationForName(KDevelop::QualifiedIdentifier const&, KDevelop::RangeInRevision const&, KDevelop::DUChainPointer<KDevelop::DUContext const>) ()
   from /tmp/kdd/usr/lib/libkdevpythonduchain.so
(gdb) #0  0x00007fff99858f4f in Python::Helper::declarationForName(KDevelop::QualifiedIdentifier const&, KDevelop::RangeInRevision const&, KDevelop::DUChainPointer<KDevelop::DUContext const>) ()
   from /tmp/kdd/usr/lib/libkdevpythonduchain.so
#1  0x00007fff9984e900 in Python::ExpressionVisitor::visitName(Python::NameAst*) ()
   from /tmp/kdd/usr/lib/libkdevpythonduchain.so
#2  0x00007fff9984eaf3 in Python::ExpressionVisitor::visitAttribute(Python::AttributeAst*) ()
   from /tmp/kdd/usr/lib/libkdevpythonduchain.so
---Type <return> to continue, or q <return> to quit---#3  0x00007fffce64933a in Python::AstDefaultVisitor::visitCode(Python::CodeAst*) ()
   from /tmp/kdd/usr/lib/libkdevpythonparser.so
#4  0x00007fff99abb622 in Python::visitorForString(QString, KDevelop::DUContext*, KDevelop::CursorInRevision) () from /tmp/kdd/usr/lib/libkdevpythoncompletion.so
#5  0x00007fff99abf276 in Python::PythonCodeCompletionContext::functionCallItems() ()
   from /tmp/kdd/usr/lib/libkdevpythoncompletion.so
#6  0x00007fff99ac4748 in Python::PythonCodeCompletionContext::completionItems(bool&, bool) ()
   from /tmp/kdd/usr/lib/libkdevpythoncompletion.so
#7  0x00007fff99ac45c3 in Python::PythonCodeCompletionContext::completionItems(bool&, bool) ()
   from /tmp/kdd/usr/lib/libkdevpythoncompletion.so
#8  0x00007fff99ac45c3 in Python::PythonCodeCompletionContext::completionItems(bool&, bool) ()
   from /tmp/kdd/usr/lib/libkdevpythoncompletion.so
#9  0x00007fffed85ca0a in KDevelop::CodeCompletionWorker::computeCompletions(KDevelop::DUChainPointer<KDevelop::DUContext>, KTextEditor::Cursor const&, QString, KTextEditor::Range const&, QString const&) () from /tmp/kdd/usr/lib/libKDevPlatformLanguage.so.10
#10 0x00007fffed85c4aa in KDevelop::CodeCompletionWorker::computeCompletions(KDevelop::DUChainPointer<KDevelop::DUContext>, KTextEditor::Cursor const&, KTextEditor::View*) ()
   from /tmp/kdd/usr/lib/libKDevPlatformLanguage.so.10
#11 0x00007fffed86262a in QtPrivate::QSlotObject<void (KDevelop::CodeCompletionWorker::*)(KDevelop::DUChainPointer<KDevelop::DUContext>, KTextEditor::Cursor const&, KTextEditor::View*), QtPrivate::List<KDevelop::DUChainPointer<KDevelop::DUContext>, KTextEditor::Cursor const&, KTextEditor::View*>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) ()
   from /tmp/kdd/usr/lib/libKDevPlatformLanguage.so.10
#12 0x00007ffff1c8cc81 in QObject::event(QEvent*) () from /tmp/kdd/usr/lib/libQt5Core.so.5
#13 0x00007ffff25d54dc in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
   from /tmp/kdd/usr/lib/libQt5Widgets.so.5
#14 0x00007ffff25dc8f6 in QApplication::notify(QObject*, QEvent*) ()
   from /tmp/kdd/usr/lib/libQt5Widgets.so.5
#15 0x00007ffff1c62d78 in QCoreApplication::notifyInternal2(QObject*, QEvent*) ()
   from /tmp/kdd/usr/lib/libQt5Core.so.5
#16 0x00007ffff1c6536b in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) ()
   from /tmp/kdd/usr/lib/libQt5Core.so.5
#17 0x00007ffff1cb01ba in QEventDispatcherUNIX::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
    () from /tmp/kdd/usr/lib/libQt5Core.so.5
---Type <return> to continue, or q <return> to quit---#18 0x00007ffff1c6132a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) ()
   from /tmp/kdd/usr/lib/libQt5Core.so.5
#19 0x00007ffff1a95ebc in QThread::exec() () from /tmp/kdd/usr/lib/libQt5Core.so.5
#20 0x00007fffed8643b8 in KDevelop::CompletionWorkerThread::run() ()
   from /tmp/kdd/usr/lib/libKDevPlatformLanguage.so.10
#21 0x00007ffff1a9a7f6 in ?? () from /tmp/kdd/usr/lib/libQt5Core.so.5
#22 0x00007fffe9f45184 in start_thread (arg=0x7fff99814700) at pthread_create.c:312
#23 0x00007ffff0f0437d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
Comment 10 Jonathan Verner 2016-12-27 16:33:03 UTC
Created attachment 103022 [details]
Modified AppRun script to run kdevelop in gdb
Comment 11 Sven Brauch 2016-12-28 00:27:12 UTC
Thanks for the trace, that looks somewhat useful. It seems remotely familiar, but I'm not sure what causes the crash ... I'll try to reproduce it with the file you provided.
Comment 12 Jonathan Verner 2016-12-28 10:51:47 UTC
Hmm, it seems the file needs to be opened in a larger project (so far, I didn't encounter the crash when editing just the file with no project open). The particular file comes from a simple django project. However, it has quite a few packages installed in a virtual_env and I am running kdevelop with this virtual env activated.
Comment 13 Jonathan Verner 2017-01-03 22:14:29 UTC
I can also reproduce the crash with the 5.1 beta release... It gives me a similar backtrace:

#0  0x00007fffa8f35555 in Python::Helper::declarationForName(KDevelop::QualifiedIdentifier const&, KDevelop::RangeInRevision const&, KDevelop::DUChainPointer<KDevelop::DUContext const>) ()
   from /tmp/kdd/usr/lib/libkdevpythonduchain.so
#1  0x00007fffa8f2c720 in Python::ExpressionVisitor::visitName(Python::NameAst*) ()
   from /tmp/kdd/usr/lib/libkdevpythonduchain.so
#2  0x00007fffa8f2bd58 in Python::ExpressionVisitor::visitCall(Python::CallAst*) ()
   from /tmp/kdd/usr/lib/libkdevpythonduchain.so
#3  0x00007fffcb9a3dca in Python::AstDefaultVisitor::visitCode(Python::CodeAst*) ()
   from /tmp/kdd/usr/lib/libkdevpythonparser.so
#4  0x00007fffa91977d2 in Python::visitorForString(QString, KDevelop::DUContext*, KDevelop::CursorInRevision) () from /tmp/kdd/usr/lib/libkdevpythoncompletion.so
#5  0x00007fffa919f98e in Python::PythonCodeCompletionContext::memberAccessItems() ()
   from /tmp/kdd/usr/lib/libkdevpythoncompletion.so
#6  0x00007fffa91a0d66 in Python::PythonCodeCompletionContext::completionItems(bool&, bool) ()
   from /tmp/kdd/usr/lib/libkdevpythoncompletion.so
#7  0x00007fffedd7bd5a in KDevelop::CodeCompletionWorker::computeCompletions(KDevelop::DUChainPointer<KDevelop::DUContext>, KTextEditor::Cursor const&, QString, KTextEditor::Range const&, QString const&) () from /tmp/kdd/usr/lib/libKDevPlatformLanguage.so.10
#8  0x00007fffedd7b7fa in KDevelop::CodeCompletionWorker::computeCompletions(KDevelop::DUChainPointer<KDevelop::DUContext>, KTextEditor::Cursor const&, KTextEditor::View*) ()
   from /tmp/kdd/usr/lib/libKDevPlatformLanguage.so.10
#9  0x00007fffedd8197a in QtPrivate::QSlotObject<void (KDevelop::CodeCompletionWorker::*)(KDevelop::DUChainPointer<KDevelop::DUContext>, KTextEditor::Cursor const&, KTextEditor::View*), QtPrivate::List<KDevelop::DUChainPointer<KDevelop::DUContext>, KTextEditor::Cursor const&, KTextEditor::View*>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) ()
   from /tmp/kdd/usr/lib/libKDevPlatformLanguage.so.10
#10 0x00007ffff1d92081 in QObject::event(QEvent*) () from /tmp/kdd/usr/lib/libQt5Core.so.5
#11 0x00007ffff26d3a9c in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
   from /tmp/kdd/usr/lib/libQt5Widgets.so.5
#12 0x00007ffff26dae96 in QApplication::notify(QObject*, QEvent*) ()
   from /tmp/kdd/usr/lib/libQt5Widgets.so.5
#13 0x00007ffff1d68248 in QCoreApplication::notifyInternal2(QObject*, QEvent*) ()
   from /tmp/kdd/usr/lib/libQt5Core.so.5
#14 0x00007ffff1d6a83b in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) ()
   from /tmp/kdd/usr/lib/libQt5Core.so.5
#15 0x00007ffff1db56e4 in QEventDispatcherUNIX::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
    () from /tmp/kdd/usr/lib/libQt5Core.so.5
#16 0x00007ffff1d667fa in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) ()
   from /tmp/kdd/usr/lib/libQt5Core.so.5
#17 0x00007ffff1b9ad6c in QThread::exec() () from /tmp/kdd/usr/lib/libQt5Core.so.5
#18 0x00007fffedd83708 in KDevelop::CompletionWorkerThread::run() ()
   from /tmp/kdd/usr/lib/libKDevPlatformLanguage.so.10
#19 0x00007ffff1b9f6a6 in ?? () from /tmp/kdd/usr/lib/libQt5Core.so.5
#20 0x00007fffe9e07184 in start_thread (arg=0x7fffa8ef1700) at pthread_create.c:312
#21 0x00007ffff100937d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
Comment 14 Sven Brauch 2017-01-11 21:20:37 UTC
Hmm, we are still not able to reproduce this. Really strange ... is the project the file is from public? Maybe it only happens when the whole project is loaded?
Comment 15 Sven Brauch 2017-01-11 23:37:15 UTC
The highlighting does look odd, though. The parameter "cls" doesn't have the same colour as the uses. That might be related.
Comment 16 Jonathan Verner 2017-01-12 11:50:48 UTC
Well, the project is semi-public (website of our department, I am not sure about my security skills and don't want people to go looking for bugs and then doing bad stuff with the website :-)). Anyway, for the time being I've made the repo public, if it helps you reproduce the bug:

https://gitlab.com/Verner/Logika-Website

I load the project with an activated python virtual env with packages listed in requirements.txt.
Comment 17 Sven Brauch 2017-01-12 15:46:11 UTC
The project does not seem to be marked as public there, or is it?

Another option is to just send me a tar archive (to mail at svenbrauch.de), I can forward it to the 1-2 other involved developers, then you don't need to make it public.
Comment 18 Sven Brauch 2017-01-14 12:17:19 UTC
Git commit 4cf23720b123dee3b4ad85957ed165d23e37e1e7 by Sven Brauch.
Committed on 14/01/2017 at 12:15.
Pushed by brauch into branch '5.1'.

container types: do not write lock when calling addContentType()

This should be done from outside. Rationale: In many cases (see e.g.
kdev-python expression visitor) the type is instantiated, a lot of
content types are added, and only then is it attached to a declaration.
In this case, no write locking is necessary, and it prevents the calling
code to hold a more coarse-scale read lock.

This requires changes to the code using the API only if the modified type
is already accessible through a declaration. I changed kdev-python
and checked kdev-ruby (it has no cases where this applies and no read-lock
is held at that time); kdev-php doesn't use this class.

M  +3    -1    language/duchain/types/containertypes.h

https://commits.kde.org/kdevplatform/4cf23720b123dee3b4ad85957ed165d23e37e1e7
Comment 19 Sven Brauch 2017-01-14 12:20:39 UTC
Git commit 600f6b34d95e506fdf941c5bfadc79aeaf7de617 by Sven Brauch.
Committed on 14/01/2017 at 12:19.
Pushed by brauch into branch '5.1'.

completion: wrap expression visitor into a read lock

Unlike in the parse job, we do not hold the parse lock for the context
which is passed to the expression visitor. Thus, it might be modified
or deleted by a parse job while we're passing the pointer around. This
means that after data is taken out of the DUContextPointer, the read
lock must not be released again while the object is in use.

Likely the reason for occasional crashes in completion in some situations.

M  +8    -7    codecompletion/context.cpp
M  +3    -0    duchain/declarationbuilder.cpp
M  +3    -5    duchain/expressionvisitor.cpp

https://commits.kde.org/kdev-python/600f6b34d95e506fdf941c5bfadc79aeaf7de617
Comment 20 Jonathan Verner 2017-01-14 15:13:27 UTC
Thanks for your work!!
Comment 21 Sven Brauch 2017-01-14 15:16:21 UTC
I hope this actually fixes the issue for you, I'm not 100% sure. Let me know whether it still happens if you get a chance to test.
Comment 22 Jonathan Verner 2017-01-19 21:06:08 UTC
I definitely will. Unfortunately, I currently don't have a kde5 devel environment. Eventually I plan to setup a devel environment and compile kdevelop from sources. Until then, though, I'll need to wait for a new Appimage with the fix to test. Thanks again for your work!