| Summary: | Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4 | ||
|---|---|---|---|
| Product: | [Developer tools] kdev-python | Reporter: | Michael G Hart <michael.george.hart> |
| Component: | general | Assignee: | Sven Brauch <mail> |
| Status: | RESOLVED FIXED | ||
| Severity: | crash | CC: | mail, mail |
| Priority: | HI | Keywords: | drkonqi |
| Version First Reported In: | 5.1.0 | ||
| Target Milestone: | --- | ||
| Platform: | openSUSE | ||
| OS: | Linux | ||
| Latest Commit: | https://commits.kde.org/kdevplatform/396dda6a43402cadefbcffa4473d62d34de14175 | Version Fixed In: | 5.1.2 |
| Sentry Crash Report: | |||
| Attachments: | attachment-7112-0.html | ||
|
Description
Michael G Hart
2017-04-16 01:55:54 UTC
Crash in kdev-python, moving ticket. Do you have debug symbols by any chance? Otherwise I can also try to reproduce this ... Created attachment 105062 [details] attachment-7112-0.html Sorry but I don't have anything bug in symbols. However with all the versions of kdevelop I tried I got the same failure wedding background parsing was turned on So I would be very surprised if you cannot replicate the problem. On Apr 17, 2017 12:02 PM, "Sven Brauch" <bugzilla_noreply@kde.org> wrote: > https://bugs.kde.org/show_bug.cgi?id=378827 > > --- Comment #2 from Sven Brauch <mail@svenbrauch.de> --- > Do you have debug symbols by any chance? Otherwise I can also try to > reproduce > this ... > > -- > You are receiving this mail because: > You reported the bug. Alright thanks, I'll try ASAP Reproducible with UE master and kdev-python 5.1-git.
=======
====> AST ====> building abstract syntax tree for "/home/flh/projects/UnrealEngine/Engine/Extras/Maya_AnimationRiggingTools/ArtToolsOSX/MayaTools/General/Scripts/Modules/ART_Core.py"
Got syntax tree from python parser: 1 1
ASSERT: "uint(i) < uint(size())" in file /usr/include/qt/QtCore/qstring.h, line 897
Thread 7 "Queue(0x8ab900)" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffc4c43700 (LWP 27501)]
0x00007fffef245a10 in raise () from /usr/lib/libc.so.6
(gdb) bt
#0 0x00007fffef245a10 in raise () at /usr/lib/libc.so.6
#1 0x00007fffef24713a in abort () at /usr/lib/libc.so.6
#2 0x00000000004058d6 in messageOutput(QtMsgType, QMessageLogContext const&, QString const&) (type=<optimized out>, context=..., msg=...)
at /home/flh/projects/kdevplatform/util/duchainify/main.cpp:79
#3 0x00007fffefefa298 in () at /usr/lib/libQt5Core.so.5
#4 0x00007fffefefbe09 in QMessageLogger::fatal(char const*, ...) const () at /usr/lib/libQt5Core.so.5
#5 0x00007fffefef73de in () at /usr/lib/libQt5Core.so.5
#6 0x00007fffc68d13bc in QString::at(int) const (this=0x7fffc4c41360, i=-1) at /usr/include/qt/QtCore/qstring.h:897
#7 0x00007fffc68e20af in Python::RangeFixVisitor::whitespaceAtEnd(QString const&) (this=0x7fffc4c41670, line=...) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:443
#8 0x00007fffc68e1fd6 in Python::RangeFixVisitor::fixAlias(Python::Ast*, Python::Ast*, int, int) (this=0x7fffc4c41670, dotted=0x7fffb8042fb0, asname=0x7fffb8042ff0, startLine=1, aliasIndex=0)
at /home/flh/projects/kdev-python/parser/astbuilder.cpp:433
#9 0x00007fffc68e12c4 in Python::RangeFixVisitor::visitImport(Python::ImportAst*) (this=0x7fffc4c41670, node=0x7fffb8059db0) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:209
#10 0x00007fffc68dbddc in Python::AstVisitor::visitNode(Python::Ast*) (this=0x7fffc4c41670, node=0x7fffb8059db0) at /home/flh/projects/kdev-python/parser/astvisitor.cpp:57
#11 0x00007fffc68e0c55 in Python::RangeFixVisitor::visitNode(Python::Ast*) (this=0x7fffc4c41670, node=0x7fffb8059db0) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:104
#12 0x00007fffc68d5f3b in Python::AstDefaultVisitor::visitCode(Python::CodeAst*) (this=0x7fffc4c41670, node=0x7fffb8051510) at /home/flh/projects/kdev-python/parser/astdefaultvisitor.cpp:94
#13 0x00007fffc68dbba4 in Python::AstVisitor::visitNode(Python::Ast*) (this=0x7fffc4c41670, node=0x7fffb8051510) at /home/flh/projects/kdev-python/parser/astvisitor.cpp:41
#14 0x00007fffc68e0c55 in Python::RangeFixVisitor::visitNode(Python::Ast*) (this=0x7fffc4c41670, node=0x7fffb8051510) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:104
#15 0x00007fffc68df46b in Python::AstBuilder::parse(QUrl const&, QString&) (this=0x7fffc4c41d90, filename=..., contents=...) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:757
#16 0x00007fffc68d3909 in Python::ParseSession::parse() (this=0x7fffb8003900) at /home/flh/projects/kdev-python/parser/parsesession.cpp:76
#17 0x00007fffc6d6dc24 in Python::ParseJob::run(QSharedPointer<ThreadWeaver::JobInterface>, ThreadWeaver::Thread*) (this=0xb72e20) at /home/flh/projects/kdev-python/pythonparsejob.cpp:157
#18 0x00007ffff60ee5d6 in ThreadWeaver::IdDecorator::run(QSharedPointer<ThreadWeaver::JobInterface>, ThreadWeaver::Thread*) () at /usr/lib/libKF5ThreadWeaver.so.5
#19 0x00007ffff60eebd8 in ThreadWeaver::Executor::run(QSharedPointer<ThreadWeaver::JobInterface> const&, ThreadWeaver::Thread*) () at /usr/lib/libKF5ThreadWeaver.so.5
#20 0x00007ffff60edaea in ThreadWeaver::Job::execute(QSharedPointer<ThreadWeaver::JobInterface> const&, ThreadWeaver::Thread*) () at /usr/lib/libKF5ThreadWeaver.so.5
#21 0x00007ffff60ed190 in ThreadWeaver::Thread::run() () at /usr/lib/libKF5ThreadWeaver.so.5
#22 0x00007fffeff156d8 in () at /usr/lib/libQt5Core.so.5
#23 0x00007fffe72ba2e7 in start_thread () at /usr/lib/libpthread.so.0
#24 0x00007fffef2fe54f in clone () at /usr/lib/libc.so.6
(gdb) bt full
#0 0x00007fffef245a10 in raise () at /usr/lib/libc.so.6
#1 0x00007fffef24713a in abort () at /usr/lib/libc.so.6
#2 0x00000000004058d6 in messageOutput(QtMsgType, QMessageLogContext const&, QString const&) (type=<optimized out>, context=..., msg=...)
at /home/flh/projects/kdevplatform/util/duchainify/main.cpp:79
#3 0x00007fffefefa298 in () at /usr/lib/libQt5Core.so.5
#4 0x00007fffefefbe09 in QMessageLogger::fatal(char const*, ...) const () at /usr/lib/libQt5Core.so.5
#5 0x00007fffefef73de in () at /usr/lib/libQt5Core.so.5
#6 0x00007fffc68d13bc in QString::at(int) const (this=0x7fffc4c41360, i=-1) at /usr/include/qt/QtCore/qstring.h:897
#7 0x00007fffc68e20af in Python::RangeFixVisitor::whitespaceAtEnd(QString const&) (this=0x7fffc4c41670, line=...) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:443
i = 0
#8 0x00007fffc68e1fd6 in Python::RangeFixVisitor::fixAlias(Python::Ast*, Python::Ast*, int, int) (this=0x7fffc4c41670, dotted=0x7fffb8042fb0, asname=0x7fffb8042ff0, startLine=1, aliasIndex=0)
at /home/flh/projects/kdev-python/parser/astbuilder.cpp:433
line = {static null = {<No data fields>}, d = 0x7ffff019b420}
lineno = 1
end = -993782736
back = 32767
#9 0x00007fffc68e12c4 in Python::RangeFixVisitor::visitImport(Python::ImportAst*) (this=0x7fffc4c41670, node=0x7fffb8059db0) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:209
alias = 0x7fffb8059df0
_container_ =
{c = {<QListSpecialMethods<Python::AliasAst*>> = {<No data fields>}, {p = {static shared_null = {ref = {atomic = {_q_value = {<std::__atomic_base<int>> = {static _S_alignment = 4, _M_i = -1}, <No data fields>}}}, alloc = 0, begin = 0, end = 0, array = {0x0}}, d = 0x7fffb8034c60}, d = 0x7fffb8034c60}}, i = {i = 0x7fffb8034c70}, e = {i = 0x7fffb8034c78}, control = 1}
aliasIndex = 0
#10 0x00007fffc68dbddc in Python::AstVisitor::visitNode(Python::Ast*) (this=0x7fffc4c41670, node=0x7fffb8059db0) at /home/flh/projects/kdev-python/parser/astvisitor.cpp:57
#11 0x00007fffc68e0c55 in Python::RangeFixVisitor::visitNode(Python::Ast*) (this=0x7fffc4c41670, node=0x7fffb8059db0) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:104
#12 0x00007fffc68d5f3b in Python::AstDefaultVisitor::visitCode(Python::CodeAst*) (this=0x7fffc4c41670, node=0x7fffb8051510) at /home/flh/projects/kdev-python/parser/astdefaultvisitor.cpp:94
statement = 0x7fffb8059db0
_container_ =
{c = {<QListSpecialMethods<Python::Ast*>> = {<No data fields>}, {p = {static shared_null = {ref = {atomic = {_q_value = {<std::__atomic_base<int>> = {static _S_alignment = 4, _M_i = -1}, <No data fields>}}}, alloc = 0, begin = 0, end = 0, array = {0x0}}, d = 0x7fffb804ee70}, d = 0x7fffb804ee70}}, i = {i = 0x7fffb804ee80}, e = {i = 0x7fffb804eea8}, control = 1}
#13 0x00007fffc68dbba4 in Python::AstVisitor::visitNode(Python::Ast*) (this=0x7fffc4c41670, node=0x7fffb8051510) at /home/flh/projects/kdev-python/parser/astvisitor.cpp:41
#14 0x00007fffc68e0c55 in Python::RangeFixVisitor::visitNode(Python::Ast*) (this=0x7fffc4c41670, node=0x7fffb8051510) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:104
#15 0x00007fffc68df46b in Python::AstBuilder::parse(QUrl const&, QString&) (this=0x7fffc4c41d90, filename=..., contents=...) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:757
__PRETTY_FUNCTION__ = "Python::CodeAst::Ptr Python::AstBuilder::parse(const QUrl&, QString&)"
hacked = {first = {static null = {<No data fields>}, d = 0x7fffb800f4c0}, second = 0}
lineOffset = 0
pyIniter = {<QMutexLocker> = {val = 140736526817705}, arena = 0x7fffcc1bb090}
arena = 0x7fffcc1bb090
flags = {cf_flags = 2304}
exception = 0x0
value = 0x0
backtrace = 0x0
cythonSyntaxRemover =
{m_code = {<QList<QString>> = {<QListSpecialMethods<QString>> = {<No data fields>}, {p = {static shared_null = {ref = {atomic = {_q_value = {<std::__atomic_base<int>> = {static _S_alignment = 4, _M_i = -1}, <No data fields>}}}, alloc = 0, begin = 0, end = 0, array = {0x0}}, d = 0x7ffff019ce40 <QListData::shared_null>}, d = 0x7ffff019ce40 <QListData::shared_null>}}, <No data fields>}, m_strippedCode = {static null = {<No data fields>}, d = 0x7ffff019b480 <QArrayData::shared_null>}, m_offset = {m_line = 0, m_column = 0}, m_deletions = {d = 0x7ffff019b480 <QArrayData::shared_null>}}
syntaxtree = 0x7fffb80ecdc8
t = {ast = 0x7fffb8051510, nodeStack = {<QVector<Python::Ast*>> = {d = 0x7fffb816c320}, <No data fields>}, m_lineOffset = 0}
fixVisitor =
{<Python::AstDefaultVisitor> = {<Python::AstVisitor> = {_vptr.AstVisitor = 0x7fffc6b00490 <vtable for Python::RangeFixVisitor+16>}, <No data fields>}, lines = {<QList<QString>> = {<QListSpecialMethods<QString>> = {<No data fields>}, {p = {static shared_null = {ref = {atomic = {_q_value = {<std::__atomic_base<int>> = {static _S_alignment = 4, _M_i = -1}, <No data fields>}}}, alloc = 0, begin = 0, end = 0, array = {0x0}}, d = 0x7fffb805f520}, d = 0x7fffb805f520}}, <No data fields>}, dots = {d = 0x7ffff019b480 <QArrayData::shared_null>}, attributeStart = {m_line = 0, m_column = 0}, static findString = {d = {d = 0xbc50b0}}, static findNumber = {d = {d = 0xbc5170}}}
#16 0x00007fffc68d3909 in Python::ParseSession::parse() (this=0x7fffb8003900) at /home/flh/projects/kdev-python/parser/parsesession.cpp:76
pythonparser =
{m_problems = {<QListSpecialMethods<QExplicitlySharedDataPointer<KDevelop::Problem> >> = {<No data fields>}, {p = {static shared_null = {ref = {atomic = {_q_value = {<std::__atomic_base<int>> = {static _S_alignment = 4, _M_i = -1}, <No data fields>}}}, alloc = 0, begin = 0, end = 0, array = {0x0}}, d = 0x7ffff019ce40 <QListData::shared_null>}, d = 0x7ffff019ce40 <QListDat---Type <return> to continue, or q <return> to quit---
a::shared_null>}}, static pyInitLock = {<QBasicMutex> = {d_ptr = {_q_value = {_M_b = {_M_p = 0x3}}}}, <No data fields>}}
matched = {first = {value = 0x0, d = 0x0}, second = false}
__PRETTY_FUNCTION__ = "QPair<QSharedPointer<Python::CodeAst>, bool> Python::ParseSession::parse()"
#17 0x00007fffc6d6dc24 in Python::ParseJob::run(QSharedPointer<ThreadWeaver::JobInterface>, ThreadWeaver::Thread*) (this=0xb72e20) at /home/flh/projects/kdev-python/pythonparsejob.cpp:157
__PRETTY_FUNCTION__ = "virtual void Python::ParseJob::run(ThreadWeaver::JobPointer, ThreadWeaver::Thread*)"
parselock = {q_val = 12349121}
urlLock = {m_url = {m_index = 66384}}
toUpdate = {m_topContext = 0x0}
parserResults = {first = {value = 0x0, d = 0x0}, second = false}
editor = {value = 0x7fffb8003240, d = 0x7ffff011cce3 <QObject::event(QEvent*)+499>}
lock = {m_lock = 0x10, m_locked = false, m_timeout = 1519604770}
#18 0x00007ffff60ee5d6 in ThreadWeaver::IdDecorator::run(QSharedPointer<ThreadWeaver::JobInterface>, ThreadWeaver::Thread*) () at /usr/lib/libKF5ThreadWeaver.so.5
#19 0x00007ffff60eebd8 in ThreadWeaver::Executor::run(QSharedPointer<ThreadWeaver::JobInterface> const&, ThreadWeaver::Thread*) () at /usr/lib/libKF5ThreadWeaver.so.5
#20 0x00007ffff60edaea in ThreadWeaver::Job::execute(QSharedPointer<ThreadWeaver::JobInterface> const&, ThreadWeaver::Thread*) () at /usr/lib/libKF5ThreadWeaver.so.5
#21 0x00007ffff60ed190 in ThreadWeaver::Thread::run() () at /usr/lib/libKF5ThreadWeaver.so.5
#22 0x00007fffeff156d8 in () at /usr/lib/libQt5Core.so.5
#23 0x00007fffe72ba2e7 in start_thread () at /usr/lib/libpthread.so.0
#24 0x00007fffef2fe54f in clone () at /usr/lib/libc.so.6
Git commit 6043407c4ee2e1b3eefac8e1f8c96a651b92a14e by Sven Brauch. Committed on 01/05/2017 at 21:44. Pushed by brauch into branch '5.1'. fix crash This loop is obviously wrong, it iterates over size()+1 indices 0..size(), but there are only size(). M +1 -1 parser/astbuilder.cpp https://commits.kde.org/kdev-python/6043407c4ee2e1b3eefac8e1f8c96a651b92a14e Not fixed, it crashes in visitAlias instead. Sorry, fixAlias In this case, it's related to the use of Windows line endings in the file. When the file is parsed in the background (but not while opened in the editor), we fail to normalise them properly and the `lines` QStringList is given a single line containing the whole document. fixAlias() doesn't check that AST node positions exist in the document before trying to access them, and this fails when the line doesn't exist. Correction, it's related to the use of MacOS Classic (why?!) line endings - i.e. just a carriage return but no newline.
Many places in kdev-python assume that lines can be split with \n, so we should probably normalise these earlier.
Alternative patch that solves this particular case:
diff --git a/parser/astbuilder.cpp b/parser/astbuilder.cpp
index 0e19f367..15df40a9 100644
--- a/parser/astbuilder.cpp
+++ b/parser/astbuilder.cpp
@@ -98,7 +98,8 @@ private:
class RangeFixVisitor : public AstDefaultVisitor {
public:
RangeFixVisitor(const QString& contents)
- : lines(contents.split('\n')) { };
+ // Split on all three variants - \r, \n or \r\n
+ : lines(contents.split(QRegExp("\\r\\n?|\\n"))) { };
void visitNode(Ast* node) override {
AstDefaultVisitor::visitNode(node);
I don't recall -- did we ever fix this issue? I know we had a discussion and a RR ... No, see https://phabricator.kde.org/D5707 People wanted (reasonable) changes to my patch, but I was a bit burned-out and never made them. Something like mwolff's suggestion should be ok, I'll try it out and write some tests. Git commit 396dda6a43402cadefbcffa4473d62d34de14175 by Francis Herne. Committed on 26/07/2017 at 19:09. Pushed by flherne into branch '5.1'. Normalize line endings when creating ParseJob from background file. When the file being parsed is open in the editor, `ParseJob::readContents()` uses `KTextEditor::Document::text()` to read the contents. At some point within KTextEditor, the line endings are normalized - even if the document uses \r or \r\n, the returned content is separated by \n only. Otherwise, the contents are read directly from the file. Until this patch, the line endings aren't normalized in this case; the ParseJob contents differ depending on whether the file is open in the editor. Several places in kdev-python (and, at a glance, other language plugins) assume that lines can be separated by, and end with, \n. Old-style Macintosh endings (\r) break this assumption, in the worst case causing a crash. Using a regex in each instance would be a pain, and one code path normalizes line endings already, so I think the best option is to do so for background files also. Differential Revision: https://phabricator.kde.org/D5707 M +4 -0 language/backgroundparser/parsejob.cpp M +15 -0 util/kdevstringhandler.cpp M +5 -0 util/kdevstringhandler.h M +29 -1 util/tests/test_stringhandler.cpp M +3 -0 util/tests/test_stringhandler.h https://commits.kde.org/kdevplatform/396dda6a43402cadefbcffa4473d62d34de14175 |