Summary: | Crash during parsing of cmake/C++/Python files | ||
---|---|---|---|
Product: | [Applications] kdevelop | Reporter: | Axel Kellermann <axel.kellermann> |
Component: | general | Assignee: | kdevelop-bugs-null |
Status: | RESOLVED DUPLICATE | ||
Severity: | crash | Keywords: | drkonqi |
Priority: | NOR | ||
Version: | 5.0.0 | ||
Target Milestone: | --- | ||
Platform: | Compiled Sources | ||
OS: | Linux | ||
Latest Commit: | Version Fixed In: | ||
Attachments: | ASAN output of heap-buffer-overflow error during parsing of CMake file |
Description
Axel Kellermann
2016-08-26 08:57:34 UTC
Encountered another crash during parsing of cmake files: #0 0x000000300147fbd6 in _int_free (av=0x7fffa8000020, p=<optimized out>, have_lock=0) at malloc.c:4015 #1 0x0000003001483abc in __GI___libc_free (mem=<optimized out>) at malloc.c:2969 #2 0x00007fffc0cb2a74 in cmListFileLexer_yyfree (ptr=0x7fffa962ceb0, yyscanner=0x7fffa962ceb0) at cmListFileLexer.c:2417 #3 0x00007fffc0cb29ea in cmListFileLexer_yylex_destroy (yyscanner=0x7fffa962ceb0) at cmListFileLexer.c:2359 #4 0x00007fffc0cb2ee3 in cmListFileLexerDestroy (lexer=0x7fffa962c750) at cmListFileLexer.in.l:421 #5 0x00007fffc0cb31a1 in cmListFileLexer_SetFileName (lexer=0x7fffa962c750, name=0x0, bom=0x0) at cmListFileLexer.in.l:504 #6 0x00007fffc0cb2fc7 in cmListFileLexer_Delete (lexer=0x7fffa962c750) at cmListFileLexer.in.l:454 #7 0x00007fffc0cb4382 in CMakeListsParser::readCMakeFile (_fileName=...) at /home/akellerm/Repos/kdesrc/kdevelop/projectmanagers/cmake/parser/cmakelistsparser.cpp:148 #8 0x00007fffbab1a58e in CMakeParseJob::run (this=0x3522310) at /home/akellerm/Repos/kdesrc/kdevelop/projectmanagers/cmake/duchain/cmakeparsejob.cpp:98 #9 0x00007fffec1dc650 in ThreadWeaver::IdDecorator::run(QSharedPointer<ThreadWeaver::JobInterface>, ThreadWeaver::Thread*) () from /usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5 #10 0x00007fffec1dcc90 in ThreadWeaver::Executor::run(QSharedPointer<ThreadWeaver::JobInterface> const&, ThreadWeaver::Thread*) () from /usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5 #11 0x00007fffec1db7e0 in ThreadWeaver::Job::execute(QSharedPointer<ThreadWeaver::JobInterface> const&, ThreadWeaver::Thread*) () from /usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5 #12 0x00007fffec1db28a in ThreadWeaver::Thread::run() () from /usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5 #13 0x00007ffff5cf684e in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5 #14 0x00000030024076fa in start_thread (arg=0x7fffb8cd4700) at pthread_create.c:333 #15 0x0000003001506b5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 And another one: #0 0x000000300147e4f5 in malloc_consolidate (av=av@entry=0x7fffa4000020) at malloc.c:4184 #1 0x000000300148155e in _int_malloc (av=av@entry=0x7fffa4000020, bytes=bytes@entry=19684) at malloc.c:3451 #2 0x00000030014835a4 in __GI___libc_malloc (bytes=19684) at malloc.c:2914 #3 0x00007ffff5cf7eb8 in QArrayData::allocate(unsigned long, unsigned long, unsigned long, QFlags<QArrayData::AllocationOption>) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5 #4 0x00007ffff5cfa9e1 in QByteArray::resize(int) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5 #5 0x00007ffff5e00a29 in QIODevice::readAll() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5 #6 0x00007ffff3975690 in KDevelop::ParseJob::readContents (this=0x34641d0) at /home/akellerm/Repos/kdesrc/kdevplatform/language/backgroundparser/parsejob.cpp:316 #7 0x00007fffbab19fd0 in CMakeParseJob::run (this=0x34641d0) at /home/akellerm/Repos/kdesrc/kdevelop/projectmanagers/cmake/duchain/cmakeparsejob.cpp:59 #8 0x00007fffec1dc650 in ThreadWeaver::IdDecorator::run(QSharedPointer<ThreadWeaver::JobInterface>, ThreadWeaver::Thread*) () from /usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5 #9 0x00007fffec1dcc90 in ThreadWeaver::Executor::run(QSharedPointer<ThreadWeaver::JobInterface> const&, ThreadWeaver::Thread*) () from /usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5 #10 0x00007fffec1db7e0 in ThreadWeaver::Job::execute(QSharedPointer<ThreadWeaver::JobInterface> const&, ThreadWeaver::Thread*) () from /usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5 #11 0x00007fffec1db28a in ThreadWeaver::Thread::run() () from /usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5 #12 0x00007ffff5cf684e in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5 #13 0x00000030024076fa in start_thread (arg=0x7fffaffff700) at pthread_create.c:333 #14 0x0000003001506b5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 I'd be happy to help with debugging, but I need some pointers where to start, and what to look for. Can you compile KDevplatform + KDevelop with ASAN and see if you spot anything? Or try `valgrind --smc-check=all-non-file kdevelop -s ...`. Or are you maybe running out of memory? I compiled kdevelop and kdevplatform with ASAN and tested again on two platforms: 1. A laptop with only 4GB of RAM, where memory actually is kind of an issue. kdevelop takes up ~3.5GB of RAM and the system starts to heavyly swap out pages. But the limits of the swap partition are never reached, so parsing gets very slow, but never runs out of memory. 2. A PC with 16GB of RAM, where memory is definitely no issue. On both machines ASAN reports the same heap-buffer-overflow error during parsing of cmake files (attached as asan_readCMakeFile_20160908.txt). Created attachment 100989 [details]
ASAN output of heap-buffer-overflow error during parsing of CMake file
Could that be related to bug 363269? Could you check the CMake file KDevelop is parsing (see debug output). Is it a binary file? I found the problem, and it's actually similar, but not the same as bug 363269. The code base being parsed contains a subfolder with example text in a couple of different asian languages, all stored as .txt files in UTF16LE. The cmake parser seems to not just read CMakeLists.txt and *.cmake, but also generally all files with the extension .txt. Apparently the encoding of the example files is handled correctly, but then the wide characters are given to the lexer that expects chars, hence the one byte buffer overflow reported (just my guess, didn't have a look at all the code involved). If I delete all the UTF16 files, the whole code base is processed successfully. So I guess the problem is kind of a duplicate of 363269, but more specific. The cmake parser seems generally to greedy in selecting the files it processes. Could that be narrowed down to only CMakeLists.txt and *.cmake? Feel free to close this one if you think it's covered by 363269. Thanks a ton for the analysis, Axel. And yes, those two bug reports have the same root cause => I'll merge them. *** This bug has been marked as a duplicate of bug 363269 *** |