Bug 385768 - segfault on parsing project: in clang::Decl::setInvalidDecl(bool) () from /usr/lib64/llvm/5/lib64/../lib64/libclangAST.so.5
Summary: segfault on parsing project: in clang::Decl::setInvalidDecl(bool) () from /us...
Status: RESOLVED FIXED
Alias: None
Product: kdevelop
Classification: Applications
Component: Language Support: CPP (Clang-based) (show other bugs)
Version: git master
Platform: Gentoo Packages Linux
: NOR normal
Target Milestone: ---
Assignee: kdevelop-bugs-null
URL: https://bugs.llvm.org/show_bug.cgi?id...
Keywords:
Depends on:
Blocks:
 
Reported: 2017-10-15 08:09 UTC by Johannes Hirte
Modified: 2020-09-07 21:49 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Johannes Hirte 2017-10-15 08:09:08 UTC
When loading digikam sourcecode into kdevelop, it dies with this segfault:

Thread 287 "Queue(0x123c740" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f57e119f700 (LWP 5956)]
0x00007f57f85566e0 in clang::Decl::setInvalidDecl(bool) () from /usr/lib64/llvm/5/lib64/../lib64/libclangAST.so.5
(gdb) bt
#0  0x00007f57f85566e0 in clang::Decl::setInvalidDecl(bool) () from /usr/lib64/llvm/5/lib64/../lib64/libclangAST.so.5
#1  0x00007f57f855672c in clang::Decl::setInvalidDecl(bool) () from /usr/lib64/llvm/5/lib64/../lib64/libclangAST.so.5
#2  0x00007f57f0687990 in clang::ASTDeclReader::VisitDecl(clang::Decl*) () from /usr/lib64/llvm/5/lib64/../lib64/../lib64/libclangSerialization.so.5
#3  0x00007f57f0688261 in clang::ASTDeclReader::VisitNamedDecl(clang::NamedDecl*) () from /usr/lib64/llvm/5/lib64/../lib64/../lib64/libclangSerialization.so.5
#4  0x00007f57f06887c1 in clang::ASTDeclReader::VisitValueDecl(clang::ValueDecl*) () from /usr/lib64/llvm/5/lib64/../lib64/../lib64/libclangSerialization.so.5
#5  0x00007f57f0688827 in clang::ASTDeclReader::VisitDeclaratorDecl(clang::DeclaratorDecl*) () from /usr/lib64/llvm/5/lib64/../lib64/../lib64/libclangSerialization.so.5
#6  0x00007f57f069580e in clang::ASTDeclReader::VisitVarDeclImpl(clang::VarDecl*) () from /usr/lib64/llvm/5/lib64/../lib64/../lib64/libclangSerialization.so.5
#7  0x00007f57f0695e83 in clang::ASTDeclReader::VisitDecompositionDecl(clang::DecompositionDecl*) () from /usr/lib64/llvm/5/lib64/../lib64/../lib64/libclangSerialization.so.5
#8  0x00007f57f069f54d in clang::ASTDeclReader::Visit(clang::Decl*) () from /usr/lib64/llvm/5/lib64/../lib64/../lib64/libclangSerialization.so.5
#9  0x00007f57f069fcd2 in clang::ASTReader::ReadDeclRecord(unsigned int) () from /usr/lib64/llvm/5/lib64/../lib64/../lib64/libclangSerialization.so.5
#10 0x00007f57f064e495 in clang::ASTReader::GetDecl(unsigned int) () from /usr/lib64/llvm/5/lib64/../lib64/../lib64/libclangSerialization.so.5
#11 0x00007f57f0675cbb in clang::ASTReader::PassInterestingDeclsToConsumer() () from /usr/lib64/llvm/5/lib64/../lib64/../lib64/libclangSerialization.so.5
#12 0x00007f57f061c435 in clang::ASTReader::StartTranslationUnit(clang::ASTConsumer*) () from /usr/lib64/llvm/5/lib64/../lib64/../lib64/libclangSerialization.so.5
#13 0x00007f57f0985b25 in clang::ParseAST(clang::Sema&, bool, bool) () from /usr/lib64/llvm/5/lib64/../lib64/../lib64/libclangParse.so.5
#14 0x00007f57f8143ae6 in clang::FrontendAction::Execute() () from /usr/lib64/llvm/5/lib64/../lib64/libclangFrontend.so.5
#15 0x00007f57f80e8dfe in clang::ASTUnit::Parse(std::shared_ptr<clang::PCHContainerOperations>, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer> >, llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>) ()
   from /usr/lib64/llvm/5/lib64/../lib64/libclangFrontend.so.5
#16 0x00007f57f80e951a in clang::ASTUnit::LoadFromCompilerInvocation(std::shared_ptr<clang::PCHContainerOperations>, unsigned int, llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>) () from /usr/lib64/llvm/5/lib64/../lib64/libclangFrontend.so.5
#17 0x00007f57f80ef558 in clang::ASTUnit::LoadFromCommandLine(char const**, char const**, std::shared_ptr<clang::PCHContainerOperations>, llvm::IntrusiveRefCntPtr<clang::DiagnosticsEngine>, llvm::StringRef, bool, bool, llvm::ArrayRef<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, llvm::MemoryBuffer*> >, bool, unsigned int, clang::TranslationUnitKind, bool, bool, bool, bool, bool, bool, bool, llvm::Optional<llvm::StringRef>, std::unique_ptr<clang::ASTUnit, std::default_delete<clang::ASTUnit> >*, llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>) () from /usr/lib64/llvm/5/lib64/../lib64/libclangFrontend.so.5
#18 0x00007f57f896fd73 in ?? () from /usr/lib64/llvm/5/lib64/libclang.so.5
#19 0x00007f584a1274c3 in llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) () from /usr/lib/llvm/5/lib64/libLLVMSupport.so.5
#20 0x00007f584a1275c4 in ?? () from /usr/lib/llvm/5/lib64/libLLVMSupport.so.5
#21 0x00007f584a1a167d in ?? () from /usr/lib/llvm/5/lib64/libLLVMSupport.so.5
#22 0x00007f587217e8aa in start_thread (arg=0x7f57e119f700) at pthread_create.c:465
#23 0x00007f58785b70bf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
Comment 1 Kevin Funk 2017-10-15 08:32:22 UTC
If you could isolate on which file(s) this is happening on, that'd be great. Then we can report a ticket upstream at the LLVM project.
Comment 2 Craig Tenenbaum 2017-10-16 02:23:22 UTC
Odd, I've seen this exact bug before and it pertains to the use of C++17 structured bindings. Looking at the backtrace itself, you can even see calls into the structured binding machinery at #7 with the call to `clang::ASTDeclReader::VisitDecompositionDecl(clang::DecompositionDecl*)`; "decomposition declaration" was just a prior name for this feature. Specifically, the cause of the segfault is an attempt to dereference a nullptr, which results from some data-structure (involved in the parsing of structured bindings), somewhere, not being initialized properly when the parser isn't in C++17 mode. I never narrowed the specific cause because fixes to CMake support obviated the issue for me, but I could look back into it if requested.

That said, I'm not sure what's causing this specific failure; some cursory investigation into digikam's sources uncovers nothing in CMake instructing the use of C++17 features at compile-time, and moreover, trying some grep patterns which should match a structured binding declaration come up empty, so either something is going very wrong the parser, or is it possible you have some local changes to the code using structured bindings, or additional projects open?
Comment 3 Craig Tenenbaum 2017-10-16 06:17:22 UTC
I can reproduce on KDevelop master and a Clang-6 checkout only a few hours old, the files that trigger it seem to be the "Managed" C++ files under 'digikam/utilities/mediaserver/upnpsdk/Platinum/Source/Extras/Managed/', and while I'm not entirely sure why Clang is trying to parse anything in there as a structured binding, it doesn't surprise me that fake, MS C++ masquerading as real C++ should cause issues. It accidentally confirms that the nullptr deref in Clang is a real bug in the parser, however, so I'll start trying to figure it out.

WORKAROUND: for the moment, so long as you don't have a burning desire or need to edit these files in KDevelop, you can add the entire directory to the project's exclude filters. While one could edit KDevelop's project file with a text editor and then launch KDevelop afterward, I was lazy and so temporarily disabled KDevelop's background parsing, added the directory to the filter from inside KDevelop, and then re-enabled background parsing. That ended the segfaulting for me.
Comment 4 Kevin Funk 2017-10-16 06:48:10 UTC
Can reproduce. Offending file is NeptuneException.cpp in digikam.git.

Test run which crashes:
/kdevelop/kdevplatform/util/duchainify/duchainify /home/kfunk/devel/src/kf5/digikam/utilities/mediaserver/upnpsdk/Platinum/Source/Extras/Managed/NeptuneException.cpp

Backtrace with a LLVM debug build (LLVM 5.0 branch):
#0  0x00007fffc52a8f5b in clang::Decl::setInvalidDecl (this=0x0, Invalid=true) at /home/kfunk/devel/src/llvm/tools/clang/lib/AST/DeclBase.cpp:112
#1  0x00007fffc52a8fad in clang::Decl::setInvalidDecl (this=<optimized out>, Invalid=<optimized out>) at /home/kfunk/devel/src/llvm/tools/clang/lib/AST/DeclBase.cpp:129
#2  0x00007fffbb7020aa in clang::ASTDeclReader::VisitDecl (this=0x7fffaa7fb0b0, D=0x7fff94029eb8) at /home/kfunk/devel/src/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp:537
#3  0x00007fffbb703da6 in clang::ASTDeclReader::VisitNamedDecl (this=0x7fffaa7fb0b0, ND=0x7fff94029eb8) at /home/kfunk/devel/src/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp:609
#4  clang::ASTDeclReader::VisitValueDecl (this=0x7fffaa7fb0b0, VD=0x7fff94029eb8) at /home/kfunk/devel/src/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp:746
#5  clang::ASTDeclReader::VisitDeclaratorDecl (this=0x7fffaa7fb0b0, DD=0x7fff94029eb8) at /home/kfunk/devel/src/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp:759
#6  0x00007fffbb70863b in clang::ASTDeclReader::VisitVarDeclImpl (this=0x7fffaa7fb0b0, VD=0x7fff94029eb8) at /home/kfunk/devel/src/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp:1260
#7  0x00007fffbb701db5 in clang::ASTDeclReader::VisitVarDecl (this=0x7fffaa7fb0b0, VD=0x7fff94029eb8) at /home/kfunk/devel/src/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp:333
#8  clang::ASTDeclReader::VisitDecompositionDecl (this=0x7fffaa7fb0b0, DD=0x7fff94029eb8) at /home/kfunk/devel/src/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp:1352
#9  clang::declvisitor::Base<clang::declvisitor::make_ptr, clang::ASTDeclReader, void>::Visit (this=0x7fffaa7fb0b0, D=0x7fff94029eb8) at tools/clang/include/clang/AST/DeclNodes.inc:445
#10 0x00007fffbb701332 in clang::ASTDeclReader::Visit (this=0x7fffaa7fb0b0, D=0x7fff94029eb8) at /home/kfunk/devel/src/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp:468
#11 0x00007fffbb7276e1 in clang::ASTReader::ReadDeclRecord (this=0x7fff9406b030, ID=19) at /home/kfunk/devel/src/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp:3623
#12 0x00007fffbb6c5d7d in clang::ASTReader::GetDecl (this=0x7fff9406b030, ID=19) at /home/kfunk/devel/src/llvm/tools/clang/lib/Serialization/ASTReader.cpp:7102
#13 0x00007fffbb727ccb in clang::ASTReader::PassInterestingDeclsToConsumer (this=0x7fff9406b030) at /home/kfunk/devel/src/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp:3674
#14 0x00007fffbb6dbefc in non-virtual thunk to clang::ASTReader::StartTranslationUnit(clang::ASTConsumer*) () at /home/kfunk/devel/src/llvm/tools/clang/lib/Serialization/ASTReader.cpp:7356
#15 0x00007fffbb821328 in clang::ParseAST (S=..., PrintStats=<optimized out>, SkipFunctionBodies=<optimized out>) at /home/kfunk/devel/src/llvm/tools/clang/lib/Parse/ParseAST.cpp:144
Comment 5 Kevin Funk 2017-10-16 07:01:00 UTC
The surprising thing is that libclang crashes, while plain clang does not:

Note: MWE is:
% cat ~/test.cpp                                                                                                                                          
#include "test.h"
% cat ~/test.h                                                                                                                                                  
#pragma once

[Serializable]
public ref class NeptuneException : Exception
{
};


% KDEV_CLANG_DISPLAY_ARGS=1 ~/devel/build/kf5/kdevelop/kdevplatform/util/duchainify/duchainify ~/test.cpp                                                   
Added 1 files to the background parser
parsing with 4 threads
Invocation: clang -ferror-limit=100 -fspell-checking -Wdocumentation -Wunused-parameter -Wunreachable-code -Wall -std=c++11 -nostdinc -nostdinc++ -xc++ -isystem/usr/include/c++/6 -isystem/usr/include/x86_64-linux-gnu/c++/6 -isystem/usr/include/c++/6/backward -isystem/usr/local/include -isystem/home/kfunk/devel/build/llvm/lib/clang/5.0.1/include -isystem/usr/include/x86_64-linux-gnu -isystem/usr/include -imacros /tmp/duchainify.J20863 /home/kfunk/test.cpp
libclang: crash detected during parsing: {
  'source_filename' : '/home/kfunk/test.cpp'
  'command_line_args' : ['clang', '-ferror-limit=100', '-fspell-checking', '-Wdocumentation', '-Wunused-parameter', '-Wunreachable-code', '-Wall', '-std=c++11', '-nostdinc', '-nostdinc++', '-xc++', '-isystem/usr/include/c++/6', '-isystem/usr/include/x86_64-linux-gnu/c++/6', '-isystem/usr/include/c++/6/backward', '-isystem/usr/local/include', '-isystem/home/kfunk/devel/build/llvm/lib/clang/5.0.1/include', '-isystem/usr/include/x86_64-linux-gnu', '-isystem/usr/include', '-imacros', '/tmp/duchainify.J20863'],
  'unsaved_files' : [],
  'options' : 781,
}
zsh: segmentation fault  KDEV_CLANG_DISPLAY_ARGS=1  ~/test.cpp

% clang -ferror-limit=100 -fspell-checking -Wdocumentation -Wunused-parameter -Wunreachable-code -Wall -std=c++11 -nostdinc -nostdinc++ -xc++ -isystem/usr/include/c++/6 -isystem/usr/include/x86_64-linux-gnu/c++/6 -isystem/usr/include/c++/6/backward -isystem/usr/local/include -isystem/home/kfunk/devel/build/llvm/lib/clang/5.0.1/include -isystem/usr/include/x86_64-linux-gnu -isystem/usr/include -imacros /tmp/duchainify.J20863 /home/kfunk/test.cpp
In file included from /home/kfunk/test.cpp:1:
/home/kfunk/test.h:3:1: warning: decomposition declarations are a C++17 extension [-Wc++17-extensions]
[Serializable]
^~~~~~~~~~~~~~
/home/kfunk/test.h:3:1: error: C++ requires a type specifier for all declarations
/home/kfunk/test.h:3:1: error: decomposition declaration cannot be declared with type 'int'; declared type must be 'auto' or reference to 'auto'
/home/kfunk/test.h:3:1: error: decomposition declaration '[Serializable]' requires an initializer
/home/kfunk/test.h:3:15: error: expected ';' after top level declarator
[Serializable]
              ^
              ;                                                                                                                                                                                                             
1 warning and 4 errors generated.
Comment 6 Craig Tenenbaum 2017-10-16 07:18:34 UTC
Yeah, I had been wondering why that is and frankly do not understand it, the offending fragment of code appears to be:

  // Marking a DecompositionDecl as invalid implies all the child BindingDecl's
  // are invalid too.
  if (DecompositionDecl *DD = dyn_cast<DecompositionDecl>(this)) {
    for (BindingDecl *Binding : DD->bindings()) {
      Binding->setInvalidDecl();
    }
  }

in Decl::setInvalidDecl(bool):clang/lib/AST/DeclBase.cpp

The issue is, when not in C++17, the reference returned by `DD->bindings()` seems to contain a nullptr, which is subsequently dereferenced with `Binding->setInvalidDecl()`. The resulting segfault should kill just about anything, right? What kind of insulation the clang driver must have is pretty special. Unless they somehow avoid being on that codepath entirely or somehow interfere with the erroneous initialization which must be occurring when otherwise not in C++17 mode. It has been my experience, thus far, that even invalid structured bindings do not result in this problem so long as the parser is in C++17 mode. Why there's no guard-check in that loop or safe initialization of that reference is as-yet unknown to me. I dunno if this would be considered sufficient information, yet, to file a bug report upstream.
Comment 7 Kevin Funk 2017-10-16 07:29:42 UTC
Reported upstream, with the limited information at hand:
  https://bugs.llvm.org/show_bug.cgi?id=34960
Comment 8 Rodrigo Fernandes 2018-09-14 15:38:09 UTC
This bug renders KDevelop unusable for me, as most projects I deal with now a days are C++17, and it would be unfeasable to manually add each of the offending files to the filter. Is there any way to set the parser to c++17 mode so the nullptr deref doesn't happen? Happy to dig submit patches to either Kdevelop / Clang to help remediate this.
Comment 9 Aaron Puchert 2020-09-07 21:49:33 UTC
The bug has been fixed, and the fix will be in Clang 11.