| Summary: | Code parsing is totally broken | ||
|---|---|---|---|
| Product: | [Applications] kdevelop | Reporter: | Eugene Shalygin <eugene.shalygin+bugzilla.kde> |
| Component: | Language Support: CPP (Clang-based) | Assignee: | kdevelop-bugs-null |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | 6yearold, b.buschinski, ekigwana, mail, mpyne |
| Priority: | NOR | ||
| Version: | git master | ||
| Target Milestone: | --- | ||
| Platform: | Gentoo Packages | ||
| OS: | Linux | ||
| Latest Commit: | https://cgit.kde.org/kdevelop.git/commit/?id=716372ae2e8dff9c51e94d33443536786e4bd85b | Version Fixed In: | 5.3.0 |
| Sentry Crash Report: | |||
| Attachments: |
Take all clang++ include paths
A bit cleaner patch Use all Clang base include dirs |
||
|
Description
Eugene Shalygin
2018-05-02 19:50:29 UTC
#define KDEV_CLANG_BUILTIN_DIR "/usr/lib64/llvm/6/lib64/clang/6.0.0/include" in the generated file; the path is wrong. FYI, $ llvm-config --includedir /usr/lib64/llvm/6/include And yes, /usr/lib64/llvm/6/lib64 is the correct path for the clang libraries And the last piece: $ clang++ -c file.cc -v clang version 6.0.0 (tags/RELEASE_600/final) Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/lib/llvm/6/bin Selected GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0 Candidate multilib: .;@m64 Candidate multilib: 32;@m32 Selected multilib: .;@m64 "/usr/lib64/llvm/6/bin/clang-6.0" -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name file.cc -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -v -coverage-notes-file /tmp/file.gcno -resource-dir /usr/lib64/llvm/6/bin/../../../../lib/clang/6.0.0 -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7 -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/x86_64-pc-linux-gnu -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/backward -internal-isystem /usr/local/include -internal-isystem /usr/lib64/llvm/6/bin/../../../../lib/clang/6.0.0/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir /tmp -ferror-limit 19 -fmessage-length 171 -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o file.o -x c++ file.cc clang -cc1 version 6.0.0 based upon LLVM 6.0.0 default target x86_64-pc-linux-gnu ignoring nonexistent directory "/usr/local/include" ignoring nonexistent directory "/include" #include "..." search starts here: #include <...> search starts here: /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/x86_64-pc-linux-gnu /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/backward /usr/lib64/llvm/6/bin/../../../../lib/clang/6.0.0/include /usr/include End of search list. It almost looks a bit like there are two directories in CMAKE_LIBRARY_DIRS in cmake and they are concatenated ...? ... CLANG_LIBRARY_DIRS this was meant to read, of course What is the output of llvm-config --libdir? That is what is used by cmake to find this directory as far as I can see. CLANG_LIBRARY_DIRS is "/usr/lib64/llvm/6/lib64" and this path is correct, as I wrote above; the path is the same as returned by llvm-config --libdir. Hm. On my system, > llvm-config --includedir /usr/include and > llvm-config --libdir /usr/lib Both is definitely not what we want to put here. I don't know why this returns a path of a completely different kind on your system. Specifically, for you, --libdir includes an extra "llvm/6/lib64" suffix, which seems just ... different from what the utility gives here. I don't see how one would compute the correct path such that it works for both outputs. Hm. Gentoo packages allow parallel installation of several major LLVM/CLang version, ad each one can be 64 bit or 32, or both in parallel. Why don't you parse clang++ -v output to get the list of default include directories? Correcting the path in the config file does not solve the problem. It might be important that my clang is installed without libcxx. Created attachment 112386 [details]
Take all clang++ include paths
After adding all the include paths from clang++ -v output to Project properties/Language support/Includes" parsing works. My suggestions therefore is to take all those paths instead of one which resides in clang dir. A trivial patch is attached.
P.S. addFrameworkDirectories() in plugins/clang/duchain/parsesession.cpp line 180 seems to return a pointer to a local temporary variable.
Created attachment 112387 [details]
A bit cleaner patch
Created attachment 112389 [details]
Use all Clang base include dirs
One more update: try to find clang executable that matches found llvm version
The patch is invalid: the dirs have to be determined at run-time, because GCC upgrade changes them. The problem is that with your patch, we suddenly depend on clang being available. That isn't really required, we just need the headers. And what do you mean by "GCC upgrade changes the paths"? GCC shouldn't be involved here at all. Do you mean that the paths have to be obtained from libclang? GCC is involved simply because Clang is using its libstd++, and no matter what version of it was used to build Clang itself, Clang uses directories from currently active GCC. Yes, we have to use paths that fit to whatever libclang you are using. And libstdc++/libc++ is not important here. This is about the compiler builtin headers, such as these: $ ls /usr/lib/clang/6.0.0/include/ adxintrin.h avx512ifmaintrin.h avx512vpopcntdqintrin.h clzerointrin.h intrin.h pkuintrin.h __stddef_max_align_t.h __wmmintrin_pclmul.h altivec.h avx512ifmavlintrin.h avx512vpopcntdqvlintrin.h cpuid.h inttypes.h pmmintrin.h stdint.h x86intrin.h ammintrin.h avx512pfintrin.h avxintrin.h cuda_wrappers iso646.h popcntintrin.h stdnoreturn.h xmmintrin.h arm64intr.h avx512vbmi2intrin.h bmi2intrin.h emmintrin.h limits.h prfchwintrin.h tbmintrin.h xopintrin.h arm_acle.h avx512vbmiintrin.h bmiintrin.h f16cintrin.h lwpintrin.h rdseedintrin.h tgmath.h xsavecintrin.h armintr.h avx512vbmivlintrin.h cetintrin.h float.h lzcntintrin.h rtmintrin.h tmmintrin.h xsaveintrin.h arm_neon.h avx512vlbitalgintrin.h __clang_cuda_builtin_vars.h fma4intrin.h mm3dnow.h s390intrin.h unwind.h xsaveoptintrin.h avx2intrin.h avx512vlbwintrin.h __clang_cuda_cmath.h fmaintrin.h mmintrin.h shaintrin.h vadefs.h xsavesintrin.h avx512bitalgintrin.h avx512vlcdintrin.h __clang_cuda_complex_builtins.h fxsrintrin.h mm_malloc.h smmintrin.h vaesintrin.h xtestintrin.h avx512bwintrin.h avx512vldqintrin.h __clang_cuda_intrinsics.h gfniintrin.h module.modulemap stdalign.h varargs.h avx512cdintrin.h avx512vlintrin.h __clang_cuda_math_forward_declares.h htmintrin.h msa.h stdarg.h vecintrin.h avx512dqintrin.h avx512vlvbmi2intrin.h __clang_cuda_runtime_wrapper.h htmxlintrin.h mwaitxintrin.h stdatomic.h vpclmulqdqintrin.h avx512erintrin.h avx512vlvnniintrin.h clflushoptintrin.h ia32intrin.h nmmintrin.h stdbool.h __wmmintrin_aes.h avx512fintrin.h avx512vnniintrin.h clwbintrin.h immintrin.h opencl-c.h stddef.h wmmintrin.h We will have to find a way to cope with your strange libdir. When you query all the variables in llvm-config, is there any that returns /usr/lib64/? Or anything we could use to find /usr/lib64/clang/6.0.0/include/varargs.h? (In reply to Milian Wolff from comment #17) > Yes, we have to use paths that fit to whatever libclang you are using. And > libstdc++/libc++ is not important here. This is about the compiler builtin > headers, such as these: Have you seen comments 10 and 11? They seem to directly contradict your statement. > We will have to find a way to cope with your strange libdir. When you query > all the variables in llvm-config, is there any that returns /usr/lib64/? Or > anything we could use to find /usr/lib64/clang/6.0.0/include/varargs.h? $ llvm-config --libdir /usr/lib64/llvm/6/lib64 $ llvm-config --includedir /usr/lib64/llvm/6/include The problem is that the clang include dir contains full version number: /usr/lib64/clang/6.0.0/include ... and /usr/lib is symlinked to /usr/lib64 on my system. Yes, I've read your comments. I don't understand why they are contradicting what I said? Clang knows the path to the compiler-builtins, and when you use all of those then sure, the problems will vanish. But only because you are lucky and you used the clang that matches the version of your libclang. This is not necessarily the case. Actually, there are systems where you have libclang but no clang. E.g. on ubuntu: https://packages.ubuntu.com/search?searchon=contents&keywords=varargs.h&mode=exactfilename&suite=artful&arch=any => libclang-common-X.Y-dev Also see https://bugs.kde.org/show_bug.cgi?id=387005#c5 - the version match is important. We will need to find a way to lookup the path that works on all systems. And thinking about it, we shouldn't do it at compile time like we do now, since libclang is usually ABI compatible and if it gets updated without KDevelop being recompiled, things can break again... Sigh, this is such a mess :-/ (In reply to Milian Wolff from comment #20) > Yes, I've read your comments. I don't understand why they are contradicting > what I said? Can't understand you, sorry. I wrote that if I put the correct path to the clang includes dir in the generated by the configure_file(), but only that path, without the base GCC include dirs, the parsing show exactly the same problem, as reported in the comment #1. May we, please, clarify this part before moving on to the path detection task? (In reply to Eugene Shalygin from comment #21) > problem, as reported in the comment #1. In the comment #0 (description), of course. Ah, that is indeed an important detail which was not clear at all from the previous comments. So to make sure: You changed KDEV_CLANG_BUILTIN_DIR to /usr/lib64/clang/6.0.0/include/, then recompiled and installed KDevelop and reran your session with CLEAR_DUCHAIN_DIR=1 (or after removing your cache in ~/.cache/kdevduchain) and the problems persist? I somehow believe that you forgot to clean the cache? If that is not the case, then indeed we need to figure out why that happens. Can you please tell me what compiler you have selected in the includes-and-defines settings for your project? Also, please set KDEV_CLANG_DISPLAY_ARGS=1 and run kdevelop then paste one of the clang invocation command lines for a C++ files. Oh, and please also tell me where your STL includes live, e.g. where is the `string` header or such, take one of the headers where you see the "identifier is not found in the std:: namespace" error. Now let me explain once more how it *should* work (and how it works on non-gentoo machines): 1) we query the build system for include paths for the given file 2) we query the compiler selected by the user for its include paths - this includes the libstdc++ or libc++ include paths, whatever the compiler is using (or what the user specifies via the compiler args) - if a C compiler is used, there are no STL include paths of course - we filter out any paths that include varargs.h from this list 3) (at compile time) we try to find the path to the builtin headers for clang that match the version of libclang we are using Now we assemble these paths in that order, 1), 2), 3) and run the clang parse job. Git commit 169371b5d3448949122d5846c7da29ab86b1a0e9 by Milian Wolff. Committed on 04/05/2018 at 15:26. Pushed by mwolff into branch 'master'. Allow overriding the path to the builtin clang compiler headers The current code to detect the path for the builtin clang compiler headers fails on Gentoo. Allow setting the KDEV_CLANG_BUILTIN_DIR environment variable for such cases as a stop-gap measure to workaround this for now. We will have to find a better approach to this problem, it seems. Most notably, we will probably have to detect the path at runtime based on the libclang we actually use then. This may be different from the one we compiled against... For now, we also report an error (only visible on the command line) when the KDEV_CLANG_BUILTIN_DIR path is "wrong", i.e. doesn't contain the varargs.h header. In such cases, we disable the kdev-clang plugin, since it would be unusable anyways. M +10 -0 plugins/clang/clangsupport.cpp M +7 -0 plugins/clang/duchain/duchainutils.cpp M +1 -0 plugins/clang/duchain/duchainutils.h M +3 -2 plugins/clang/duchain/parsesession.cpp https://commits.kde.org/kdevelop/169371b5d3448949122d5846c7da29ab86b1a0e9 So, if you updated to current master, then please tell me what you get when you run kdevelop like this: CLEAR_DUCHAIN_DIR=1 KDEV_CLANG_BUILTIN_DIR=/usr/lib64/clang/6.0.0/include/ kdevelop do you still see the errors? Thank you for the last commit! It greatly simplified testing for me. It seems I made a mistake earlier, because now supplying only the clang include path works, as you assured it has to. Here is the list of include dirs I copied from the log: -isystem/usr/lib64/gcc/x86_64-pc-linux-gnu/8.1.0/include/g++-v8 -isystem/usr/lib64/gcc/x86_64-pc-linux-gnu/8.1.0/include/g++-v8/x86_64-pc-linux-gnu -isystem/usr/lib64/gcc/x86_64-pc-linux-gnu/8.1.0/include/g++-v8/backward -isystem/usr/include -I<project include dir> -isystem /usr/lib64/clang/6.0.0/include/ OK, good! I was worried for a moment there :) Now we just need to find a way to fix this for Gentoo... And do the lookup at runtime... I'm unable to find a direct reference to the include directory within files, that are installed as part of "libclangxxx" in other distributions. Only those belonging to the clang package contain this path. right, we may need to implement heuristics similar to what the other tools out there that use libclang do. cf.: https://github.com/Rip-Rip/clang_complete/issues/238 If it is possible to get libclang version at runtime, I would add a configuration parameter to pass a regular expression that transforms the clang version string into the includes directory path. I hope distributions maintain the same scheme for various clang versions. That sounds super terrible to me. Why exactly can there not be an option in llvm-config which simply prints this path? Long-term fixing that in libclang itself would be ideal. For now, we need to support older libclang and llvm-config anyways, so coming up with a heuristic is required anyways. We could also just say: We depend on clang being available for the version of libclang we use at runtime, then query that clang and use its path. I'd be fine with that too. Why the added code checks for varargs.h? LLVM/Clang installed from FreeBSD packages doesn't contain this header in /include. All other includes seem to be in place. it was just a heuristic that worked well on my system :) do you have a better suggestion? (In reply to Milian Wolff from comment #34) > it was just a heuristic that worked well on my system :) do you have a > better suggestion? I've got an impression that this problem is specific to gentoo. At least on FreeBSD we don't need to locate that directory, as we do not package libclang separately from clang and even LLVM itself. But if you don't want to make this check conditional, I'd vote for looking for something like __clang_cuda_cmath.h Gentoo does package the complete Clang package too. Here's the patch we are applying to our Clang package: https://svnweb.freebsd.org/ports/head/devel/llvm60/files/clang/patch-tools_clang_lib_Headers_CMakeLists.txt?view=markup So, using vadefs.h instead of varargs.h would suffice for FreeBSD. current master now uses cpuid.h, does this work on gentoo? (In reply to Milian Wolff from comment #38) > current master now uses cpuid.h, does this work on gentoo? What should have changed? kdevplatform.shell: Could not load plugin "kdevclangsupport" , it reported the error: "The clang builtin include path \"/usr/lib64/llvm/6/lib64/clang/6.0.1/include\" is invalid (missing varargs.h header).\nTry setting the KDEV_CLANG_BUILTIN_DIR environment variable manually to fix this.\nSee also: https://bugs.kde.org/show_bug.cgi?id=393779" Disabling the plugin now. I can potentially help with testing here as I also use Gentoo and ran into this before trying the env var as suggested in the error message. Current master does not work under gentoo. It gives kdevplatform.shell: Could not load plugin "kdevclangsupport" , it reported the error: "The clang builtin include path \"/usr/lib64/llvm/7/lib64/clang/7.0.0/include\" is invalid (missing cpuid.h header).\nTry setting the KDEV_CLANG_BUILTIN_DIR environment variable manually to fix this.\nSee also: https://bugs.kde.org/show_bug.cgi?id=393779" Disabling the plugin now. which is not existing directory. Manually setting CLEAR_DUCHAIN_DIR=1 KDEV_CLANG_BUILTIN_DIR=/usr/lib64/clang/7.0.0/include/ kdevelop works fine. I think the solution is to use #define KDEV_CLANG_BUILTIN_DIR "@CLANG_INCLUDE_DIRS@" in the libclang_include_path.h instead of building the include dir manually. I will post a patch soon. :) https://phabricator.kde.org/D15895 hope this helps :) Git commit 716372ae2e8dff9c51e94d33443536786e4bd85b by Bernd Buschinski. Committed on 04/10/2018 at 18:36. Pushed by buschinski into branch '5.3'. Use CLANG_INCLUDE_DIRS for clang include dir Summary: Use CLANG_INCLUDE_DIRS for clang include dir instead of manually building it. This fixes the error: kdevplatform.shell: Could not load plugin "kdevclangsupport" , it reported the error: "The clang builtin include path \"/usr/lib64/llvm/7/lib64/clang/7.0.0/include\" is invalid (missing cpuid.h header).\nTry setting the KDEV_CLANG_BUILTIN_DIR environment variable manually to fix this.\nSee also: https://bugs.kde.org/show_bug.cgi?id=393779" Disabling the plugin now. As CMake with CLANG_INCLUDE_DIRS already tests if this folder is present, it should work across all distros. Reviewers: #kdevelop, kfunk Reviewed By: #kdevelop, kfunk Subscribers: mwolff, aaronpuchert, brauch, kossebau, kfunk, arrowd, kdevelop-devel Tags: #kdevelop Differential Revision: https://phabricator.kde.org/D15895 M +20 -3 cmake/modules/FindClang.cmake M +1 -1 plugins/clang/libclang_include_path.h.cmake https://commits.kde.org/kdevelop/716372ae2e8dff9c51e94d33443536786e4bd85b Git commit 86bad1f222716c9a204423f29080298b05713b2c by Kevin Funk. Committed on 04/10/2018 at 21:42. Pushed by kfunk into branch 'wip/windows-fixes'. clang: Also detect Clang builtin dirs at runtime M +5 -1 plugins/clang/clangsupport.cpp M +43 -5 plugins/clang/duchain/clanghelpers.cpp M +2 -0 plugins/clang/duchain/clanghelpers.h https://commits.kde.org/kdevelop/86bad1f222716c9a204423f29080298b05713b2c Git commit ec5477f351b6c4b25b7f6533314628ebdd7c2945 by Kevin Funk. Committed on 05/10/2018 at 12:39. Pushed by kfunk into branch '5.3'. clang: Also detect Clang builtin dirs at runtime Summary: Changes: * Add a helper function retrieving the Clang version at runtime * Auto-detect bundled copy of Clang builtin headers on Windows * Print lots more debug output during startup TODO: Wondering if we want to add more code detecting this directory at runtime on other platforms, too. But for now, only Windows is interesting for us. Reviewers: kossebau, aaronpuchert, brauch Reviewed By: brauch Subscribers: brauch, kdevelop-devel Tags: #kdevelop Differential Revision: https://phabricator.kde.org/D15955 M +2 -0 plugins/clang/clangsupport.cpp M +44 -5 plugins/clang/duchain/clanghelpers.cpp M +2 -0 plugins/clang/duchain/clanghelpers.h https://commits.kde.org/kdevelop/ec5477f351b6c4b25b7f6533314628ebdd7c2945 |