Bug 354311 - GCC compiler provider leads to C++ parser issues (missing highlighting e.g.)
Summary: GCC compiler provider leads to C++ parser issues (missing highlighting e.g.)
Status: RESOLVED FIXED
Alias: None
Product: kdevelop
Classification: Applications
Component: Language Support: CPP (Clang-based) (show other bugs)
Version: git master
Platform: Mageia RPMs Linux
: NOR normal
Target Milestone: 5.0.0
Assignee: kdevelop-bugs-null
URL:
Keywords: release_blocker
Depends on:
Blocks:
 
Reported: 2015-10-24 16:31 UTC by Piotr Mierzwinski
Modified: 2015-11-13 22:58 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
not colorized code (1) (90.78 KB, image/png)
2015-10-25 22:51 UTC, Piotr Mierzwinski
Details
not colorized code (2) (63.85 KB, image/png)
2015-10-25 22:52 UTC, Piotr Mierzwinski
Details
not colorized code (3) (49.58 KB, image/png)
2015-10-25 22:52 UTC, Piotr Mierzwinski
Details
KDevelopSessions - constructor (1) (114.30 KB, image/png)
2015-10-28 18:50 UTC, Piotr Mierzwinski
Details
KDevelopSessions - constructor (2) (121.75 KB, image/png)
2015-10-28 18:51 UTC, Piotr Mierzwinski
Details
KDevelopSessions::loadSessions() (1) (54.18 KB, image/png)
2015-10-28 18:52 UTC, Piotr Mierzwinski
Details
KDevelopSessions::loadSessions() (2) (54.64 KB, image/png)
2015-10-28 18:53 UTC, Piotr Mierzwinski
Details
KDevelopSessions::match(Plasma::RunnerContext &context) (74.81 KB, image/png)
2015-10-28 18:54 UTC, Piotr Mierzwinski
Details
KDevelopSessions - "problems" (155.32 KB, image/png)
2015-10-28 18:55 UTC, Piotr Mierzwinski
Details
KDevelopSessions "problems" + Show Imports (1) (244.36 KB, image/png)
2015-11-03 21:07 UTC, Piotr Mierzwinski
Details
KDevelopSessions "problems" + Show Imports (2) (278.66 KB, image/png)
2015-11-03 21:08 UTC, Piotr Mierzwinski
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Piotr Mierzwinski 2015-10-24 16:31:37 UTC
I my project (Qt4 based located at: git://git.qtcmd.org/qtcmd2.git) I can observe that many lines in many cpp file are not colorized, whereas in headers all is fine. The code is fully compilable (in KDevelop 5) and application it's working.
I observed that in one function/method the same class one global member is colorized and in second function it didn't. This is very wired. Second thing which are not colorized are local functions (in its class). Here is the same situation like with member (in one method the same function is colorized and in second didn't). I've attached couple of screenshots to visualize the situation.
The members are type QString, QMap, QWidget and other.
I thought this is because my project is in Qt4, but the same I've observed opening KDevelop project (master branch). I was checking some file from debugger implementation.
This is very annoying, because in not colorized code the helpful KDevelop functions, like jump to definition, not working. After refresh of file some of members are underlined as failed.

I've cleaned KDevelop cache (kdevduchain) before I open the project.
I've recreated also project files removing old project file and open it (project) again in KDevelop. Chose is parser C++11. I'm not able to change on older C++ standard. My project is based on cmake. qmake is set correctly on qt4 version, because I'm able to compile the project from the scratch. There is no set any neither "Includes/Imports" nor "Defines" in project settings. Compiler is gcc (auto-detected). In configuration of "Clang Language support" all options are set.

KDevelop 4.7.x (its parser) coloring the same code correctly.

Reproducible: Always

Steps to Reproduce:
1. Run KDevelop
2. open any bigger (several dozen of kilobytes) cpp file 


Actual Results:  
There are black lines in code (not colorized)

Expected Results:  
All should be colorized, because is compilable and previous KDevelop version does it correctly.

KDevelop cloned in afternoon on 2015-10-24.
Tested with Qt4 based project located at: git://git.qtcmd.org/qtcmd2.git
Comment 1 Piotr Mierzwinski 2015-10-25 22:51:46 UTC
Created attachment 95122 [details]
not colorized code (1)
Comment 2 Piotr Mierzwinski 2015-10-25 22:52:10 UTC
Created attachment 95123 [details]
not colorized code (2)
Comment 3 Piotr Mierzwinski 2015-10-25 22:52:38 UTC
Created attachment 95124 [details]
not colorized code (3)
Comment 4 Milian Wolff 2015-10-27 10:41:36 UTC
Do you see any problems regarding include files not being found? Also see: http://reviews.llvm.org/rL249982 which will improve this situation somewhat.
Comment 5 Piotr Mierzwinski 2015-10-28 18:49:13 UTC
For me it seems that include files are found (are included directly in the same file). I think this is not the root cause. The code is fully compilable.
I would like to illustrate the problem one more time.

I attached couple of screens referring to kdevelopsessions.cpp, that show strange behavior.
On the other side you can always open this file in KDevelop 5 beta and check how parser works.

Example line from constructor:
    KDirWatch *historyWatch = new KDirWatch(this);
This line is fully colorized. Properly header is included on top of this file. But just couple lines below the same pointer (historyWatch) in not colorized (is black) - why?. The same like constants from class KDirWatch (used in connect macro).
Next line is pretty colorized:
    foreach (const QString &dir, sessiondirs) {
But looking to its body we can see that nothing is colorized. Why?

Above situation is happen just after opening mentioned file. Please also note, I've rebuilt KDevelop (brach 5.0, cloned today afternoon) and opened project in running the same version of KDevelop (built several dozen minutes before). After opening the project I was waiting till KDevelop finishes rebuild its cache. All is visible on attached screen: "kdevelop_-_not_clorized_code_4.1"

Time for a strange behavior. I refreshed the file (using F5). The result you can check looking into next attached screen: "kdevelop_-_not_clorized_code_4.2". Now we can see that magically mentioned pointer and constants are colorized, but before pretty colorized line: "KDirWatch *historyWatch = new KDirWatch(this);" has underlined constructor and error sounding like: "No matching constructor for initialization of KDirWatch" (all of them you can check in attached: "kdevelopsessions_problems"). So now constructor is wrong and before refresh was fine :/. A bit strange. I don't want to describe next things which are visible on the screen-shot.

Next thing. Global member is colorized in header file and not colorized in cpp file. Let's check the "m_session" member. I don't attach screen from header file, because this member is colorized there. Look at screen-shot: "kdevelop_-_not_clorized_code_5.1", which shows definition of method "void KDevelopSessions::loadSessions()". Note, this is view for time before refreshing the file. You can see that member "m_session" is black (not colorized). The same on begin of method "void KDevelopSessions::match(Plasma::RunnerContext &context)". Here is also not colorized. Going further you can see that uses of local members in body of foreach loop also are not colorized ("session", "cfg"). So let refresh file. This is present on the next attachment - "kdevelop_-_not_clorized_code_5.2". I will not comment, all is visible. The next screen "kdevelop_-_not_clorized_code_5.2" shows next function "KDevelopSessions::match(Plasma::RunnerContext &context)", where "m_session" member is colorized (as parameter of foreach loop). Why here can be colorized and before can't?
Colorized declaration in header file:
        QVector<Session> m_sessions;
The problems reported in KDevelop are in another screen-shot: "kdevelopsessions_problems"

After these tests my conclusion is parser doesn't work properly. Despite code is fully compilable some part is colorize, some doesn't. In my Qt4 based project its behavior is the same, whereas in KDevelop 4.7.x I can't observe such things and all is fine colorized.

In this moment I have installed following versions of llvm:
libllvm-devel-3.7.0-2.mga6
llvm-3.7.0-2.mga6
libllvm3.7-3.7.0-2.mga6

For KDevelop project "C/C++ parser" in configuration is set on "C++11" value."

About fix in llvm.
I'm not sure how fast changes from "trunk" in llmv project might be released as stable and then when will be packaged by Linux distributions. I think not too fast.

By the way. Recently at evening I'm always on irc, channel kdevelop, nick: "piotrm", so I could give you some more details if required.
Comment 6 Piotr Mierzwinski 2015-10-28 18:50:08 UTC
Created attachment 95186 [details]
KDevelopSessions - constructor (1)
Comment 7 Piotr Mierzwinski 2015-10-28 18:51:37 UTC
Created attachment 95187 [details]
KDevelopSessions - constructor (2)
Comment 8 Piotr Mierzwinski 2015-10-28 18:52:43 UTC
Created attachment 95188 [details]
KDevelopSessions::loadSessions() (1)
Comment 9 Piotr Mierzwinski 2015-10-28 18:53:19 UTC
Created attachment 95189 [details]
KDevelopSessions::loadSessions() (2)
Comment 10 Piotr Mierzwinski 2015-10-28 18:54:11 UTC
Created attachment 95190 [details]
KDevelopSessions::match(Plasma::RunnerContext &context)
Comment 11 Piotr Mierzwinski 2015-10-28 18:55:04 UTC
Created attachment 95191 [details]
KDevelopSessions - "problems"
Comment 12 Milian Wolff 2015-11-01 15:16:57 UTC
First, please note that include paths in KDevelop, as well as issues shown in the editor, are not necessarily related to whether or not your code is actually compilable. There may be bugs in KDevelop that result in the wrong include paths being passed to clang, or bugs in the clang-c API that drop parts of the AST, or ...

So whether or not your code compiles is for the sake of this bug report unimportant.

Now, I've just opened kdevsessions.cpp and could not reproduce your issues. If you open the "Problems" toolview, and then click on "show imports" and go through the list - are there any errors shown in the list?
Comment 13 Piotr Mierzwinski 2015-11-03 21:06:20 UTC
I mentioned about code is compilable to eliminate the reason of this problem could be bugs in code.

I nothing added into project configuration (I'm talking about opened KDevelop project). All settings are default. So my configuration looks as following:
Configure CMake settings
 - everything is default. If you need I can provide all my default settings (as screen-shots) or in different way.
Configure Make settings
 - Abort of first error: checked
 - Number of simultaneous jobs: 2
 - Make executable: make
 - Active environment profile: default
Configure Language Support
 - Includes/Imports: empty
 - Defines: empty
 - Compilers: Auto-detect GCC
 - C/C++parser: Predefined profile: C++11
 "Reparse project" is checked. Compiler for path: GCC

Used tools: KDevelop and KDevPlatform was cloned at 28.10.2015 and were built using:
$ cmake --version
 cmake version 3.4.0-rc2
$ gcc --version
 gcc (GCC) 5.2.1 20151020
$ clang --version
clang version 3.7.0 (tags/RELEASE_370/final)
Target: i586-mageia-linux-gnu
Thread model: posix
$ rpm -qa | grep llvm
 libllvm-devel-3.7.0-2.mga6
 llvm-3.7.0-2.mga6
 libllvm3.7-3.7.0-2.mga6
$ rpm -qa | grep clang
 libclang3.7.0-3.7.0-2.mga6
 clang-3.7.0-2.mga6
 libclang-devel-3.7.0-2.mga6

Described here problem I met in KDevelop built from sources. Recently in my distribution were released packages for KDevelop 4.90.90, so I decided to check them (on second machine). Thus I have installed them and re-tested (also with project KDevelop). Unfortunately I got the same result :(.

And back to your question.
In opened Problems toolview I have 29 problems (before I attached screen-shot) and when I turn on "Show Imports" then I get 158 problems!
There are over a dozen TODO and FIXME, couple for qt5, but the most are related to compiler (gcc-5.2.1). I attached screen-shots that you would be able to see what kind of problems are. And by the way. When I refresh file, then I get 160 problems :/.

I don't know what is your configuration (compiler, clang and llvm version) maybe it is matter here.
Comment 14 Piotr Mierzwinski 2015-11-03 21:07:33 UTC
Created attachment 95300 [details]
KDevelopSessions "problems" + Show Imports (1)
Comment 15 Piotr Mierzwinski 2015-11-03 21:08:00 UTC
Created attachment 95301 [details]
KDevelopSessions "problems" + Show Imports (2)
Comment 16 Milian Wolff 2015-11-03 21:15:43 UTC
Comment on attachment 95301 [details]
KDevelopSessions "problems" + Show Imports (2)

ah I have an idea what the reason might be here - you have only gcc installed and not clang, correct? when you go to 

project -> open configuration -> language support -> Compilers

what is selected in the bottom right corner? "Compiler for path: [Clang]" or is it "... [GCC]"? What is auto-detected by kdevelop in the view above? Can you install Clang (if not done so already) and then select it in this config dialog for your project root and let kdevelop reparse (or stop kdevelop, wipe the cache in ~/.cache/kdevduchain, then restart). Does this improve your situation?
Comment 17 Milian Wolff 2015-11-03 21:16:53 UTC
Just a further note: If this indeed helps with your situation, I at least know how to reproduce and will then try to improve your situation - I still want kdevelop to support gcc (and other toolchains) - I fear we'll need to provide some macro shims for GCC compatibility then.
Comment 18 Piotr Mierzwinski 2015-11-03 23:39:32 UTC
[MW] ah I have an idea what the reason might be here - you have only gcc installed and not clang, correct? 
[PM] Actually in moment of test, yes. Only gcc.
[MW] when you go to project -> open configuration -> language support -> Compilers what is selected in the bottom right corner? "Compiler for path: [Clang]" or is it "... [GCC]"?
[PM] Yes.
[MW] What is auto-detected by kdevelop in the view above? 
[PM] GCC
I mentioned about it above in paragraph: "Configure Language Support" :/.

When I finished write previous post I installed clang compiler. And I added info about it into description, it could be a bit confusing. I didn't change it (compiler) in Project Configuration. Later (after your answer) I did it and pressed Apply, OK. Project was re-parsing, but it takes a long of time, so I stopped it (closing KDevelop), removed ~/.cache/kdevduchain and started KDevelop again. This definitely takes less time, what as I observed. Done. Of course now all is fine with kdevelopsession.cpp :-). Oh there are couple  exceptions.
On top of file there is:
bool kdevelopsessions_runner_compare_sessions(const Session &s1, const Session &s2) {
    QCollator c;
    return c.compare(s1.name, s2.name) < 0;
}
I think QCollator is not colorized. And when I move cursor over this class name then nothing happen (nothing is shown), but when I put there for example "QWidget w;" then it is properly colorized and helper tool tip is showing after cursor will be over "QWidget".
Second thing.
KDevelopSessions::KDevelopSessions(QObject *parent, const QVariantList& args)
    : Plasma::AbstractRunner(parent, args)
Here namespace: "Plasma" doesn't not colorized, whilst in method "KDevelopSessions::match" is use of "if" body does it. Check below:
        if (listAll || (!term.isEmpty() && session.name.contains(term, Qt::CaseInsensitive))) {
            Plasma::QueryMatch match(this);
Here "Plasma" is colorized.

Similar situation happens in file: languages/clang/clanghighlighting.cpp for "KDevelop" namespace (in calling following constructors: "KDevelop::CodeHighlightingInstance" and "KDevelop::CodeHighlighting". In other instances, KDevelop is colorized. Check this code:

ClangHighlighting::Instance::Instance(const KDevelop::CodeHighlighting* highlighting)
    : KDevelop::CodeHighlightingInstance(highlighting)
{
}

ClangHighlighting::ClangHighlighting(QObject* parent)
    : KDevelop::CodeHighlighting(parent)
{
}
Comment 19 Piotr Mierzwinski 2015-11-03 23:57:46 UTC
I checked also my own Qt4 project (located at git://git.qtcmd.org/qtcmd2.git). To be sure, all path are OK, I added couple of Includes/Imports related to Qt4 headers (QtGui, QtCore, QtMultimedia, QtNetwork). And almost everything was colorizing fine, except name of signals and slots inside macro connect/disconnect. Example:
	connect(m_pFindFileDlg, SIGNAL(signalStartFind(const FindCriterion &)), this, SLOT(slotStartFindFiles(const FindCriterion &)) );
So here "signalStartFind" and "slotStartFindFiles" are not colorized. Additionally arguments of these functions are not colorized as well. Hence I can't use "Jump to definition" being by cursor over slot name :/. Just I remind that KDevelop4 handles it correctly.

I was testing more and found next issues.
1. There is the code (src/filelistview.cpp:
	QVBoxLayout *pMainLayout = new QVBoxLayout(this);
	pMainLayout->addWidget(m_widget1);
	pMainLayout->addWidget(m_widget2);
	pMainLayout->addWidget(m_widget3);
...
	pMainLayout->setContentsMargins(0, 0, 0, 0);
Here "addWidget" are not colorized, but "setContentsMargins" is. Why?

2. There is code (a bit more complicated than above):
   QModelIndex srcIndex = static_cast <const VfsFlatListProxyModel *>(m_tvFileList->currentIndex().model())->mapToSource(m_tvFileList->currentIndex());
Here "currentIndex" and "mapToSource" is not colorized.

3. There is such class (in separate file "treeview.h"):
class TreeView : public QTreeView
{
public:
	explicit TreeView( QWidget* parent = 0 ) : QTreeView(parent) {}

protected:
	virtual bool viewportEvent(QEvent* event);
	virtual void mousePressEvent( QMouseEvent *event );

};

In different class: FileListView (its header) I have following declaration:
	TreeView              *m_tvFileList;
and I include header "treeview.h"
In one of method located in definition of FileListView there are following calling:
	m_tvFileList = new TreeView(this);
	m_tvFileList->setItemsExpandable(false);
	m_tvFileList->setRootIsDecorated(false);
	m_tvFileList->installEventFilter(this);
	m_tvFileList->setAlternatingRowColors(m_bAlternatingRowColors);
	m_tvFileList->setEditTriggers(QAbstractItemView::NoEditTriggers);
	m_tvFileList->setSortingEnabled(true); // sorting is always enabled
where only "installEventFilter" is colorized. Why?
Later (in other method) there is such calling:
			m_tvFileList->setCurrentIndex(m_currentIndex);
where "setCurrentIndex" is not colorized. I observed that other calling of methods for m_tvFileList are not colorized. For me this is weired. Is this correct behaviour? Please note that "qtreeview" header is placed in QtGui header which is included in project configuration.
Of course I set Clang in "Compiler in path" and rebuilt project. In Problems with turning on "Show Imports" there is nothing interested besides issues of type "Unused parameter", TODO, and FIXME. If above description is not clear I recommend open mentioned project in new KDevelop.
Comment 20 Milian Wolff 2015-11-04 09:58:02 UTC
OK, I think we found the big issue then. Could you do me a favor and report two new bugs, one  for the QCollator case and one for the SIGNAL/SLOT issue. Note that I could reproduce both already.

This way, we can keep track of the issues separately, as they are unrelated from the big issue at hand here that you reported initially, namely GCC incompatibility.

Thanks again for your high quality bug reports, this helps us a lot!
Comment 21 Piotr Mierzwinski 2015-11-04 19:40:42 UTC
Bug 354846 is referring to QCollator
Bug 354854 is referring to SIGNAL/SLOT.

I will report also other issues what I found yesterday and today. All referring to colorizing and not showing tool tip.
Sometime just enough open some file from project and scrolling it easily find issue.
Comment 22 Milian Wolff 2015-11-13 22:58:25 UTC
Git commit 170edf2537c07233ed7cea52f10ff016c72c7a2e by Milian Wolff.
Committed on 13/11/2015 at 22:58.
Pushed by mwolff into branch '5.0'.

Improve GCC compatibility by adding stubs for some builtins.

These are important when parsing the GCC provided intrinsics for
SSE & friends. The new unit test ensures that it works as intended.

Some caveats:

- This won't work properly on non-ia32 platforms.
- Potentially we need similar things for compatibility with the Intel
  compiler.
- There are still some errors generated in xmmintrin.h b/c some
  builtins are defined by both, GCC and Clang, but the latter are
  stricter. Thus we get errors reported when the GCC intrinsics try to
  pass const pointers as non-const parameters etc.

M  +2    -0    languages/clang/duchain/CMakeLists.txt
A  +1133 -0    languages/clang/duchain/gcc_compat.h     [License: LGPL (v2+)]
M  +16   -0    languages/clang/duchain/parsesession.cpp
M  +77   -1    languages/clang/tests/test_duchain.cpp
M  +6    -0    languages/clang/tests/test_duchain.h

http://commits.kde.org/kdevelop/170edf2537c07233ed7cea52f10ff016c72c7a2e