Bug 405585

Does it have to do with this?: https://phabricator.kde.org/D18516

Created attachment 118967 [details]
Kate 17 (leap) vs Kate 18 (tumbleweed): Image of block highlighting

Created attachment 118968 [details]
Kate 17 (leap) vs Kate 18 (tumbleweed): Image of top level code folding

I'll take a look tomorrow in more detail, but it looks like it might be. It seems the end is one-off.  The snippet of validated xml below focuses on highlighting blocks/code folding (see original attachment for xml and sample test file):

<context attribute="Normal"           lineEndContext="#stay"        name="_SingleLineTitle">
  <RegExpr attribute="TitleSection0"  context="#stay"               String="^(={1})\s+\S(?:.*[^=])(?:\s+\1)?$"
             beginRegion="SingleLineFolding" endRegion="SingleLineFolding" />
  <RegExpr attribute="TitleSection1"  context="#stay"               String="^(={2})\s+\S(?:.*[^=])(?:\s+\1)?$"
             beginRegion="SingleLineFolding" endRegion="SingleLineFolding" />
  <RegExpr attribute="TitleSection2"  context="#stay"               String="^(={3})\s+\S(?:.*[^=])(?:\s+\1)?$"
             beginRegion="SingleLineFolding" endRegion="SingleLineFolding" />
  <RegExpr attribute="TitleSection3"  context="#stay"               String="^(={4})\s+\S(?:.*[^=])(?:\s+\1)?$"
             beginRegion="SingleLineFolding" endRegion="SingleLineFolding" />
  <RegExpr attribute="TitleSection4"  context="#stay"               String="^(={5})\s+\S(?:.*[^=])(?:\s+\1)?$"
             beginRegion="SingleLineFolding" endRegion="SingleLineFolding" />
</context>

I am running OpenSuSE Leap 15 which has Kate 17.12.3 (folding ok) and OpenSuSE Tumbleweed 20190315 which has Kate 18.12.3 (folding not ok). I am using Kate's Help > About Kate to obtain Kates version, as well as KDE Framework and QT versions as originally reported.

The problem occurs when using beginRegion+endRegion in the same rule.
Ex:

<StringDetect String="hello" endRegion="region1" beginRegion="region2" />


In this case, the string "hello" is part of the "region1" and "region2" regions simultaneously, which generates folding problems. The behavior you expect (and the ideal behavior) is that "hello" isn't part of "region1", but of "region2".

I have checked and there are many XML files that use endRegion+beginRegion and in which there is an incorrect folding due to this bug.

Proposed patch: https://phabricator.kde.org/D19999

Git commit a24144952688f6e6b8700d5c86c563264715f8a7 by Christoph Cullmann, on behalf of Nibaldo González.
Committed on 24/03/2019 at 13:07.
Pushed by cullmann into branch 'master'.

Fix endRegion folding in rules with beginRegion+endRegion (use length=0)

Summary:

There is a problem of folding in rules with beginRegion + endRegion.

Example:
```
<StringDetect String="hello" endRegion="region1" beginRegion="region2" />
```
In this case, the "hello" string is part of the "region1" and "region2" regions simultaneously, which generates folding problems. The ideal behavior is that "hello" isn't part of "region1", but of "region2". That is, don't publish length of endRegion folding in rules with beginRegion+endRegion.

For example, in `.desktop` files:
{F6715422}

The following XML files use rules with beginRegion + endRegion:
* ansforth94.xml
* apparmor.xml
* asciidoc.xml
* asp.xml c.xml
* cisco.xml
* desktop.xml
* fortran.xml
* haml.xml
* ini.xml
* isocpp.xml
* julia.xml
* lex.xml
* lua.xml
* makefile.xml
* perl.xml
* pony.xml
* rhtml.xml
* rpmspec.xml
* ruby.xml
* template-toolkit.xml
* txt2tags.xml
* yacc.xml

Reviewers: #framework_syntax_highlighting, cullmann, dhaumann, vkrause

Reviewed By: #framework_syntax_highlighting, cullmann

Subscribers: kwrite-devel, kde-frameworks-devel

Tags: #kate, #frameworks

Differential Revision: https://phabricator.kde.org/D19999

M  +2    -2    autotests/folding/Makefile.fold
M  +2    -2    autotests/folding/folding.cpp.fold
M  +1    -1    autotests/folding/highlight.asp.fold
M  +20   -20   autotests/folding/highlight.cpp.fold
M  +1    -1    autotests/folding/highlight.f90.fold
M  +2    -2    autotests/folding/highlight.lex.fold
M  +3    -3    autotests/folding/highlight.pl.fold
M  +1    -1    autotests/folding/highlight.pony.fold
M  +1    -1    autotests/folding/highlight.prg.fold
M  +19   -19   autotests/folding/highlight.rb.fold
M  +3    -3    autotests/folding/highlight.spec.fold
M  +7    -7    autotests/folding/highlight.t2t.fold
M  +2    -2    autotests/folding/highlight.y.fold
M  +15   -1    autotests/folding/test.desktop.fold
M  +3    -3    autotests/folding/test.ini.fold
M  +6    -6    autotests/folding/usr.bin.apparmor-profile-test.fold
M  +14   -0    autotests/html/test.desktop.html
M  +14   -0    autotests/input/test.desktop
M  +14   -0    autotests/reference/test.desktop.ref
M  +9    -2    src/lib/abstracthighlighter.cpp

https://commits.kde.org/syntax-highlighting/a24144952688f6e6b8700d5c86c563264715f8a7

The patch will be included in KDE Frameworks 5.57, available approx. on April 13, 2019

How can I test this patch?

OpenSuSE Tumbleweed just updated libKF5SyntaxHighlighting5 to 5.56.

I looked at KDE downloads (https://neon.kde.org/download), unstable and neon via docker. Unstable is dated: 17-Mar-2019. I'm not sure about neon/docker.

Is there doc?

The git-unstable Neon ISO should be updated.

You can also compile KSyntaxHighligting and copy "libKF5SyntaxHighlighting.so.5.57.0" in /usr/lib64/ or /usr/lib/x86_64-linux-gnu/,
or install it.

https://phabricator.kde.org/source/syntax-highlighting/

(In reply to prusselltechgroup from comment #8)
> How can I test this patch?
> 
> OpenSuSE Tumbleweed just updated libKF5SyntaxHighlighting5 to 5.56.

You can just use the Tumbleweed-based Krypton .iso, it uses KDE:Unstable repos with builds from git master: https://en.opensuse.org/SDB:Krypton_and_Argon

(In reply to Fabian Vogt from comment #10)
> (In reply to prusselltechgroup from comment #8)
> > How can I test this patch?
> > 
> > OpenSuSE Tumbleweed just updated libKF5SyntaxHighlighting5 to 5.56.
> 
> You can just use the Tumbleweed-based Krypton .iso, it uses KDE:Unstable
> repos with builds from git master:
> https://en.opensuse.org/SDB:Krypton_and_Argon

Whoops, that was a broken link: https://en.opensuse.org/SDB:Argon_and_Krypton

I tested neon-unstable-20190401-1143 in a VM. 

And the patch fixed it. Kate syntax highlighting works as it has since KDE 4:

View > Code Folding > Fold Top Level Nodes 
and
View > Code Folding > Unfold Top Level Nodes 

I'll definitely look into opensuse tumbleweed repos for testing.

A couple of cleanup items:

1. I had a typo in my first entry. The current directory location for a user's own or override syntax highlight definition file, on linux, is:

     $HOME/.local/share/org.kde.syntax-highlighting/syntax/

The documentation is located here:

    https://docs.kde.org/trunk5/en/applications/katepart/highlight.html#idm45858413298848

The documentation says to use the command below to find your user folder ($HOME/.local/share):

    qtpaths --paths GenericDataLocation

When I ran the command on neon-unstable, I got:

    qtpaths: could not find a QT installation of ''

It did not prevent Kate from finding the syntax highlight definition file.

2. During the learning process, it was very helpful to read, grep/sed the system syntax highlight definition xml files. I read alert.xml to find which elements were available. I searched neon-unstable and didn't find any syntax highlight definition files. Doing an Internet search, I found the link below, which describes that they are now compiled into the KSyntaxHighlighting library.

    https://kate-editor.org/2018/03/10/improving-syntax-highlighting-files/

For myself and folks coming along, where would we find the xml source for the syntax highlight definition files?

The KSyntaxHighlighting library (part of KDE Frameworks) is used for the syntax highlighting engine in Kate and KDevelop:

https://kate-editor.org/2016/11/15/ksyntaxhighlighting-a-new-syntax-highlighting-framework/

The repository is located at: 
https://phabricator.kde.org/source/syntax-highlighting/ 

Or in GitHub (mirror): https://github.com/KDE/syntax-highlighting

In the folder "data/syntax/" there are the XML files. Also in "data/schema/" there is a script to validate the XML files

Also here (they are grouped according to the KDE Frameworks version): https://kate-editor.org/syntax/