Bug 399348 - Adding delimiters to "additionalDeliminator" sometimes causes delimiters to stop being recognized
Summary: Adding delimiters to "additionalDeliminator" sometimes causes delimiters to s...
Status: RESOLVED FIXED
Alias: None
Product: frameworks-syntax-highlighting
Classification: Frameworks and Libraries
Component: syntax (show other bugs)
Version: 5.50.0
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: KWrite Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-10-03 18:24 UTC by Christopher Jacobs
Modified: 2019-08-15 18:37 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
Minimal syntax highlighter for testing additionalDeliminator. (1.42 KB, text/xml)
2018-10-03 18:24 UTC, Christopher Jacobs
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Christopher Jacobs 2018-10-03 18:24:20 UTC
Created attachment 115392 [details]
Minimal syntax highlighter for testing additionalDeliminator.

SUMMARY

There's some weirdness when the following characters are included as "additionalDeliminator"s, such that keyword matching appears to break down in some situations:

  - " (or ")
  - ' (or ')
  - `
  - @

The weird part is that the behavior of the `<keyword String=...` rule doesn't appear to be consistent when "additionalDeliminator" contains one or more of these values --- sometimes highlighting breaks, sometimes it doesn't. I've attached a minimal syntax highlighting file to illustrate what's happening. And here's the "test file" I'm using:

   keyword
   {keyword}
   `keyword`
   'keyword'

As an example, in the test definition file I attached, there are a few separate lines defining some additional delimiters. All but one are commented out, but by commenting each one in, you can see where things get a little weird:

  Case 1: additionalDeliminator=""

   The keyword is highlighted 2 out of 4 times, missing the keyword between backticks and single quotes. Bummer, but we learned that we should include these as additional delimiters.

  Case 2: additionalDeliminator="`" or additionalDeliminator="&apos;"

    The keyword is highlighted 3 out of 4 times. Expected.

  Case 3: additionalDeliminator="&apos;`"

    The keyword is highlighted only 3 of 4 times again. This time it's missing the one between curly braces. Huh? How did we lose a delimiter?

  Case 4: additionalDeliminator="&apos;`{}"

    Same as case 3 (3 of 4 instances, highlighted). Even trying to add the curly braces back, explicitly, doesn't fix the issue. Just in case, I also tested that the order doesn't appear to matter here.

There's some other weird collisions that occur with the four characters I listed above and other characters, but as far as I can tell, as long as these four characters don't appear anywhere in the "additionalDeliminator" list, all other characters appear to be working together as expected. That is, I can include all non-identifier-like symbols easily reached on my American keyword and not appearing in the list above, like so:

    `additionalDeliminator="!#$%&amp;()*+,./:;&lt;=&gt;?[\]^{|}~"`

and everything works as expected. Once a single quote gets in there, though:

    `additionalDeliminator="!#$%&amp;&apos;()*+,./:;&lt;=&gt;?[\]^{|}~"`

it breaks again.

  
SOFTWARE VERSIONS
(available in About System)
KDE Plasma Framework Version: 5.50.0
KDE Syntax-highlighting Framework Version: 5.50.0
Qt Version: 5.11.2
Comment 1 Christoph Cullmann 2019-08-15 18:37:32 UTC
Git commit ff06ba217e403b21055cae1baf7f4751a582c987 by Christoph Cullmann, on behalf of Jonathan Poelen.
Committed on 15/08/2019 at 18:37.
Pushed by cullmann into branch 'master'.

fix additionalDeliminator attribute

Summary:
The initialization of the `Rule::m_wordDelimiter` cache was done before the delimiters were extracted, which can invalidate the character number.

Reviewers: #framework_syntax_highlighting, dhaumann, cullmann

Reviewed By: #framework_syntax_highlighting, cullmann

Subscribers: kwrite-devel, kde-frameworks-devel

Tags: #kate, #frameworks

Differential Revision: https://phabricator.kde.org/D23134

A  +4    -0    autotests/folding/fix_d399348.addDelim.fold
A  +11   -0    autotests/html/fix_d399348.addDelim.html
A  +4    -0    autotests/input/fix_d399348.addDelim
A  +24   -0    autotests/input/syntax/fix_d399348_additional_deliminators.xml
A  +4    -0    autotests/reference/fix_d399348.addDelim.ref
M  +3    -3    src/lib/rule.cpp

https://commits.kde.org/syntax-highlighting/ff06ba217e403b21055cae1baf7f4751a582c987