Bug 418778

Summary:	Very slow VHDL syntax highlighting
Product:	[Frameworks and Libraries] frameworks-syntax-highlighting	Reporter:	Gernot Gebhard <gg>
Component:	syntax	Assignee:	KWrite Developers <kwrite-bugs-null>
Status:	RESOLVED FIXED
Severity:	normal	CC:	christoph
Priority:	NOR
Version First Reported In:	5.66.0
Target Milestone:	---
Platform:	Manjaro
OS:	Linux
Latest Commit:	https://commits.kde.org/syntax-highlighting/764957b1d23c48a160f0faca623843f678eb2a52	Version Fixed/Implemented In:
Sentry Crash Report:
Attachments:	Flamegraph for highlighting /lib/gaisler/leon3v3/iu3.vhd

Description Gernot Gebhard 2020-03-12 12:07:08 UTC

Created attachment 126746 [details]
Flamegraph for highlighting /lib/gaisler/leon3v3/iu3.vhd

SUMMARY

The syntax highlighting for VHDL files takes very long to complete.


STEPS TO REPRODUCE

1. Download and extract https://www.gaisler.com/products/grlib/grlib-gpl-2019.4-b4246.tar.gz (from https://www.gaisler.com/index.php/downloads/leongrlib)
2. Use kate to open ./lib/gaisler/leon3v3/iu3.vhd from the extracted archive
3. Hit CTRL+END to scroll to the end of the file


OBSERVED RESULT

It takes _very_ long until the end of the file is actually reached.
On my machine this is ~30secs (CPU is an Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz)
Note that the file itself is only 190KB large.


EXPECTED RESULT

Much faster syntax highlighting.


SOFTWARE/OS VERSIONS

Linux: Manjaro
KDE Plasma Version: 5.17.5
KDE Frameworks Version: 5.66.0
Qt Version: 5.14.1 (built against 5.14.0)


ADDITIONAL INFORMATION

Attached is a flamegraph showing that most of the time is spent in libpcre2 (process has been terminated after scrolling to end of file has finished).
Hence I assume this is related to using regular expressions extensively in the syntax highlighting file.

Comment 1 Christoph Cullmann 2020-03-12 22:49:51 UTC

I think the issue is that most regex here are marked "dynamic=true" and for these kind of regex stuff we neither pre-compile nor cache the match indices...

Comment 2 Christoph Cullmann 2020-03-12 22:59:42 UTC

https://phabricator.kde.org/D28015

Comment 3 Christoph Cullmann 2020-03-13 18:37:30 UTC

Merged

https://phabricator.kde.org/D28015

but still slow ;=)

Comment 4 Christoph Cullmann 2020-03-13 19:03:08 UTC

Next try:

https://phabricator.kde.org/D28039

Now highlighted in under one second ;=)

Comment 5 Christoph Cullmann 2020-03-14 14:27:41 UTC

Git commit 764957b1d23c48a160f0faca623843f678eb2a52 by Christoph Cullmann.
Committed on 14/03/2020 at 14:28.
Pushed by cullmann into branch 'master'.

optimize dynamic regex matching

Summary:
allow such matches to be cached, too, by guarding the cache
with the last used captures

makes VHDL highlighting instantanious for bug 418778

Test Plan:
make && make test

example from bug 418778 is fast!

Reviewers: dhaumann, vkrause, nibags

Reviewed By: dhaumann

Subscribers: mwolff, kwrite-devel, kde-frameworks-devel

Tags: #kate, #frameworks

Differential Revision: https://phabricator.kde.org/D28039

M  +13   -1    src/lib/abstracthighlighter.cpp
M  +2    -3    src/lib/rule.cpp
M  +9    -3    src/lib/rule_p.h

https://commits.kde.org/syntax-highlighting/764957b1d23c48a160f0faca623843f678eb2a52