Created attachment 156303 [details] a log file that causes the issue Kate is at version 23.03.70 SUMMARY When editing certain files (doesn't have to be a file - I have the same issue with pasting text into a new editor), cursor movement or scrolling with the mouse is very slow, as an example: it takes ~2 seconds for the cursor to respond to HOME or END. I haven't been able to understand what features of these texts cause this behavior - to allow me to synthetically generate these files, but I can reliably get this issue to exhibit by getting text from AWS Cloudwatch logs, where when viewing a log stream I click Actions -> "Copy Search Results (ASCII)" (though weirdly, using "Copy Search Results (CSV)" does not result in problematic text even though it is very similar text in size and width). I'm attaching a sample file that shows this problem for me. STEPS TO REPRODUCE 1. Use Kate to open a file or paste text with some characteristics (see attached file for example) 2. Use the keyboard or mouse to move around the text. OBSERVED RESULT The movement is slow and jerky, and sometimes Kate freezes for a second or two (the compositor grayscales the window). Especially moving to start and end of line is very slow. EXPECTED RESULT Text movement should be responsive SOFTWARE/OS VERSIONS Linux/KDE Plasma: KDE Plasma Version: 5.27.80 KDE Frameworks Version: 5.104.0 Qt Version: 5.15.8 ADDITIONAL INFORMATION With the same file, gedit also behaves weirdly.
Can't reproduce on a quick try. Can you run kate using the perf profiler? Perhaps use https://github.com/KDAB/hotspot and attach to Kate and start recording before it gets slow. Once you stop the recording, it will show you a flamegraph which might explain more why this is happening.
(In reply to Waqar Ahmed from comment #1) > Can't reproduce on a quick try. > > Can you run kate using the perf profiler? Perhaps use > https://github.com/KDAB/hotspot and attach to Kate and start recording > before it gets slow. Once you stop the recording, it will show you a > flamegraph which might explain more why this is happening. I'll check that out.
(In reply to Waqar Ahmed from comment #1) > Can you run kate using the perf profiler? I ran hotspot with the repro and uploaded the perfdata file to here: https://code.geek.co.il/dumps/kate-paste-aws-log.perfparser.gz According to this report, the worst offender is `QTextLine::cursorToX(int*, QTextLine::Edge) const`, spending 20.9% of the test cycles by itself and 48.7% including its children.
Created attachment 156781 [details] Screenshot showing hotspot results
What happens if you disable the "Show Whitespace Indicators" option in Settings->Appearance?
A. Setting "show whitespace indicators" to "None" definitely solves the performance issue during cursor movement (pasting the problematic text is still way way too slow, but probably due to some other hotspot - I'll analyze later). B. Interestingly - it still shows whitespace tab characters.. ?!
Yeah, tab are treated differently. The problem is that the file you linked contains way too many trailing spaces in each line (2000+ sometimes). So, if you want to keep the spaces setting enabled, a good local fix would be to just rtrim the whole file by - Hit F7 - type "rtrim" and press enter This can be optimized in theory as the width of a space is constant. We can just use that to quickly paint all the spaces. Problem is that dynamic word wrapping makes things a bit trickier. Maybe we can use something like QTextOption::ShowSpacesAndTab + draw the spaces ourselves. That makes it a bit easier to optimize this case. ---- For part 2 => slow paste, I think this is because of "Indent on Paste". Do you have it enabled? Your profile data shows that it is enabled, but maybe its something else. I am not sure why indent on paste is getting executed on a `.log` file in the first place as it doesn't have an indenter.
(In reply to Waqar Ahmed from comment #7) > The problem is that the file you linked contains way too many trailing > spaces in each line (2000+ sometimes). So, if you want to keep the spaces > setting enabled, a good local fix would be to just rtrim Its not actually "trailing" spaces - the format is ASCII tabular format with lines starting and ending with a pipe character. naively rtriming it will not help - but thanks for pointing out a simple workaround. Generally, I'm less interested in workarounds (though the point about painting white spaces causing the issue did indeed help) and more in pointing at issues that may hinder KDE adoption :-) > This can be optimized in theory as the width of a space is constant. We can > just use that to quickly paint all the spaces. Problem is that dynamic word > wrapping makes things a bit trickier. Interestingly (or not - you decide), the word wrapping behavior of lines - such as in this example - where there's a few dozen text characters at the beginning, followed by thousands of spaces and a terminating non-white-space character - is that the line after the last wrap of actual text contains almost all the spaces remaining on the line, even though it exceeds the width of the window it doesn't wrap and there is no horizontal scrollbar (the additional spaces are inaccessible), then another dynamic wrap and the last 110 characters of the line (which includes 109 spaces and a terminating pipe). This 110 character width is consistent across all the lines in the file, regardless of how many dynamic wraps each line has. The spaces in the beginning of the white space span, before the last dynamic wrap, can't be selected using either keyboard or mouse. Looks really weird, but also ... intentional? > For part 2 => slow paste, I think this is because of "Indent on Paste". Do > you have it enabled? Your profile data shows that it is enabled, but maybe > its something else. That is correct. Disabling "Adjust indentation of code pasted from the clipboard" solves the "freeze on paste" issue. > I am not sure why indent on paste is getting executed on > a `.log` file in the first place as it doesn't have an indenter. Well - its not a log file when I paste it: I just open a new Kate buffer and paste some ASCII into - it has no way of knowing that I intend it to be a "log". I tried to set the syntax mode to "log (simplified)" (in the bottom right corner select box) but that didn't help as pasting still auto-indented the lines. Not that I understand why it indents the lines to begin with - all the lines start with the same character ("|") and end with the same character ("|").
> is that the line after the last wrap of actual text contains almost all the spaces remaining on the line, even though it exceeds the width of the window it doesn't wrap and there is no horizontal scrollbar (the additional spaces are inaccessible), Indeed. This is what I meant by "ShowSpacesAndTabs". We currently (or by default? dont remember) this is disabled. This option shows spaces and wraps them like normal text, otherwise trailing spaces are ignored and text isn't wrapped till there is a non-space char. Yes, this means we waste a huge amount of time painting spaces into the void or rather at the same position.... Whether this is intentional or not, I don't know. > Well - its not a log file when I paste it: I just open a new Kate buffer and paste some ASCII into - it has no way of knowing that I intend it to be a "log". Interesting, will check whats going on.
A possibly relevant merge request was started @ https://invent.kde.org/frameworks/ktexteditor/-/merge_requests/491
> Not that I understand why it indents the lines to begin with Do you have a default indenter set maybe? Check Configure Kate -> Editing -> Indentation What are your indenter settings? On a quick try I can't reproduce the slowness with Indent On Paste + Cstyle indenter.
Git commit 02a76278f9596e35cb99b42065a4a529b51a1992 by Waqar Ahmed. Committed on 02/03/2023 at 15:47. Pushed by cullmann into branch 'master'. Optimize rendering spaces with dyn wrapping When dynamic wrapping is enabled and a QTextLine has trailing spaces, then those spaces aren't really shown unless QTextOption::ShowTabsAndSpaces is enabled. Thus we shouldn't waste time rendering spaces at the same position. With this commit, rendering of spaces is split into 3 steps: 1. Collect all spaces in the line by iterating over the text in reverse 2. Iterate over all the collected spaces in reverse because we want to look at the spaces in the beginning of line first and convert each space position into a QPointF. Whenever we encounter a point which was equal to the last collected point, we stop the loop and break. 3. Render all the collected space points at once. M +30 -8 src/render/katerenderer.cpp M +1 -1 src/render/katerenderer.h https://invent.kde.org/frameworks/ktexteditor/commit/02a76278f9596e35cb99b42065a4a529b51a1992
(In reply to Waqar Ahmed from comment #11) > Do you have a default indenter set maybe? Check Configure Kate -> Editing -> > Indentation > > What are your indenter settings? On a quick try I can't reproduce the > slowness with Indent On Paste + Cstyle indenter. My default indenter was XML (I'm not sure if it was the default or if I changed it, and I don't really understand the differences between most of the options). With a default C style indenter the paste freeze is much shorter - something like 4 seconds instead of more than 10 seconds - but still noticeably large for a ~200 lines block, even if lines are very long. It also adds less of the unexpected indentation - but still does. When I compare this to grabbing an equivalently large (in character count) block with much more reasonably sized lines - which is 6000 lines long - pasting (with indentation on paste enabled) takes a much longer time, with the C style indenter - about 20 seconds (and also there are no unexpected indentation). My conclusion is that the "indent on paste" is just slow - very likely because it is very expensive ¯\_(ツ)_/¯ I also compared pasting into my main IDE - Eclipse - which does a lot of formatting on paste (not just indentation) and it handles all the cumbersome paste tasks relatively quickly: I couldn't get it to delay more than 2 seconds, regardless of content size and shape. So maybe it isn't inevitable that formatting during paste is slow, but I'm done barking up that tree ;-)
(In reply to Waqar Ahmed from comment #12) > Git commit 02a76278f9596e35cb99b42065a4a529b51a1992 by Waqar Ahmed. ... > Optimize rendering spaces with dyn wrapping I checked the latest master, and cursor movement behavior when dynamic word wrapping is enabled is *much* improved. But not so much when it is off (I prefer working with text files without dynamic wrapping, personally).
Git commit d9593e16f03461965df3e925a6a965aaf7b61aa0 by Waqar Ahmed. Committed on 02/03/2023 at 18:26. Pushed by waqar into branch 'kf5'. Optimize rendering spaces with dyn wrapping When dynamic wrapping is enabled and a QTextLine has trailing spaces, then those spaces aren't really shown unless QTextOption::ShowTabsAndSpaces is enabled. Thus we shouldn't waste time rendering spaces at the same position. With this commit, rendering of spaces is split into 3 steps: 1. Collect all spaces in the line by iterating over the text in reverse 2. Iterate over all the collected spaces in reverse because we want to look at the spaces in the beginning of line first and convert each space position into a QPointF. Whenever we encounter a point which was equal to the last collected point, we stop the loop and break. 3. Render all the collected space points at once. (cherry picked from commit 02a76278f9596e35cb99b42065a4a529b51a1992) M +30 -8 src/render/katerenderer.cpp M +1 -1 src/render/katerenderer.h https://invent.kde.org/frameworks/ktexteditor/commit/d9593e16f03461965df3e925a6a965aaf7b61aa0
A possibly relevant merge request was started @ https://invent.kde.org/frameworks/ktexteditor/-/merge_requests/493
Git commit 55ff7ac5a76773ac3b77398238688e2dbe34e574 by Waqar Ahmed. Committed on 03/03/2023 at 15:03. Pushed by cullmann into branch 'master'. Improve performance of rendering spaces with dyn wrap disabled We don't need to find spaces that are not in the visible view. M +8 -0 src/render/katerenderer.cpp https://invent.kde.org/frameworks/ktexteditor/commit/55ff7ac5a76773ac3b77398238688e2dbe34e574
Git commit 294543a2977c4181bb0842c2a45f573f8335fd03 by Christoph Cullmann, on behalf of Waqar Ahmed. Committed on 03/03/2023 at 19:42. Pushed by cullmann into branch 'master'. Improve cstyle performance With this patch I can notice 7x-8x improvement in performance. The main culprit was `tryMultiLineStringLiteral` which searched the whole document for a multiline literal beginning. I have now limited it to 25 lines max. Besides, the multiline string matching is useless for every other language besides C++ so return early if the mode is no C++ Related: bug 466531 M +1 -0 autotests/input/indent/cstyle/indentpaste5/expected M +1 -0 autotests/input/indent/cstyle/indentpaste5/origin M +1 -0 autotests/input/indent/cstyle/string2/expected M +1 -0 autotests/input/indent/cstyle/string2/origin M +9 -0 src/script/data/indentation/cstyle.js https://invent.kde.org/frameworks/ktexteditor/commit/294543a2977c4181bb0842c2a45f573f8335fd03
Should now be fast without dynamic wrapping too > with the C style indenter - about 20 seconds (and also there are no unexpected indentation This was a bug/perf issue in Cstyle indenter which I have fixed now. It's now 7-8 times faster than before according to my measurements with a C++ code of about 5K lines. It was taking 9+ seconds and has now dropped to 1.4 seconds. There is about a ~900 ms call penalty i.e., the call overhead of calling into the Javascript indentation functions. So the actual work is being done in ~500ms. Even though this is a big improvement its still very slow in my eyes but there is not much we can do to squeeze out a bigger gain. I will take a look at the xml one when I find some time. Besides that, I think the important problems in this bug report are now all addressed. Thanks for the report. If you find more issues, feel free to open new bugs.
Thanks a lot for working on these fixes!
Git commit 67e566875597f0c1b1c0e0fafc482efcf4512b88 by Waqar Ahmed. Committed on 04/03/2023 at 08:55. Pushed by waqar into branch 'kf5'. Improve cstyle performance With this patch I can notice 7x-8x improvement in performance. The main culprit was `tryMultiLineStringLiteral` which searched the whole document for a multiline literal beginning. I have now limited it to 25 lines max. Besides, the multiline string matching is useless for every other language besides C++ so return early if the mode is no C++ Related: bug 466531 M +1 -0 autotests/input/indent/cstyle/indentpaste5/expected M +1 -0 autotests/input/indent/cstyle/indentpaste5/origin M +1 -0 autotests/input/indent/cstyle/string2/expected M +1 -0 autotests/input/indent/cstyle/string2/origin M +9 -0 src/script/data/indentation/cstyle.js https://invent.kde.org/frameworks/ktexteditor/commit/67e566875597f0c1b1c0e0fafc482efcf4512b88
Git commit 48398746c6896492615bd32caaf5d343127b27a6 by Waqar Ahmed. Committed on 04/03/2023 at 08:55. Pushed by waqar into branch 'kf5'. Improve performance of rendering spaces with dyn wrap disabled We don't need to find spaces that are not in the visible view. M +8 -0 src/render/katerenderer.cpp https://invent.kde.org/frameworks/ktexteditor/commit/48398746c6896492615bd32caaf5d343127b27a6