Version: (using KDE 4.3.4) Installed from: Debian testing/unstable Packages Kate and others katepart editors are very slow with large files, especially with files which have long strings, especially when highlighting is on. When I try to scroll such files in Kate or try to move cursor with keyboard my CPU usage become very high and moving is very slow. However scrolling same files with Kate 2.5.10 from KDE 3.5.10 is very fast. You can try to move cursor on line 23 in my attachment, it is very slow. My current Kate version is 3.3.4
Created attachment 40448 [details] file with long lines
KDE4.4 RC3 (4.3.98), Kate 3.3.98 - behavior is similar if syntax highlight is on. Scrolling is smooth and fast when highlight is off.
SVN commit 1090237 by mwolff: optimize: apply RegExp optimization I know from GeSHi: match the whole line and cache the result, that way we can save many regexp calls for lines, esp. for regexps that don't match at all. profiling showed that it got up to 10x faster for a big mysql dump CCBUG: 145686 CCBUG: 225228 M +16 -0 katehighlight.cpp M +28 -5 katehighlighthelpers.cpp M +11 -0 katehighlighthelpers.h WebSVN link: http://websvn.kde.org/?view=rev&revision=1090237
http://users.physik.fu-berlin.de/~milianw/callgrind.out.5823.gz as you can see with the above callgrind, more than 60% of the time is spent in QTextEngine::format(). Qt/Nokia has to improve it, or maybe it's even X11 / fonts under Linux that's slow.
*** This bug has been confirmed by popular vote. ***
*** Bug 169549 has been marked as a duplicate of this bug. ***
As I wrote in the other bug report: I reported the bug upstream and you guys have to wait for them to fix this issue. http://bugreports.qt.nokia.com/browse/QTBUG-8389
Found another bottleneck in Qt, esp. for those files that have long lines but are still "small", meaning it takes only a few seconds to open them. Afterwards, if you select the file, you'll hit this: http://bugreports.qt.nokia.com/browse/QTBUG-9549
*** Bug 238246 has been marked as a duplicate of this bug. ***
I noticed this bug first in Kile and then replicated it in Kate. Typing quickly with syntax highlighting on eats up +80% of my CPU (Intel 1.06Ghz Core Solo ULV)! That's quite horrendous for a text editor. I'm running Kubuntu 10.04 and KDE 4.4.4. I wish the issue were resolved DOWNSTREAM too.
(In reply to comment #10) > I noticed this bug first in Kile and then replicated it in Kate. Typing quickly > with syntax highlighting on eats up +80% of my CPU (Intel 1.06Ghz Core Solo > ULV)! That's quite horrendous for a text editor. I'm running Kubuntu 10.04 and > KDE 4.4.4. I wish the issue were resolved DOWNSTREAM too. Its simply not possible to fix this inside kate (unless they completely re-invent the wheel about laying/painting text).
(In reply to comment #11) > (In reply to comment #10) > > I noticed this bug first in Kile and then replicated it in Kate. Typing quickly > > with syntax highlighting on eats up +80% of my CPU (Intel 1.06Ghz Core Solo > > ULV)! That's quite horrendous for a text editor. I'm running Kubuntu 10.04 and > > KDE 4.4.4. I wish the issue were resolved DOWNSTREAM too. > > Its simply not possible to fix this inside kate (unless they completely > re-invent the wheel about laying/painting text). You can't be serious. So are future versions of Katepart (e.g. in KDE 4.5) are going to perform in precisely the same miserable way? Can someone recommend me different native KDE text and LaTeX editors?
I can type in the attached file with syntax highlighting just fine, even in the 40k chars line. But yes, long lines are unfixable. You can't render bidi-text and think that such long lines work, they don't work well with qt, they don't work well with gtk. You always need to layout the full line. We won'T fix that, text files are really not supposed to have more than 10k chars in one line, really....
(In reply to comment #13) > I can type in the attached file with syntax highlighting just fine, even in the > 40k chars line. > But yes, long lines are unfixable. You can't render bidi-text and think that > such long lines work, they don't work well with qt, they don't work well with > gtk. You always need to layout the full line. > We won'T fix that, text files are really not supposed to have more than 10k > chars in one line, really.... Long lines work fine in other environments like Windows. In any case, if long lines are not recommended then I don't see the point of even including the option of dynamic wrap. The whole point of that option is to be able to have long lines which wrap to conform to the size of your window. If you work on screens of differing resolutions then this feature is a must-have, as it allows one to work in different sized windows with text always extending to the edges (without any fiddling). Static wrap is not ideal in such circumstances.
I use dynamic word wrap the whole time. But really, which text document has > 10k lines even with dyn. word wrap? For latex, you will at least do line breaks after paragraphs. If that doesn't fit your needs, that is ok, you can choose an other editor. But for the average user, this is really no problem. Beside, as said, for the attached example, it still works just fine here with KDE 4.5 trunk Kate.
Unfortunately lines can be *much* smaller than 10K for Kate to choke. I can attest to that. I've set a static wrap to 100 characters and copy and pasted a document that chokes on dynamic wrap and it works fine on static wrap, and no line is more than a normal-sized paragraph. So obviously Kate has serious trouble working with even not-so-large lines. Moreover this problem doesn't arise in e.g. a Firefox text box, so I should expect that a more advanced text editor should be able to handle dynamic wrapping with ease. I really don't think this problem should be swept under the rug as some here are suggesting.
If the problem is important for you, then report an additional bug to qt that nokia improves their textlayouting/rendering. As kate is qt based, there is no way for us to fix that.
Bugs have already been filed to Qt regarding the issue and I've voted for them.
I had filed Bug 238246 which was marked as duplicate of this. Technically, this is the case, but the effect is more disturbing: Kate stalls and is completely unusable when you throw significantly horrible text at it. As "test case" I added http://download.dojotoolkit.org/release-1.4.2/dojo.js - just open that file and then try to perform normal operations such as scrolling, adjusting windows size, marking text etc... The scenario where a user wants to have a look at a website's source or at one of the linked .js files isn't too exotic. Even though a 'normal' user wouldn't do so, the number of techies among KDE users and particularly among Kate users is high - and many of them use Kate to look at different types of source files. I understand that minimized JavaScript code is nothing a sane person would want to read, but frequently you don't know a file's content until you open it. Another example where this happens: a paragraph of a latex document can get quite long in a thesis or when writing a book. Given that there is a significant chance to click on a file that "stalls" Kate when being loaded is a problem that deserves attention. Depending on the filetype and processing power of a computer this happens for lines significantly shorter than in the dojo example. I understand that bidirectional text + syntax highlighting impose higher complexity leading to this behaviour. However, it is still a bug and does not happen in all editors (supposedly b/c they don't do bidi). If we cannot fix the performance, we should consider other workarounds. What about detecting the length of the longest line when a file is loaded and automatically offering to deactivate bidirectional text (or syntax highlighting) if the line lengths surpass a defined (or even configurable) value that is known to lead to the mentioned problems?
again: this is not solvable inside kate to my knowledge. so please save our time and wait for a fix in Qt...
If it's not fixable, could you please consider to offer a workaround? Maybe it would be possible to optionally deactivate the bidirectional rendering of the Qt component if the user doesn't want/need it? I really love using Kate but this makes the editor useless at points (mine just crashed because of this). :( I just had a few different files open, opened one with, apparantly, too long a line and the whole program crashed/became completely unresponsive. I see your point that lines shouldn't be that long but it's not always in the Kate users control if they open a file that they themselves didn't write and don't know the line length of (and I don't want to be held liable for some people not writing readable code with enough line breaks).
*** Bug 248909 has been marked as a duplicate of this bug. ***
http://bugreports.qt.nokia.com/browse/QTBUG-9549 is closed! "Closing this task as the requested information has not been forthcoming after 60 days." I have posted one way how to reproduce this in Kate. Can anybody post more technical way to reproduce method to this bug?
*** Bug 258751 has been marked as a duplicate of this bug. ***
pasting from bug #258751: 1) do the equivalent of wc prior to open (or paste) into the file, if the numbers are crazy switch indenting off (bonus make crazy numbers configurable) please, could you at least workarond the problem _disabling_ HL for know broken files? those with very long lines? having to _kill_ kate everytime I forgot that it's not able to open a .sql or a .js is very annoying, and disabling HL entirely for those files it's not an option.
kate works now fast, but kwrite does have the same problem as kate bevore. I'am using KDE 4.6.0 and working with kwrite is damn slow.
*** Bug 267628 has been marked as a duplicate of this bug. ***
*** Bug 267353 has been marked as a duplicate of this bug. ***
*** Bug 270515 has been marked as a duplicate of this bug. ***
The problem is not, that kate crashes on files with long lines. The problem is the unexpected loss of data in other open files. If you cannot solve the problem itself, maybe try to make it better. Possible solutions: 1. A warning message, if a file is opened, that has a line, that is longer than e. g. 500 chars. (Like Photoshop does on various operations) 2. Automatic deactivation of highlightning, if it takes to long to render. (Like KDE does with desktop effects) 3. A single process for every text file. (Like Chrome does for tabs)
This is really annoying (KDE 4.6.2 here), and honestly I never met an editor before that fails so horribly when confronted with a text document containing long (as in, loooooong) lines. I also do not understand how this can be related to font rendering: Loading a 500KiB document is - of course - working fine normally. Just if those 500KiB do not contain a single newline, it's making issues. The amount of text to be rendered is exactly the same.
*** Bug 274019 has been marked as a duplicate of this bug. ***
It is pathetic that this bug persists for more than 3 years... after just some days using Kate, I stumbled upon this bug pasting one terminal output in an new file, and then finding an unresponsive kate eating all my CPU after that... no sintax highlithing, no use for bi-directional text output, nothing. Only few kbytes of text. Konsole had no problems splitting the line, but kate does... that don't sounds right. Notepad++ on Windows also handles this smoothly, and even the crappy stock notepad handles this way better than kate. Please, someone, fix this bug in some way.
It's a pity, but Kate is slow now, e.g. in comparence to QtCreator on the same computer and OS. QtCreator works well with attachment in comment #1 though it uses Kate syntax rules to highlight this file.
I run Qt: 4.7.2 KDE Development Platform: 4.7.00 (4.7.0) Kate: 3.7.0 and I also notice this bevaiour. Usualy, mysql dumps have the values of a table in one long file. You cannot work with those in kate.
I made some test among the editors. The test file was a 50 MB SQL dump, in which I wanted to search for the "INSERT INTO `xxxx`" term. The file itself had total of 8656 lines. Longest line found on line 2126 with 1,045,815 characters. I used vim, mcedit, Kate, Qt Creator and Notepad++ to show how the different editors behave. vim: Memory: 57 MB Time to open: 4 sec Time to search: 0.5 sec mcedit: Memory: 55 MB Time to open: immediately Time to search: 15 sec Qt Creator [core.PlainTextEditor] (just to show how Qt's own text editor work): Memory: 560 MB Time to open: 1 minute 35 seconds Time to search: infinity. After 4 minutes I killed it. Kate: Memory: 210 MB Time to open: 1 sec Time to search: infinity. After 4 minutes I killed it. Notepad++ on Wine: Memory: 115 MB Time to open: immediately Time to search: 8 seconds All the aforementioned editors had SQL syntax highlighting. The "INSERT INTO" line was one of the longest. I'm sorry, but I don't have any Gnome-ish thing on my computer so I cannot test them. Feel free to link to this comparison elsewhere.
I can only say that with the future kate from kde 4.8 and a little modified Qt 4.8, kate works with long lines almost as fast as with short lines. --- a/src/gui/text/qtextengine.cpp +++ b/src/gui/text/qtextengine.cpp @@ -2276,7 +2276,8 @@ QTextCharFormat QTextEngine::format(const QScriptItem *si) const } if (specialData && specialData->resolvedFormatIndices.isEmpty()) { int end = si->position + length(si); - for (int i = 0; i < specialData->addFormats.size(); ++i) { + for (int i = 0, size = specialData->addFormats.size(); i < size; ++i) { const QTextLayout::FormatRange &r = specialData->addFormats.at(i); if (r.start <= si->position && r.start + r.length >= end) { if (!specialData->addFormatIndices.isEmpty()) before, 20% spent on size(), after, only 0.5%.
I was wrong with my previous comment, kate was only very little faster, but nothing else. What made it be as fast as usual is the following workaround: In the menu settings, in Open/Save, limit the line length to 1024. If there are longer lines, they will be automatically wrapped.
*** Bug 283641 has been marked as a duplicate of this bug. ***
*** Bug 293130 has been marked as a duplicate of this bug. ***
*** Bug 293184 has been marked as a duplicate of this bug. ***
Note: The problem was fixed upstream by Olivier, see https://bugreports.qt-project.org/browse/QTBUG-8389 or https://codereview.qt-project.org/#change,31089 and https://codereview.qt-project.org/#change,31090 I'll try to test these patches in Kate now and hope we can then disable the annoying "too long lines detected, entering read-only + word-wrap mode" in Kate for the Qt versions that will ship this patch. Cheers
Too bad, opening the dojo.js file from above is still hitting a bottleneck in QTextEngine::format() - i.e. the issue is not yet resolved.
That's a shame. I was hoping this bug would've been squashed by Olivier's patch.
Btw the QTextEngine::format() issue was fixed by me for Qt5 since some time already. It was not accepted for Qt4 though. See https://codereview.qt-project.org/#change,34118 Here is the abandoned patchset for Qt4: https://codereview.qt-project.org/#change,34112 So, stay tuned for Qt5 :)
It seems that starting kile with the option --graphicssystem raster makes the problem go away.
Henning, "raster" graphicssystem in Qt is the default since several years, so I guess your software is very old. Please consider updating the distribution to maintained versions.