Bug 64569

Ok I tried this and am running CVS head.  I don't see any type of freeze or crash 
here.  However, on line 279, that freagin line is 207,326 characters long!!!  Are you 
sure kwrite is frozen or is it just working? 
 
Could you try again and give it a bit of time to finish as highlighting 200k of html 
code could be somewhat slow as well as katepart having to page in the data with our 
buffer system. 
 
Interesting performance test for katepart anyhow ... 

but on my 400Mhz PC it took a few minutes to make it responding... 
 
These kind of html source isn't rare : I often see it on comercial sites 
(maybe to make it harder to read the source) 
 
So it's rather important... 
 
Jos

Subject: Re:  viewing source of an html page freezes kwrite

>
> ------- Additional Comments From yurkjes@iit.edu  2003-09-19 22:26 -------
> Ok I tried this and am running CVS head.  I don't see any type of freeze or
> crash here.  However, on line 279, that freagin line is 207,326 characters
> long!!!  Are you sure kwrite is frozen or is it just working?
>
> Could you try again and give it a bit of time to finish as highlighting
> 200k of html code could be somewhat slow as well as katepart having to page
> in the data with our buffer system.
>
> Interesting performance test for katepart anyhow ...
amazing stuff, even the line length calc is awfull with such a trashy line, 
not considering that this breaks totally the sense of normal text files to be 
human readable, don't see any way to make that faster/better, such pages are 
just borked for texteditors. (some automatically breaking of the long lines 
into normal ones could help, but wouldn't be usefull for normal cases)

So it's not a complete freeze correct? I want to be clear about that.  
  
It is rather important because even on my P4 3.06, everytime the cursor flashes at 
the end of that line, CPU hits 8% (full debug katepart though).  That's not nice. 
 
I have a textWidth speed up for katerenderer but it doesn't help too much (config-
>tabWidth is expensive :)  Perhaps I'll investigate further 

Subject: Re:  viewing source of an html page freezes kwrite

>
> ------- Additional Comments From yurkjes@iit.edu  2003-09-20 00:14 -------
> So it's not a complete freeze correct? I want to be clear about that.
>
> It is rather important because even on my P4 3.06, everytime the cursor
> flashes at the end of that line, CPU hits 8% (full debug katepart though). 
> That's not nice.
>
> I have a textWidth speed up for katerenderer but it doesn't help too much
> (config-
>
> >tabWidth is expensive :)  Perhaps I'll investigate further
perhaps we should precache the tabWidth in an int in the renderer which we 
update on updateConfig() :)
but, with that lenght, even the sole qt width methodes should kill any speed

Yeah, that's what I'm currently attempting to do.  Although I can't seem to figure out 
exactly when updateConfig gets called.  Seems that the tabWidth could actually be 
changed and the renderer might not get notified if it's just in its updateConfig.  So 
for right now I've just  kept the calculation local but moved it outside the textWidth 
loops. 
 
The _only_ other source of speedup on the renderers side may be to get dirty with 
the run-length of our attributes so we avoid finding the width of a character every 
single time (i.e. we should only get it once for the entire length of the run). 
 
Have to re-run cachegrind here soon because I know there were some other speed 
hits in kate though.  Right off hand I remember something about maxScrollPos and 
some places where it gets called in a very vicious cycle quite often by several other 
methods. 

From some of the stress testing I did on kate post-3.1, I found that the 
highlighter slows to a crawl with big lines like this because the whole line was 
being copied in memory once per regexp in the current context per char - 
which means that the user will see a freeze of several minutes before kate 
starts responding again.  I fixed this ages ago in cvs head, so using cvs you 
won't see the problem that the reporter is describing. 
 
Jos

I see. Hmmm, well it somewhat makes me feel better that trying the same file with 
vim (wrap is on by default) is useless (I killed it after about 30seconds on my 
machine).  Hmm alright I did a :set nowrap and vim still sucks ;) or I just don't know 
how to use it lol. Kate's dynamic wordwrap is much more useable even in this case 
though it's somewhat slow to navigate on those wrapped lines. 
 
I'll probably commit the tabWidth refactoring sometime in the future though as that 
completely eliminates tabWidth from ever showing its face in cachegrind and instead 
other functions pop up to take its place ;) And it makes sense too.  Otherwise I 
won't touch the renderer as the performance problem isn't really there even in this 
case and I'm not sure it could be eliminated anyhow. The whole run-length stuff I 
mentioned was just an idea while we were on the topic :) Sorry to scare you. 

Ah, now that I think about it I am aware of the dynamic wrap speed problems 
that this is actually referring to (the highlighting stuff can still have an impact but 
I doubt this is the case here).  By all means, hack away, but what really needs to 
happen is we need to intelligently cache LineRanges for very long wrapped 
lines.  What I was working on before was a cache that only remembered a few 
values for long lines eg. the character length of a long line at 10 wraps, 20, 30 
etc or any other interval, that makes it easy to calculate to the required ranges 
(the speed problem comes in working out which column the nth range should 
start from, ie. we have to calculate from the start each time).  So you'd be 
welcome (and it will remain useful post-bidi) to do that, if you wish.  Or anything 
else you'd like to try of course :) The only difficulty I will have is when I try to 
re-sync the bidi branch, but that was always going to be the case ;) 

perhaps there could be some workaround in the buffer to take care that no 
xxxxxxxxxxxxxx long lines should be loaded but splitted on loading to a normal size 
after asking the user per messagebox to do so, or ? that would fix any prob with such 
monster dump files 

better name, not html specific at all, and no freeze, only REALLLLLLL long waiting ;)

*** Bug 78666 has been marked as a duplicate of this bug. ***

It is not only for very long lines. Kate makes the CPU work 100% when "left"ing (moving right) in a line as long as 147. The problem occurs only after around the 10th column (or character). I would not call 147 a huge number. So, something really sucks with Kate.
May be related: http://bugs.kde.org/show_bug.cgi?id=93928

It is very very irritating.

The *original* problem is still valid: editing or navigating the 300.000-some 
characters line in a HTML (or probably other highlighted) file is not 
practically doable, dynamically wrapped or not.

No crashes, just slowness.

Even passing through the line with the cursor stops kate for quite a while. 
Moving a char right or left in that line takes seconds on my system, typing 
one character takes minutes. Turning off the highlight makes typing and 
navigating significantly faster.

I suspect calclating the line width is the problem.

As said, for documents with lines of 1000000 we won't work, for all other stuff, like just xxxx long lines, we have that stuff fixed in KDE 3.5, as the hl there is a lot faster