Bug 378579 - kate (and kwrite) on large files: version 3.7 is fine, version 15 sucks
Summary: kate (and kwrite) on large files: version 3.7 is fine, version 15 sucks
Status: RESOLVED INTENTIONAL
Alias: None
Product: kate
Classification: Applications
Component: general (show other bugs)
Version: 15.12
Platform: Debian stable Linux
: NOR normal
Target Milestone: ---
Assignee: KWrite Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-04-09 08:05 UTC by Rick Graves
Modified: 2018-08-17 10:29 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
It shows the cursor one line down from the top. (255.91 KB, image/png)
2017-04-09 08:05 UTC, Rick Graves
Details
Thinkpad T500 with kubuntu drives for 14.04 & 16.04 (1.51 MB, image/jpeg)
2017-04-09 12:01 UTC, Rick Graves
Details
testing kubuntu 16.04 (kate version 5/15) on a kubuntu 14.04 (kate version 3) desktop (1.38 MB, image/jpeg)
2017-04-10 09:55 UTC, Rick Graves
Details
4 core i5 3230M 2.60GHz laptop (1.51 MB, image/jpeg)
2017-04-12 11:47 UTC, Rick Graves
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Rick Graves 2017-04-09 08:05:23 UTC
Created attachment 104928 [details]
It shows the cursor one line down from the top.

I am on kubuntu, so I get the kate (and kwrite) version the ubuntu people give me.

I have been using kwrite to maintain a big file (25 mb). (Note that kwrite uses the kate editor.)

On kubuntu 14.04 (kate version 3), kwrite worked great.

When I first tried kubuntu 16.04 (kate version 5/15) in early 2016, kwrite was clearly not the same:

1. For the arrow keys, keyboard repeat works at the top of the file, but not further down. In kate version 5/15, the display does not update -- you cannot see where the cursor is. If you hold an arrow key down for a while and release, after some seconds, the cursor freezes then reappears somewhere. In kate version 3, anywhere in the file, you see exactly where the cursor is at every instant, you can release the arrow key when you want, and the cursor is where you expect it to be.

2. Pasting text into the file (anywhere except the top), kate version 5/15 takes several seconds for the text to appear. Using kate version 3, the text appeared instantly.

After evaluating kubuntu 16.04 in early 2016, the kwrite problem was a show stopper -- I declined to upgrade kubuntu, and I have been on 14.04 until recently. But then in March, a more pressing consideration made me upgrade to 16.04, and kwrite is still a problem on my big file.

In 2016, I tried tweaking settings in hope of fixing the problem, but I did not find a solution.

Note that I encountered this problem using the same hardware.  Right now, I have two hard drives for my Lenovo Thinkpad T500 laptop with 8 GB memory, one hard drive with kubuntu 14.04 (kate version 3), and another hard drive with kubuntu 16.04 (kate version 5/15).  I can shut the laptop down and swap hard drives. You can say my laptop is too slow, but kate version 3 works perfectly OK on my laptop -- no complaints.  But on the same laptop, kate version 5/15 sucks.

I have encountered deniers, so I assembled a big file -- anyone can try it themselves.  I googled "Richard Stallman", went to some hit pages, and copied some text by or about him. I pasted into a file and duplicated. I used the fmt utility to break the long lines into shorter lines (the file I edit in real life has mostly short lines). The file is almost 27 mb (too big to attach). I gzipped it, and uploaded it here:

http://www.advanced-app.com.hk/MiscJunk/Stallman_Richard_short.txt.gz

For both the following tests, I used wget to download the file and gzip -d to extract it.

For the latter of the two tests below, see the attached picture. It shows the cursor one line down from the top.  The test, conducted near the bottom of the file, is to determine, after pressing down arrow once, how long it takes for the cursor to appear one line down from the top (as explained in detail below).

On the Lenovo Thinkpad T500 laptop with 8 GB memory running kubuntu 16.04 (kate version 5/15):
I went to the bottom of the file, and from the bottom I selected up to and including the line "The consequences:". I copied the selected text. I used PgUp several times until I came to the heading "The GNU Manifesto". Above that heading, I hit Paste (Ctrl-V), and counted "one thousand one, one thousand two, one thousand three, ..." The inserted text appeared between one thousand five and one thousand six. I hit Ctrl-Z. I pressed the up arrow key, and kept pressing it while I counted ""one thousand one, one thousand two, one thousand three, one thousand four, one thousand five" then I released the up arrow key, immediately pressed down arrow once (quickly) and started counting again. The cursor did not appear one line down from the top until AFTER I counted up to one thousand eight.

On the SAME Lenovo Thinkpad T500 laptop with 8 GB memory, running kubuntu 14.04 (kate version 3):
I went to the bottom of the file, and from the bottom I selected up to and including the line "The consequences:". I copied the selected text. I used PgUp several times until I came to the heading "The GNU Manifesto". Above that heading, I hit Paste (Ctrl-V). There was no need to count, as the inserted text appeared instantaneously (to my human perception). I hit Ctrl-Z. I pressed the up arrow key, and kept pressing it while I counted ""one thousand one, one thousand two, one thousand three, one thousand four, one thousand five" then I released the up arrow key, immediately pressed down arrow once (quickly). There was no need to count again, as cursor had come to a stop instantaneously after releasing the Up arrow, so the cursor moved down to the second line immediately after pressing the Dn arrow. While I was pressing the up arrow key, the cursor never disappeared, but moved up smoothly and continuously.

Same results using kate and kwrite.  Exactly the same resuls on different computers -- on big files, kate version 3 is fine, kate version 5/15 sucks.

My hope is the problem can be fixed, that kate version 5/15 can be a good on big files as kate version 3.
Comment 1 Dominik Haumann 2017-04-09 10:37:57 UTC
I have not yet tested this here locally, but I once heard that the graphics driver may cause very slow behavior. It is unlikely you hit this, but still this may be one reason. Besides that, Kate 3 uses Qt4, and Kate 5 uses Qt 5 - so also the underlying Qt stack may cause this. This needs to be profiled.
Comment 2 Dominik Haumann 2017-04-09 10:47:01 UTC
Just tested this: For me, following your instructions, Kate reacts snappy without any delays, so I cannot reproduce. And if we cannot reproduce we cannot do anything about it.

Do you have any highlighting activated?

What graphics card do you have? Could you google for this driver and kate, whether you find other users that also have this issue?

How fast is your CPU?
Comment 3 Rick Graves 2017-04-09 11:53:11 UTC
Bug 378330 seems to be related, but that one is only about the paste delay. I use kate for small files (python scripts) and have no problem pasting into small files.

If my only problem was the paste delay, I would live with it and I would never have complained. For me, the much bigger aggravation is the arrow keys do not work with keyboard repeat in 16.04 and 16.10, kate version 5/15 (kate says version 15, kwrite says kate version 5).
Comment 4 Rick Graves 2017-04-09 12:01:37 UTC
Created attachment 104929 [details]
Thinkpad T500 with kubuntu drives for 14.04 & 16.04

One laptop, two hard drives, 1) kubuntu 14.04 & 2) kubuntu 16.04.
Comment 5 Rick Graves 2017-04-09 12:04:53 UTC
The Thinkpad T500 uses the ATI Mobility Radeon HD 3650  display adapter (http://www.thinkwiki.org/wiki/Category:T500).  I have the same problem with kate and kwrite on my home desktop, which has a dual core Athlon II CPU and 16 GB of memory, on which kinfocenter says the display adapter is NVIDIA GT218 [GeForce 210].
Comment 6 Rick Graves 2017-04-09 12:21:41 UTC
How does one activate highlighting? 

I am mostly using the default configuration.  For kate, I set the line length limit to zero and I use spaces not tabs.
Comment 7 Rick Graves 2017-04-10 09:55:45 UTC
Created attachment 104933 [details]
testing kubuntu 16.04 (kate version 5/15) on a kubuntu 14.04 (kate version 3) desktop

On my alternate office desktop, I unplugged the hard drive on which kubuntu 16.04 (kate version 3) is installed and installed kubuntu 16.04 (kate version 5/15) on a spare hard drive.
Comment 8 Rick Graves 2017-04-10 10:02:04 UTC
I tried kubuntu 16.04 (kate version 5/15) on my alternate office desktop (see the picture I uploaded just now).  Following the test procedure described, after hitting paste (Ctrl-V), the text took 5 seconds to appear, and after hitting down arrow once, it took 6 seconds for the cursor to appear one line down from the top.

The hardware is an Athlon II dual core CPU, 8 GB of memory, nVidia GeForce 8400GS display card, monitor with 1920 x 1080 pixels resolution.

Yes on this hardware kate version 3 (kubuntu 14.04) performs well, the pasted text appears instantly, the cursor appears one line down instantly.

I believe my hardware all greatly exceeds the minimum recommended for kubuntu.

Two questions:

1) What is the minimum recommended hardware to run a QT5 application?  (I looked and did not find on google.)

2) What hardware does the maintainer use to test kate and kwrite?

Thanks.
Comment 9 Maciej Mrozowski 2017-04-10 17:02:11 UTC
I checked your procedure and input text file, and cannot reproduce the problem with kate-16.12.3, on i5-4300M 2.6GHz laptop. Pasting is instantaneous, when scrolling the text using arrow keys, cursor stops blinking and visibly travels line by line. Could you perhaps try some live dvd KDE distro with latest plasma apps? For instance this one http://cdimage.ubuntu.com/kubuntu/releases/17.04/beta-2/
Comment 10 Rick Graves 2017-04-12 11:47:24 UTC
Created attachment 104985 [details]
4 core i5 3230M 2.60GHz laptop

laptop on which kate/kwrite performance is good on big files.
Comment 11 Rick Graves 2017-04-12 11:54:11 UTC
Good news.  See the picture I uploaded -- a 4 core i5 3230M 2.60GHz laptop (Lenovo Thinkpad Edge E431, came with Windows 8).  Testing as described using kubuntu 16.04 (kate/kwrite version 5/15), performance is good.

For use on big files, it seems the minimum hardware requirements for the kate editor exceed the minimum hardware requirements for kubuntu. 

Is the bottom line that for big files, kate needs 4 cores, 2 cores are not enough?

In an ideal world, this would be up front, and people like me would not have to discover it by trial and error.
Comment 12 Nate Graham 2017-09-05 02:15:15 UTC
This is clearly a performance bug, not some kind of intentional hardware requirement. Thanks for finding it, Rick!
Comment 13 Rick Graves 2017-09-07 01:26:59 UTC
Some thoughts:

1) Package maintainers are volunteers and guys like me appreciate their efforts.

2) kubuntu is not kde, but kde needs distributions.

3) Ideally, kde could encourage package maintainers to test using systems that approximate the minimum hardware recommended by some popular distributions.  If package maintainers only test using much more powerful hardware, they can remain oblivious to, and in denial of, the problems encountered by some rank and file users.
Comment 14 Christoph Cullmann 2018-08-17 10:29:21 UTC
I can agree that current Kate versions use more resources for large files.
On any "current" machine that you bought in the last 1-2 years, that is normally no issue for anything below 100MB, if it has no expensive hl or extremely long lines.
We will not have the resources to change the architecture to something that will do it better.
Therefore this works for me as designed, I am sorry it brought inconveniences for you.