Version: 2.2.1 (using KDE KDE 3.2.1) Installed from: RedHat RPMs OS: Linux Kate does not compare well to other editors (ex nedit) when handling large files. The editor is not really slower to load or display large files, but simple operations are slower. For example, scolling with scroll bar and selecting text. I did not do exhaustive tests. I just don't use kate when manipulating large files One way of testing this is to make a dump of some database (for example your bug database). mysqldump dbname > dump.sql Open the resulting SQL file in nedit and kate and compare the behavior. I can see the difference with a 4MB table dump. Both editors use comparable display options : line numbers, syntax highlighting, no line wrap. One thing I see could make kate slower is font rendering, but does it explain the whole issue?
no, the block wise buffering and swaping is causing the slower speed I guess, therefor normal typing + other stuff remains fast even with 20 MB files or so, that should perhaps be discussed in the katepart developer corner, if the whole block buffer architecture is worth the effort, only to be bit faster on very big files
kate in head has no O(1) behaviour for many buffer internal things, will handle large files much better
After having to edit a db dump which resulted a 24Mb file (275000 lines) I can confirm that Kate just can't really work with big files. The scroll (using the scroll knob) is plain impossible. I tried the same file on several editors...Here's the feel I got from those... In parenthesis is the memory needed as reported by top. Kate was 39MB. Emacs(34MB) can scroll downwards (and at EOF) very fast, has problems on upwards scrolling. So does vi(m) (32MB). Nedit(57MB) seems to handle all this neatly. Scroll knob completely usable. No penalties on upwards scrolling. Bottom line, you can't work with kate you can't work with huge files, with Emacs and vi you almost can, and you won't notice you work with huge files with Nedit. (OK, except on start-up :-) Now, since we all love kate, can someone come up with a fix? ;-)
Hi everybody, I would like to see the KDE text handling improved as well. Kate and KWrite (both) have a hard time loading big files as well. I suppose it might be the text editor widget and not the application itself which is slow. For example, the plain wrapped text file "ChangeLog" from SuSE 9.1 on the DVD root directory, 3.6MB, takes about 12 seconds to load, at 100% CPU, on an otherwise unloaded P4-1800 with 512MB RAM. Unpacking ARCHIVES.gz to ARCHIVES (113 MB) on the harddisk will freeze kate and kwrite for longer than two minutes, while "vim" (console) will load it in three seconds. "kvim" (KDE application, to make the comparison valid) will take about 10 seconds to load the 113MB file and you can freely scroll back and forward (though it will be a bit lagging behind). Thank you!
I too am seeing very slow behavior in kate when editing a 22MB text file. Xemacs, which is usually very slow, is much faster than Kate for both loading and editing the file. The machine is a 650MHz Sun box with 1.25GB of RAM. Searching in Xemacs is almost instantaneous whereas it takes many many seconds to find the next match in Kate. Other operations are also typically a lot faster in Xemacs for very large files. This is with KDE 3.2.3.
in kde 3.3.0/1 the loading is much faster, but it still has several regressions with large files, kde 3.3.2 will have some more fixes around for the most common problems, but as soon as autoindentation kicks in, large files are still slow on editing, this needs some tweaking
I've found that it's not just autoindentation that causes this problem, though I haven't been able to find out what does cause it. I've run into this on files as small as 10KB with very long lines, or as small as 20KB with shorter lines. kate/kwrite initially seem very quick and responsive, but after about 10-30 minutes of usage (depending on the size of the file), they become slow and almost unusable. This time includes a lot of switching between applications (including other kate/kwrite editors) and desktops. Admittedly, this isn't the most amazing machine in the world (Pentium III 600MHz with 256MB of RAM), but the sharp drop-off in performance leads me to think that my hardware is not the problem. If this isn't actually related to this bug report, I'd be happy to open a new one for it.
Watching this bugs activity view I recognize as a first action setting down the bugs severity from "normal" to "wishlist". Ooops, I switched from emacs to kate, but if that problem won't be solved within a few weeks to months I'm sad to say I will have to go back to good old emacs. Watching that low activity on that bug gives only slight hope.
If you would dare to try a up to date Kate, like the one shipped with KDE 3.4.0 or better 3.4.1 (bugfixes won't hurt), than you would know, that Kate now has zero problems with working with 50 MB files or more, if you have highlighting on for them, sure, to goto last line will take long, as yes, HL needs time, sorry ;) For plain text, Kate is more than 10000000000 times faster than Emacs, as you just can type there instant, while Emacs here takse ages to move it's gap if you insert at random positions. What still needs fixing: a) VERY long lines, like 10000 columns, but he, I still think that's corner cases, where vim fails too,a nd which shouldn't be considered text files b) faster HL, which will be tricky, as Kate does more extensive stuff than Emacs/Vim does there
Hi Christoph, > If you would dare to try a up to date Kate, like the one shipped with KDE > 3.4.0 or better 3.4.1 (bugfixes won't hurt), than you would know, that Kate > now has zero problems with working with 50 MB files or more 1. CPU: AMD Duron 1300 2. mem: 1 GB 3. kernel: 2.6.11.11 4. kate: 2.4.1 (KDE 3.4.1) Let me tell you what happened about 5 minutes ago: I tried to load a 50 MB text file from a remote host. Network transfer happened within about 10 secs, then the hard disk started to groan... waiting... waiting...system doesn't react to any keystroke or mouse move... waiting... 3min later kate disappeared, . The system now is available again, but kded has blown up from 100MB to 220MB and it won't free any byte till next reboot. O.k., maybe it's a "remote" problem... copying that file to my local hard disk, restarting kate, STRG-O... the game stays the same (except for kded's behaviour: it temporarily blows up to about 400 MB, but then go's down again to 220MB). This is exactly what I reported a week ago (http://bugs.kde.org/show_bug.cgi?id=107607). And I don't agree in calling this "zero problems"... :/ Kind regards Frank
Cannot (really) confirm. P4 1.8GHz, 512MB RAM, Maxtor 80G harddisk, SuSE 9.3, KDE 3.4.1 RPMs from SuSE. 93MB text file created by "while true; do cat ~/.y2log >> templog; done" and eventually pressing Ctrl-C: jens@get1431p4:~> ls -la templog -rw-r--r-- 1 jens users 92593629 2005-07-25 13:22 templog On a system that's been active for 6 days (uptime) without reboot or re-login, and quite a number idle of konq / kontact / konsole / ... sessions running, I tried this: Run vmstat every second to show system "business" (note, a folding@home client ate all the spare CPU with nice=19, so my CPU is always at 99%): jens@get1431p4:~> kate templog & vmstat 1 [1] 19296 procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 2 1 513464 19376 4748 199828 2 2 22 26 36 25 99 1 0 0 3 0 513464 12144 4768 206132 0 0 6172 0 1447 732 98 2 0 0 2 0 513464 9776 4844 207148 0 0 1008 0 1243 1082 96 4 0 0 1 2 513464 6960 5448 208068 0 0 892 1044 1317 791 98 2 0 0 4 0 513464 5704 5488 208744 0 0 628 112 1147 1035 97 3 0 0 ASSERT: "m_currentContainer==container" in kate/app/kateviewmanager.cpp (196) 3 0 513464 5632 5500 207592 0 0 7552 0 1221 2967 91 9 0 0 3 0 513464 5760 5376 207620 0 0 27264 0 1378 1154 94 6 0 0 3 1 513464 4908 5332 208572 0 0 26632 0 1462 1270 95 5 0 0 2 1 513464 6288 4968 201912 0 0 29560 8 1353 1151 93 7 0 0 1 2 513464 6860 5208 193460 0 0 7548 11480 1315 1091 78 22 0 0 3 0 513464 6416 4888 193884 0 0 2176 1108 1229 798 93 7 0 0 3 0 513460 6324 4904 194572 108 0 8232 344 1287 1047 83 17 0 0 3 0 513460 6500 4808 194088 0 0 9520 660 1197 913 75 25 0 0 3 1 513460 6416 4740 194324 0 0 2304 68 1148 805 93 7 0 0 2 4 513460 4924 3820 196264 0 0 2052 4 1113 744 95 5 0 0 2 2 513460 5392 3804 196612 0 0 2264 248 1137 832 93 7 0 0 1 3 513460 5196 3804 196236 0 0 2608 5588 1185 809 92 8 0 0 3 1 513460 5536 3800 195852 0 0 4356 4424 1210 835 89 11 0 0 1 4 513460 5056 3916 195148 0 0 4868 28112 1240 822 86 14 0 0 1 3 513460 6804 3924 194420 0 0 904 36028 1469 792 91 9 0 0 1 2 513460 6316 4040 195592 0 0 2308 4964 1374 1025 93 7 0 0 2 1 513460 5024 4132 196860 0 0 5768 3404 1222 953 84 16 0 0 1 1 513460 5952 3836 196196 0 0 4996 532 1147 868 87 13 0 0 2 0 513456 7020 3820 195276 0 0 4484 376 1169 843 89 11 0 0 1 4 513456 5004 3776 196656 0 0 4104 332 1153 822 90 10 0 0 1 1 513452 6300 3784 196088 0 0 3460 248 1140 816 89 11 0 0 1 5 513452 5964 3972 194628 0 0 2220 38852 1297 788 91 9 0 0 2 0 513452 6720 4032 195452 0 0 1796 9496 1383 1030 93 7 0 0 1 2 513452 5376 4064 196812 0 0 4260 8 1140 853 89 11 0 0 1 2 513452 5904 4076 196252 0 0 5768 472 1186 940 83 17 0 0 1 3 513452 5392 4036 196760 0 0 4100 324 1154 816 89 11 0 0 1 4 513452 5748 3868 196636 0 0 11952 452 1286 936 93 7 0 0 1 3 513452 5148 3736 197468 0 0 14100 396 1279 1026 95 5 0 0 2 2 513452 5456 3840 196684 0 0 8332 22996 1232 880 96 4 0 0 1 2 513452 6388 3844 196264 0 0 7560 9852 1411 1096 96 4 0 0 2 1 513452 4924 3900 197776 0 0 9612 168 1363 1035 95 5 0 0 2 0 513452 5768 3896 197016 0 0 31756 0 1362 1154 95 5 0 0 3 0 513452 6544 4000 196896 0 0 9632 0 1341 1351 93 7 0 0 1 1 513452 5784 4308 197272 0 0 496 0 1191 927 97 3 0 0 1 0 513448 5660 4328 197332 24 0 88 0 1125 718 99 1 0 0 1 0 513448 6472 4640 196180 0 0 0 616 1173 655 100 0 0 0 Here (after about 30 seconds) kate finished loading and I pressed Ctrl-C. Then I scrolled around in the document using the mouse on the scrollbar (which was quite fast), and closed kate again. [1]+ Done kate templog jens@get1431p4:~> ps axu|grep kded jens 6665 0.0 3.0 210392 15644 ? S Jul19 2:16 kded [kdeinit] jens 19303 0.0 0.1 1900 752 pts/7 S+ 13:25 0:00 grep kded jens@get1431p4:~> ps axu|grep kded jens 6665 0.0 3.0 210392 15648 ? S Jul19 2:16 kded [kdeinit] jens@get1431p4:~> free total used free shared buffers cached Mem: 515988 485684 30304 0 4752 189016 -/+ buffers/cache: 291916 224072 Swap: 996020 513236 482784 This was after closing kate. Nothing too unusual about it (I don't know about the supposed memory hole in kded, but my system didn't *feel* more slowly afterwards). The only thing I would have to complain about would be Kate's *long* startup time (over 10 seconds in my example), which is also very annoying if you "only" want to quickly view some HTML source in Konqueror and it starts up Kate. Jens
> jens get1431p4:~> kate templog & vmstat 1 O.k., doing the same way: 2 1 494744 12912 32260 373128 0 0 0 0 1078 4281 58 41 0 1 1 1 494732 11016 27664 290656 0 0 0 208 1073 4289 57 43 0 0 0 3 494732 11336 10760 248892 0 0 12 92 1009 4213 43 30 0 27 0 3 494732 10952 7592 245712 0 0 20 20 1028 4273 16 13 0 71 0 3 494696 11292 4360 232972 0 128 0 128 1014 4219 20 17 0 63 1 0 495412 11008 4440 229664 0 1288 84 1524 1172 4295 15 26 0 59 0 5 496288 10952 4496 227024 1308 1448 1660 1952 1277 4534 17 36 0 48 0 1 499716 11224 4580 220156 132 4580 192 5924 1598 4424 21 32 0 47 4 2 501288 11208 4620 219732 180 3024 560 3340 1372 4555 24 18 0 59 0 1 501772 11372 4636 219576 304 1252 524 1352 1252 4467 14 12 0 74 0 3 502348 11132 4636 219328 124 1436 124 1496 1218 3132 11 7 0 82 0 6 502952 10888 4652 219484 168 1580 544 1664 1252 3904 15 12 0 74 0 5 506372 10888 4668 216752 88 6620 196 7412 1425 4042 14 17 0 69 procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 0 5 512056 11016 4688 216576 164 6364 552 6464 1207 3146 15 10 0 75 0 4 518084 10888 4608 213704 192 6540 472 6764 1247 4653 29 19 0 52 0 3 523196 11016 4612 212312 284 6472 308 6720 1261 4283 18 15 0 67 0 4 529240 11080 4616 209756 336 7172 384 7696 1289 4171 13 12 0 75 0 5 535884 10888 4612 207056 284 8836 676 10148 1476 4280 18 18 0 64 0 6 542200 11016 4632 206008 432 7220 656 8164 1278 4286 19 18 0 63 1 4 551828 11144 4636 203528 72 10016 208 11060 1437 4265 18 19 0 [...] 0 18 1052240 10988 3088 59364 564 536 1236 1632 1431 2032 7 14 0 79 0 19 1052240 11284 3080 57936 588 852 1320 1400 1343 685 10 20 0 70 0 10 1052248 10960 3096 58252 368 572 1020 2868 1673 557 6 6 0 88 0 10 1052248 10896 3088 58628 488 488 964 2568 1574 509 6 6 0 88 0 13 1052200 11664 3092 58476 636 664 1972 3016 1689 1027 6 13 0 81 0 13 1052248 10960 3116 58348 728 416 1332 1176 1472 1472 8 11 0 81 0 17 1052248 11556 3132 55140 360 884 2016 2200 1438 1950 8 29 0 63 0 10 1052248 10960 3140 50216 1332 336 2408 1532 1458 3269 11 14 0 75 0 10 1052244 11124 3124 48044 796 532 1996 784 1350 1052 8 20 0 72 0 12 1052248 11080 3148 48272 372 600 1456 724 1254 1813 7 15 0 78 8 8 1052248 11432 3112 47224 740 412 4268 732 1356 3395 9 26 0 65 0 11 1052144 10896 3112 47384 780 616 1888 912 1287 649 7 15 0 78 0 9 1052248 11404 3144 48096 40 1164 1468 1380 1247 506 5 8 0 87 0 9 1052244 11600 3156 47720 288 256 1536 332 1238 615 5 10 0 85 0 17 1052248 11216 3128 46964 772 372 3948 948 1354 944 8 29 0 63 [ At this point the colonel.... ehm, the kernel told kate to leave the building] Kind regards Frank
OK, let's compare configurations P4 1.8GHz, 512MB RAM, 80G harddisk, SuSE 9.3 with KDE 3.4.1 and both create a virgin user account with empty or nonexistant .kde directory and no default configuration settings. Please bzip2 me the file you are loading into Kate, so I can try with your exact file. If it's private data, create a new one, try the new file yourself, then send it to me if the error persists. If not, we have to find out why *this* file chokes Kate. I'll try your file, in the mean time: Create a series of files with 10, 20, 30, 40, 50 MB (at first) and try to load these. Close KDE and (as root) kill all left-over KDE processes after logout, after every file. Run the above command, pressing Ctrl-C as soon as Kate has finished loading the file. Count the lines vmstat printed (= seconds), or alternatively, execute kate myfile & vmstat 1 | nl which should number the lines (you need a terminal window wider than 80 chars for that though). I'll do the same. Let's see where we get and where the bottleneck is. Jens
Thank you, Jens, but I think I got it! Those files I tried to load are generated under Unix, but have MS-DOS like line endings (\r\n). So kate seems to have some problems with that, maybe the data is like one single line to kate. As an old emcasian I didn't waste any thought about that. Anyway, is there something in the settings where one can tell kate not to worry about different types of line endings? Regards Frank
any chance the folks with the problem have their /tmp on a ramdisk/tmpfs?
Sidenote: no, Kate won't handle MSDOS or Mac files as one line ;) This should be not taken into considerations.
You're right, Christian. But here is the next track. Up to now I always loaded the big boyz using STRG-O and the file open dialogue. Loading them by running "kate bigboy.txt" from the command line everything works fine. It's loading big files with STRG-O that makes kate crashing! Kind regards Frank
Sorry, bullshit: watch my own comment #12, there running kate from the command line with a big file didn't work. 2 hours ago it worked, 5 minutes ago it didn't. The best thing I could do: going on holidays...
I can only say: KDE 3.5 will handle big files better, good enough for my needs at least, if there are probs, that might be bugs, yes, but than I need backtraces and more ;)
Hm, it is nice to see you guys talking about _megabytes_ of text making kate slow. I made an old machine up and running. Speed is .3 GHz, KDE is 3.5.5 (Debian Testing/Etch). It becomes slow right at few _kilobytes_. It's pretty familiar fast when starting a new text from scratch, but as it is growing longer, kate becomes slower and slower. I didn't yet figure out a way to in fact measure it, but it feels somewhat lagging. The interesting part in using a 300 MHz machine is, that character rendering is so slow, one can see how it gets applied: First the character appears somewhat bold, about half a second later it is clean antialised. But: This happens to be for every single character. I am not sure if the rendering doesn't make use of a rendered characters cache. But if it doesn't, I'd suggest to make use of such a one. At least for old machines. Currently, I am working on a text file about 100 kB in size (a Mediawiki markup source; I'd look up the exact size if you're interested in). I noticed, that time between keypress and actual appearing of a character increases with growing file size. Although I am not nagging because of the slowness of dealing with *really* large files, such as 24 MBs (yes, I could go for a new machine), I wonder why the rendering time increases depending on file size. And no, any highlighting is set to off; only auto-completion is set to on. Oh, and wrapping is set to on, too, and almost all lines are wrapped indeed. (This might relate to Comment #6, above.)
Am Montag, 29. Januar 2007 schrieb Wolfram R.Sieber: > character rendering is so slow, one can see how it gets applied: First the > character appears somewhat bold, about half a second later it is clean > antialised. A possible workaround could be setting Kate's font to a bitmap font.