Bug 85178 - KDE Applications are UNABLE to handle LARGE FILE Efficiently
Summary: KDE Applications are UNABLE to handle LARGE FILE Efficiently
Status: RESOLVED NOT A BUG
Alias: None
Product: kde
Classification: I don't know
Component: general (show other bugs)
Version: unspecified
Platform: Slackware Linux
: NOR normal
Target Milestone: ---
Assignee: Unassigned bugs mailing-list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-07-14 15:01 UTC by Mohd Asif Ali Rizwaan
Modified: 2007-12-18 11:55 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mohd Asif Ali Rizwaan 2004-07-14 15:01:08 UTC
Version:            (using KDE KDE 3.2.3)
Installed from:    Slackware Packages
Compiler:          3.2 
OS:                Linux

I have noticed that KDE and GNOME and other (Mozilla) are unable to handle large files effeciently without losing responsiveness.

Like KWord can't handle large Documents whose pages are in 100's. Konqueror can't load a >3MB file quickly. KWrite, Kate, Kedit are also getting unresponsive for atleast 10-15 seconds to load a mere greater than 3 MB file.

Surprisingly, I have found that Elvis and Emacs and Links (not lynks) browser are very efficient in handling large files.

Please Observe:
Elvis, Emacs, and Links browser takes 2-3 seconds to load a >3mb file. The processor utilization goes to 100% for just 1-2 seconds here.

whereas any KDE application, Mozilla 1.7, firefox 0.9, Netscape 6.1, gedit, galeon etc. takes more than *15* seconds and still remain unresponsive. upto 15 seconds the processor utilization goes to 100% (used ksim). And As long as the BIG file is loaded, the application which is loaded with the Big file remain somewhat sluggish, like scrolling becomes slow or takes 1-2 seconds.


Expected behavior of KDE Applications:
1. Like 'elvis' text editor KDE application MUST use temporary files for loading any file, small or big. Here is elvis about:

-------------------------------------------------------
"1. WHAT IS ELVIS-2.2_0
Elvis is a clone of vi/ex, the standard UNIX editor. Elvis supports nearly all of the vi/ex commands, in both visual mode and ex mode. Elvis adds support for multiple files, multiple windows, a variety of display modes, on-line help, and other miscellaneous extensions.

Like vi/ex, Elvis stores most of the text in a temporary file, instead of RAM. This allows it to edit files that are too large to fit in a single process' data space.  Also, the edit buffer can survive a power failure or crash. "
-------------------------------------------------------
please note:
"Elvis stores most of the text in a temporary file, instead of RAM. This allows it to edit files that are too large to fit in a single process' data space."

This is what I feel KDE lacks... KDE Applications are sluggish with medium to large (1mb-10mb) files due to loading the file into ram only (as I assume, I don't have proof, because I am not a developer :))

Please Follow 'Elvis' way of handling files, which is quite efficient. Or Emacs also loads huge files as quickly as in one to two seconds.

Amazingly a well formatted 5 MB English-To-Hindi HTML files gets loaded in "LINKS" (not "lynx") browser in just 2-3 seconds whereas all other browsers take around 15-30 seconds or just get hung.

I humbly request you superb developers to kindly look into the 'Efficient loading of files' by elvis, emacs, and 'links' browser. and 'Inefficient loading of files by KDE applications.'

MY Specs:
Pentium celeron 1.7GHZ
RAM 196 (atleast 100 mb remain free most of the run-time)
Swap 128
reiserfs 3.6
KDE 3.2.3
Slackware 10
Comment 1 jos poortvliet 2004-07-15 00:03:03 UTC
a speed up whouldnt hurt, maybe this is an idea?
Comment 2 Waldo Bastian 2004-07-15 10:59:47 UTC
While we are at it we should not forget to also solve world PEACE and HUNGER.
Comment 3 jos poortvliet 2004-07-15 11:20:51 UTC
yeah, thats right. 

and maybe ark could unpack files a bit faster (eg it shoulnt tage 5 mins on a 1 ghz to extract a 4 meg tar.gz, shoulnt it? used drag'n'drop from konqueror, its using a kioslave for this, is that so slow?)

More speed, again, is good - but I dont think ppl want to use their time for this, it might cost alot time and it wont be fun I guess. maybe just close the bugreport, start learning to code, and make it yourself...
Comment 4 Mohd Asif Ali Rizwaan 2004-07-17 05:23:34 UTC
here is the link to the English to Hindi Dictionary:

Please save it to your hard disk (using httrack) because konqueror is quite slow with Huge files :( or use Links or Emacs or Elvis editor:

http://www.aksharamala.com/hindi/e2h/search.php?from=1&t=1&p=.&l=1&rows=5000

After downloading the HTML file, please test it with Konqueror, Kwrite/Kate, Elvis and emacs. you can see a great difference between konqueror and links, elvis and kwrite.

Interestingly if you can find a >3 mb binary file in your filesystem, you can get it open with elvis in just 1-2 seconds :). thanks.
Comment 5 Mohd Asif Ali Rizwaan 2004-07-17 05:27:46 UTC
oops, the rows=5000 should be 50000

here is the update:
http://www.aksharamala.com/hindi/e2h/search.php?from=1&t=1&p=.&1=1&rows=50000

thanks.
Comment 6 Mohd Asif Ali Rizwaan 2004-07-17 05:32:50 UTC
sorry, typo again (i feel that bugs.kde.org should allow changing of comments, atleast for HTTP links)

http://www.aksharamala.com/hindi/e2h/search.php?from=1&t=1&p=.&l=1&rows=50000
Comment 7 Mohd Asif Ali Rizwaan 2004-07-17 05:51:11 UTC
Waldo Bastian wrote:
> While we are at it we should not folget to also solve world PEACE and HUNGER

-----------------------------
"The world is only a name; the indiviual is the reality. You can go on trying to find the world all over the world, and you will not find it, you will always find the individual. Words like the 'world', the 'society', the 'religion', the 'nation', are mere words with no content behind them -- empty containers."

Except you, there is no world.
------------------------------
Read the remaining at:
http://www.geocities.com/yahugrup/world2.htm
Comment 8 Mohd Asif Ali Rizwaan 2004-08-19 22:48:14 UTC
here is one more English to Hindi Dictionary Which I would love to use with Konqueror for my refrence.
http://www.indictrans.org/Dictionaries/English/Ban_dict_html/a.html
Comment 9 jos poortvliet 2004-08-19 23:45:39 UTC
the latest works nice (kde 3.3), the first link you gave (3.4 mb html file) is very slow indeed in konqueror. it slows down more and more - starts reading with 130 kb/sec, slows down to 15-20 kb/sec and the system becomes fully loaded and gets slow. takes alot of time... I havent finished loading it, because its too slow. kwrite seems to open immediately, but stops responding when I attemt to scroll down... full system use again.

hope this can be fixed, its quite annoying. although you dont stumble on very large files very often.
Comment 10 Jesse 2004-08-21 04:08:26 UTC
Hi, could those of you experimenting with kwrite/kate please put additional comments into bug 77503.  For some time Kate has used a buffer system which has only kept a certain number of blocks paged in at any given time.  This didn't quite work in the past but has been somewhat improved (hopefully).

There's an option in the Save/Load settings that lets you determine how many 'blocks' to keep in memory.  Could those messing around with large files play with these settings and monitor both load times and memory usage?  Your help is greatly appreciated and useful information will get you kind words from the devs :) A good cachegrinding during load would also be much appreciated if you have the time.
Comment 11 Aaron J. Seigo 2004-11-19 11:41:37 UTC
this bug report borders on the useless. file specific bug reports against specific apps since each app must take care of these things separately. one big "a bunch of KDE apps, and a bunch of non-KDE apps too, don't work well with big files" is not useful as a BR.