This is simlar to bug #145686

I remember that I once pushed a change upstream which added a few new features to the MySQL highlighting file. I think I used a few RegExpes which could probably lead to the decrease in performance.

I'll have a look at it eventually.

*** Bug 192495 has been marked as a duplicate of this bug. ***

@Matthias: Can you update to KDE 4.4 and then compile the development version from gitorious according to this howto:
http://gitorious.org/kate/pages/Building%20Kate

The file buffer has been rewritten completely and as Milian said there were some speedups in the highlighting. It should work again... Can you confirm?

This problem is confirmed in Ubuntu 10.04 when trying to open a 2.8 GB file. It crashes everytime I try to open the file.

lsb_release -rd
Description: Ubuntu 10.04.1 LTS
Release: 10.04

apt-cache policy kwrite
kwrite:
  Installed: 4:4.4.2-0ubuntu2
  Candidate: 4:4.4.2-0ubuntu2
  Version table:
 *** 4:4.4.2-0ubuntu2 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
        100 /var/lib/dpkg/status

Downstream bug may be found at:
https://bugs.launchpad.net/ubuntu/+source/kdebase/+bug/620789

(In reply to comment #5)
> This problem is confirmed in Ubuntu 10.04 when trying to open a 2.8 GB file. It
> crashes everytime I try to open the file.
> 
> lsb_release -rd
> Description: Ubuntu 10.04.1 LTS
> Release: 10.04
> 
> apt-cache policy kwrite
> kwrite:
>   Installed: 4:4.4.2-0ubuntu2
>   Candidate: 4:4.4.2-0ubuntu2
>   Version table:
>  *** 4:4.4.2-0ubuntu2 0
>         500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
>         100 /var/lib/dpkg/status
> 
> Downstream bug may be found at:
> https://bugs.launchpad.net/ubuntu/+source/kdebase/+bug/620789

This is a joke, right?  Do you even have enough RAM to perform this?

I don't see how somebody can have a 2.8 GB text file:

Assuming an average A4 page has roughly 3 kB of data [1], a file of 2.8 GB would then be about 1 Million of A4 pages...


[1] http://www.virtualmv.com/wiki/index.php?title=Data#How_many_bytes_on_an_A4_page.3F

(In reply to comment #6)
> (In reply to comment #5)
> > This problem is confirmed in Ubuntu 10.04 when trying to open a 2.8 GB file. It
> > crashes everytime I try to open the file.
> > 
> > lsb_release -rd
> > Description: Ubuntu 10.04.1 LTS
> > Release: 10.04
> > 
> > apt-cache policy kwrite
> > kwrite:
> >   Installed: 4:4.4.2-0ubuntu2
> >   Candidate: 4:4.4.2-0ubuntu2
> >   Version table:
> >  *** 4:4.4.2-0ubuntu2 0
> >         500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
> >         100 /var/lib/dpkg/status
> > 
> > Downstream bug may be found at:
> > https://bugs.launchpad.net/ubuntu/+source/kdebase/+bug/620789
> 
> This is a joke, right?  Do you even have enough RAM to perform this?

You don't need ten GB of ram to open that, with a text-editor built for being able to handle such large files. The solution is pretty easy: read the file into memory only in chunks.

> I don't see how somebody can have a 2.8 GB text file:

Database dump, Log-Files, text-file with losts binary data attached (I know lots of *nix-installers doing this). There are endless use-cases.

> Assuming an average A4 page has roughly 3 kB of data [1], a file of 2.8 GB
> would then be about 1 Million of A4 pages...

So? Nobody said he's going to print this.

I agree with Andreas. It is not rare to find big files nowadays :-)

Usually I use vim/gedit when I've to open big files due to the kate limitations :-(

If somebody implements a lazy loading, will be happy.
But I don't see me doing that atm.
It was once implemented (at least in parts, but still even there everything was first converted to unicode and swapped)

Perhaps relevant information
When opening the VMware Workstation install shell script in KWrite (you can also test, it is a free download with (also free) registration), KWrite works fine at first.
This file is several hundred MiB, with a small shell code portion at the beginning followed by raw data.

Scroll down, and once you get to the raw binary part, KWrite locks up, and quickly begins to swallow multiple GiB of memory.  It completely bogged down the system, and I had to use kill(8) to stop it.

The workstation installer contains binary data at the end, you can't assume that the qt text layouter will work ok with >> 1MB lines of binary blobs.
To have a better handling of large files is a nice wish, but unlikely to happen anytime soon. Given the average use is for programming, multi-GB files are not that typical.

What I would be most interested in learning is why KDE3 can deal with huge text files so much better than KDE4.  Surely something can be done?

I agree that multi-GiB files are not of much concern, but text-format databases and the like regularly reach into the dozens or hundreds of megabytes, and should be considered.

Even if it's a solution as clunky as a checkbox upon open, or auto-detect, for "large file mode".

Dear user,

this wish list item is now closed, as it wasn't touched in the last two years and no contributor stepped up to implement it.

The Kate/KTextEditor team is very small and we can just try to keep up with fixing bugs. Therefore wishs that show no activity for two years or more will be closed from now on to keep at least a bit overview about 'current' wishs of the users.

If you want your feature to be implemented, please step up to provide some patch for it. If you think it is really needed, you can reopen your request, but keep in mind, if no new good arguments are made and no people get attracted to help out to implement it, it will expire in two years again.

We have a nice website kate-editor.org that provides all the information needed to contribute, please make use of it. For highlighting improvements our user manual shows how to write syntax definition files.

Greetings
Christoph