Bug 189652

I can confirm this problem.  I used konsole w/ kde 3.x for many years and never had any issues with temp space.  Under 4.x though, I'm generating about 1GB per week of temp files (granted I run lots of shells, but is probably about 1000x more data than the actual text in the scrollback, what gives?).  

In my case, I have a home directory with a quota on it, and unless I manually restart konsole regularly, I run into the quota which causes konsole to crash.  I then have to remove the temp files manually before restarting konsole or it will continue to crash.

And, since the tmp files are all in some kind of binary format, I cannot even recover the scrollback of a crashed session, unless there is a way to do this that I'm not aware of.

On my Gentoo system, konsole temp files are linked from my ~.kde4/tmp-name to /tmp/kde-user.

I tested using unlimited scrollback and the clear scrollback menu option removed the history/tmp files.

I can confirm that using konsole with Unlimited scrollback in Fedora Core 12 causes the temporary files in /tmp/kde-BLAHBLAHBLAH created by konsoel to grow very large. konsole filled /tmp with very little effort. It caused 100% CPU usage and I couldn't figure out why until I noticed /tmp had filled. Disabling the Unlimited scrollback cured the problem of konsole running at 100%.

This is always reproducible.

kdebase-4.3.5-3.fc12.x86_64

If /tmp had not been a separate partition it probably would have caused a lot more problems than it did.

Here's a data point regarding the issue that illustrates the symptom very clearly.  Starting on a completely fresh Konsole session, here is my tmp directory. (the path being ".kde3.5" is just an artifact of upgrades, I'm running 4.3.4).

mculbert@iridium ~/.kde3.5/tmp-iridium  $ konsole -v
Qt: 4.5.3
KDE: 4.3.4 (KDE 4.3.4)
Konsole: 2.3.2
mculbert@iridium ~/.kde3.5/tmp-iridium  $ ls -l
total 24
drwx------ 2 mculbert users  16384 2010-03-22 10:49 .
drwx------ 8 mculbert users   8192 2010-01-08 11:23 ..
-rw------- 1 mculbert users      0 2010-03-22 10:49 konsolewZ1387.tmp
-rw------- 1 mculbert users      0 2010-03-22 10:49 konsoleUU1387.tmp
-rw------- 1 mculbert users      0 2010-03-22 10:49 konsoleAE1387.tmp
mculbert@iridium ~/.kde3.5/tmp-iridium  $


There are empty temp files, as expected.  Now, I'm going to cat a ~500K plain text file.

mculbert@iridium ~/.kde3.5/tmp-iridium  $ ls -l ~/txt/2006-November.txt
-rw-rw-r-- 1 mculbert admins 522769 2006-12-02 00:29 /home/mculbert/txt/2006-November.txt
mculbert@iridium ~/.kde3.5/tmp-iridium  $ cat /home/mculbert/txt/2006-November.txt
[trimmed]
mculbert@iridium ~/.kde3.5/tmp-iridium  $ ls -l
total 8840
drwx------ 2 mculbert users    16384 2010-03-22 10:51 .
drwx------ 8 mculbert users     8192 2010-01-08 11:23 ..
-rw------- 1 mculbert users    49632 2010-03-22 10:51 konsoleqr1387.tmp
-rw------- 1 mculbert users  8931600 2010-03-22 10:51 konsoleBc1387.tmp
-rw------- 1 mculbert users    12408 2010-03-22 10:51 konsoleVx1387.tmp
mculbert@iridium ~/.kde3.5/tmp-iridium  $


Now look at the temp files. catting a 510Kb file results in almost *9MB* of temp space used.  Just for comparison, I converted the same text to a PDF:

567981 2010-03-22 10:55 /home/mculbert/txt/2006-November.pdf

And an MS-Word doc:

1334784 2010-03-22 10:56 /home/mculbert/txt/2006-November.doc

What this means is that Konsole is using 6.8 times more temp space to store shell output than it would take if that same output was converted to an *MS office document*.

With disk usage like this, and a bunch of active shells, it's easy to take up huge amounts of disk space in a short period of time.  In my case, I'm still running into disk quotas regularly exclusively because of konsole, which leads to crashes across nearly everything running out of my home directory since apps like Firefox can no longer write to their databases.

One last thing, I tried setting my scrollback to max, but limited (99999 lines). And this is what RAM usage did when catting that same text file repeatedly:

Fresh start:
mculbert@iridium ~  $ ps -eo rss,args |grep konsole
22384 konsole
mculbert@iridium ~  $ cat txt/2006-November.txt

First:   34804 konsole
Second:  45948 konsole
Third:   57060 konsole
Fourth:  68164 konsole
Fifth:   79288 konsole
Sixth:   90400 konsole
Seventh: 101520 konsole
Eighth:  112636 konsole
Ninth:   113176 konsole  (mem usage plateaus here, I assume the scrollback is full)
 
So, 9*510Kb == ~4.5MB of text == ~110MB of RAM, or about 1.13KB per *line* of scrollback.

More stuff.  I hexdumped the temp file for my current session and it becomes clear that konsole is storing a huge amount of data *per character* stored in the temp files.  Here's a short example:

mculbert@iridium ~/.kde3.5/tmp-iridium  $ echo "UNIQUE UNIQUE"
UNIQUE UNIQUE

Then, I hexdump the temp file and find this:

000006d0  00 00 00 01 01 00 00 00  55 00 00 01 00 00 00 01  |........U.......|
000006e0  01 00 00 00 4e 00 00 01  00 00 00 01 01 00 00 00  |....N...........|
000006f0  49 00 00 01 00 00 00 01  01 00 00 00 51 00 00 01  |I...........Q...|
00000700  00 00 00 01 01 00 00 00  55 00 00 01 00 00 00 01  |........U.......|
00000710  01 00 00 00 45 00 00 01  00 00 00 01 01 00 00 00  |....E...........|

Looks like about 11 bytes of extra data per character of actual text stored.  What could possibly necessitate storing so much extra data?

By the way, this is incredibly easy to confirm.  Just start konsole, cat a file of a known size, and see how much disk or RAM gets eaten compared to the size of the file.

Can a dev confirm this so this bug could be moved into the realm of hopefully getting fixed?

> More stuff.  I hexdumped the temp file for my current session and it becomes
> clear that konsole is storing a huge amount of data *per character* stored in
> the temp files. 

I know that Konsole used to store 11 bytes of information per character in older versions (going back to KDE 3.5).  2 bytes for the actual character, 1 byte of formatting flags and 8 bytes of color information (4 bytes each for foreground and background color).  In the default color space there are only 16 possibilities for foreground and background color so a single byte would suffice.  However a while back (again, in KDE 3.5 or perhaps even earlier) support for full RGB colors was added (giving one byte per color channel plus another byte to specify the color space being used) to the existing code without refactoring it to use a more efficient data structure.

I thought this had been changed some time ago to store the formatting and character information separately though (in other words only have one formatting information object per range of equally-formatted text).  I cannot remember if this was done for KDE 4.3 or 4.4 though.

Confirmed also on KDE 4.6

Further to the problem of ludicrous amounts of space being consumed by these obsolete tempfiles, if a Konsole tab passes 2GB of scrollback stored the entire application crashes.

(In reply to comment #8)
> Further to the problem of ludicrous amounts of space being consumed by these obsolete tempfiles, if a Konsole tab passes 2GB of scrollback stored the entire application crashes.

That 2GB problem may be #146483. See its comment #2.

Konsole seems to be deleting tmp files now, closing this.