Bug 188528

To see the spaces added, just try to mark the /proc/cpuinfo text I pasted - text will be highlighted (and spaces in the first 14 lines, which shouldn't be there).

Could you check with current trunk? There are some recent commits addressing the trailing whitespace issue.

Lines in the history will have all the spaces.  Lines not in the history should have no spaces at the end.  
trunk has a fix where 1 space at the last spot of the line was deleted.

What's the difference between "lines in the history" and "lines not in the history"? Aren't all lines technically "in the history"?

No, lines that are visible on the screen are not in the history yet.

To test this, have 1 line in the history and 1 line not and then do a copy/paste.  The line in the history will have spaces at the end; the line not in the history will have no spaces in the end.

Oh, I see.

Anyway, if lines from history get extra spaces added, I consider it a bug.

Other implementations (i.e. rxvt) don't add such spaces.

> Lines in the history will have all the spaces. 
> Lines not in the history should have no spaces at the end. 

That's wrong - whether or not a line has been moved to a scrollback buffer is supposed to be a hidden implementation detail from the user's point of view.

These extra added spaces is a really ugly "feature" for developing and communication with other developers:

- mark/copy text from konsole, paste it to another file - spaces get appended: "md5sum" will now differ, "diff" will report changes etc.

- mark/copy text (i.e. logfile, debug messages, config files) from konsole, paste it to to an email - recipients will see the text awfully garbled.

*** Bug 189058 has been marked as a duplicate of this bug. ***

*** Bug 173315 has been marked as a duplicate of this bug. ***

I do a lot of shell scripting, and this new bug is a nightmare.
You can't copy text, which is in the scrollback buffer, without getting these white space characters at the end of each line.

Because of this bug I must use Putty for Linux instead of Konsole.
Please fix the trailing spaces.
Thanks

This bug can even be dangerous. I have a tool that reports filenames one on each line. Before the newline a backslash is printed, so I can easily copy&paste the output as input to a command:

filenum1 \
file2 \
file3

Copy&paste to e.g. "svn add" all these files:

svn add [middle mouse click]

Usually all the files are put on the command line. But with the spaces the backslash will escape a space and the "svn add" is executed too early. The next filename will be called as a command. Now imagine this is a script named "deletemyhomedirectory.pl".

> The next filename will be called as a command.

That's a different bug (maybe bug 89299) :-). In general I feel Konsole needs to prompt when pasting more than one line of text. Sometimes the clipboard simply doesn't contain what you think it does (especially if it isn't working correctly, which happens occasionally on trunk).

Regarding comment #14:
No, my case is not a different bug, at least as I understand this initial report here. The problem is not in _pasting_ to konsole, but in _copying_ from konsole. 

I will type spaces as "*" and the newline as "+" in this example: When copying (by just selecting the output with the mouse) the following from konsole:

filenum1 \+
file2 \+
file3+

... you get something like:

filenum1 \****+
file2 \*******+
file3*********+

When pasting this again with the middle mouse the backslash will have no effect, but it should have escaped the newline at the end.

Copying from gnome-terminal into konsole is no problem.

Sorry. I didn't mean that the erroneous trailing space wasn't affecting you, just that 'executing commands due to pasting many lines' is a related problem. You are seeing that also, though on further thought I realize that in this case you actually *want* all those lines pasted... so in your case, fixing the "reckless paste" bug doesn't actually help.

konsole --version
Qt: 4.5.2
KDE: 4.3.1 (KDE 4.3.1)
Konsole: 2.3

From a systems admin point of view, this makes konsole unusable for most network configuration changes.

I think this is the code that is causing this issue.  A new line grabs the length of the previous line; hence after a long line, all short lines afterwards will have extra spaces.

A new line will have size == 0

Screen.cpp - added in Revision 571962 

688 	 if (size == 0 && cuY > 0)
689 	{
690 	screenLines[cuY].resize( qMax(screenLines[cuY-1].size() , cuX+w) );
691 	}

I have this with KDE 4.3.85 too. And it's making my live to a small h*ll right now :) As many people I use output from scripts for a lot of things.

*** Bug 223172 has been marked as a duplicate of this bug. ***

*** Bug 223193 has been marked as a duplicate of this bug. ***

Hoping this will get fixed soon. I have to switch to gnome-terminal for most of my operations, which doesn't properly fullscreen.

Personally, I prefer xterm:
- non-broken copy-paste
- always comes up in a predictable size
- proper support for NFD unicode characters (as generated by Macintosh)
- compact, yet readable font

Ten months later, KDE 4.3.4, and the bug is still not fixed,
turning attention to itself with every cut/paste, followed by
a laborious manual whitespace trimming. Please fix it!

From the debugging I've done, this is clearly not true:

Screen.cpp:1200
        // retrieve line from history buffer.  It is assumed
        // that the history buffer does not store trailing white space
        // at the end of the line, so it does not need to be trimmed here

Created attachment 40272 [details]
kdebase-4.3.95-konsole-extra-spaces.patch

The attached patch appears to fix this for me.  It takes the approach that the assumption about the history buffer is wrong, and always removes spaces from the ends of lines.  An alternate approach would be to make sure the history buffer doesn't contain spaces.  That *might* take less memory, but I'm not sure how to do that.

Although certainly preferable to the current situation, this patch would introduce the inverse problem: remove spaces in the case where there really are some.

From my understanding, the problems seems to be due to some kind of "off-by-one" error, where the length of a line is stored, but somehow applied to the few following lines as well, rather than to just to the line to which it should apply...

The code change mentioned in comment #18 is 3years 5mo old - has this bug really been around that long?  I didn't think so.  Also, changing the code to remove that section doesn't appear to have any effect for me.

The space stripping is already being done to active lines on the screen.  My patch does it to both active and history lines, which is at least consistent, if not entirely correct.  It is though, as you say, preferable to the current situation.

I'm not sure konsole can handle extra spaces at the ends of lines properly as written though.  It appears that it writes spaces directly into the line data to act as clear characters, rather than handling clearing as a separate operation.  Has it ever been able to do this?  It doesn't now:

E.g.:

[orion@orca ~]$ echo "     "


Select the "output" and you'll get nothing.

(In reply to comment #28)
> The code change mentioned in comment #18 is 3years 5mo old - has this bug
> really been around that long?  I didn't think so.  Also, changing the code to
> remove that section doesn't appear to have any effect for me.

Hi Orion,

I'm glad to see some activity on this bug. I've tried Konsole4 every now and then since about 1 1/2 years and this bug has always kept me from using it. I've never used Konsole4 before that time. The versions of Konsole3 I used 1 1/2 years ago certainly did *not* have this bug.

I hope this helps.

Created attachment 40305 [details]
kdebase-4.3.95-konsole-extra-spaces.patch

After further testing it does appear that removing the code in comment #18 does indeed fix the problem.  In fact, it allows you to remove the stripping of trailing spaces from the active lines.  It should also reduce the memory usage of the history buffer for short lines.  Updated patch attached.

Now the echo "   " selection works in the active lines.  (Whereas before it only worked with lines in the history buffer!)

Guys, 
do you see how far we are behind professional terminal emulators like poderosa or securecrt?
Would you spend your money for tool like konsole?
The same situation is with e-mail clients (kmail) and with web browsers (konqueror). 
And what do developers do? They just time to time discard everything and start completely new branch before they have finished previous one. Or they work on sci-fi features like Akonadi...

(In reply to comment #31)
Hi Thomas,

Although you may be a troll I feel I need to post this one reaction on this subject.

 Guys, 
> do you see how far we are behind professional terminal emulators like poderosa
> or securecrt?
> Would you spend your money for tool like konsole?

I've been disappointed by software I payed for more than enough. I prefer to use free software and toss it whenever I don't like it. Once I read that this is fixed I'll give Konsole another spin. Until then I'll stick with XTerm which is also free software.

> The same situation is with e-mail clients (kmail) and with web browsers
> (konqueror). 

As for web browsers: Firefox is a free web browser, it serves my needs. I don't use Konqueror.

> And what do developers do? They just time to time discard everything and start
> completely new branch before they have finished previous one. Or they work on
> sci-fi features like Akonadi...

When KDE 4 was just released I shared your opinion about developers discarding everything. But now I think that actually a lot of code from KDE3 has been reused. The switch to plasma clearly has brought a lot of new possibilities to the desktop and I am amazed by the rate at which old KDE 3 features that where lost in the transition to KDE4.0 have reappeared in 4.2 and 4.3.

SVN commit 1083161 by hindenburg:

Fix issue where history lines had extra spaces at end.
Thanks to Orion Poplawski for patch.

I will backport to 4.4.1 later.

BUG: 188528


 M  +2 -22     Screen.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1083161

Okay, I see where the stripping of white space is needed in the active screen - when running vim.  Not when running less though, so not all curses apps.

Orion, what instances did you find where the stripping is required?

When copying from a terminal window running vim or less.  Not absolutely always, but most of the time.

SVN commit 1094735 by hindenburg:

SVN commit 1083161 by hindenburg:

Fix issue where history lines had extra spaces at end.
Thanks to Orion Poplawski for patch.

CCBUG: 188528



 M  +2 -22     Screen.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1094735

Guys, it seems to be fixed now if I select text from UNIX machines.

But if I connect to Cisco device it mostly adds 1 space at the end of each line.
I'm reporting this, because it works fine in Putty for Linux, but not in Konsole

From content switch:
--------------------



css1# 
css1# 
css1# 
css1# 

###All lines above have 1 space (I pressed enter)
css1# exit
css1# Connection to css1 closed by remote host.
Connection to css1 closed.







###Each line where I pressed Enter contains space
css1# 
css1# 
css1# 
css1# 
css1# 
css1# show clock 

Date: 03-16-2010
Time: 09:33:17

TimeZone: :  +00:+00:+00

Summer time: Disabled

css1# 






From 2 layer switch:
--------------------

sw21>sho
sw21>show clo

####here is space after clock
sw21>show clock 
09:21:33.073 GMT Tue Mar 16 2010
sw21>
sw21>




sw21>
sw21>
###here are 3 spaces
sw21>show in   

###here is 1 space (autocompleated after TAB)
sw21>show inventory 

####1 space (I pressed enter)
sw21>show inventory 
NAME: "1", DESCR: "WS-C3560G-48TS"
PID: WS-C3560G-48TS-E  , VID: V02, SN: FII2649Z

sw21>



###4 spaces (I pressed enter)

###4 spaces
sw21>enable    

###1 space
Password: 
sw21#sho
sw21#show runn

###1 space (I pressed enter here)
sw21#show running-config 
Building configuration...

Current configuration : 7426 bytes
!
! Last configuration change at 14:37:07 BST Sun May 3 2009 by root
! NVRAM config last updated at 00:43:24 BST Wed Apr 15 2009 by root
!
version 12.2
no service pad
service timestamps debug datetime msec localtime
service timestamps log datetime msec localtime
service password-encryption

I'm experiencing this problem with KDE 4.4.2 (openSUSE RPMs).  Is this a regression or has the fix not yet made it to 4.4.2?

It looks like there are some edge cases where it still shows up.

I.e. I see spaces added for text which is _not_ in history - KDE 4.4.2 on Mandriva.

I have seen this for awhile too and still with konsole 4.4.2 on debian.

I have seen this for awhile too and still with konsole 4.4.2 on debian.

Didn’t use to do it for a while in MC, but since 4.4.0 it does it all the time.  First I thought it might be a ShiftModifier “feature”, but I just looked into the source and couldn’t find anything like that.

Then I thought it might be a problem in Application-mode mouse, but mcedit --nomouse does the same.

Joe and less both work just fine.

It doesn’t do this when MC is in black-and-white mode; I’m thinking that in BW mode, MC is smart and doesn’t unnecessarily fill the lines up with spaces (which it needs to when in colour mode so there’s a proper blue fill)...  So in colour mode, Konsole feels these are useful spaces.

Just my two cents...

The same here. Problem is with (blue) background in MC. I'm using KDE 4.4.2 on Kubuntu. Do something with this, please.

It's not only mc.

Try this:

- fill the screen with some text (i.e. cat /etc/services or dmesg)
- decrease the text as much as you can (ctrl + -)
- mark the text
- paste it somewhere

The lines on top will have extra spaces.

My biggest annoyance in KDE4 ;)

When I cat file into console it mark copy&paste correctly, but when is in MC a white spaces are not removed and selected text is almost unusable.

Me KDE 4.4.85 (KDE 4.5 Beta2) compiled almost daily from SVN(GIT)

Thanks for help with this one

Maybe the behaviour with MC (is this the file manager midnight commander?) is not a bug, because perhaps MC does fill the background with colored spaces? Perhaps this is MC theme dependend? Does this happen with MC in another terminal (like xterm or gnome-terminal), too?

Yes, mc, mcedit. This is Konsole regression. It was working in 4.1. There's no issue in gnome-terminal.

This bug is extremely annoying. I'm on Ubuntu 10.04 with konsole 4:4.4.2-0ubuntu2 and I want to copy text between f.i. mutt and vim occasionally.

Hans, you should check if there is a more recent version of KDE for Ubuntu. If I am not wrong, the issue was partially fixed with 4.4.3 or 4.4.4. It works much better now in KDE4.4.4 than in 4.4.2, even if there seem some situations left where the problem still occurs.

Howdy there!

I've bypassed the problem by installing mrxvt.  No more trouble.  Much faster.  
Much simpler.  Entirely reliable.

Gary

On Saturday 17 July 2010 06:03:48 Hans Lambermont wrote:
> https://bugs.kde.org/show_bug.cgi?id=188528
>
>
> Hans Lambermont <hans@lambermont.dyndns.org> changed:
>
>            What    |Removed                     |Added
> ---------------------------------------------------------------------------
>- CC|                            |hans@lambermont.dyndns.org
>
>
>
>
> --- Comment #50 from Hans Lambermont <hans lambermont dyndns org> 
> 2010-07-17 12:03:32 --- This bug is extremely annoying. I'm on Ubuntu 10.04
> with konsole
> 4:4.4.2-0ubuntu2 and I want to copy text between f.i. mutt and vim
> occasionally.

I'm sorry.  This part of the patch was clearly wrong and needed in certain situations:

@@ -1230,19 +1223,6 @@
         Character* data = screenLines[screenLine].data();
         int length = screenLines[screenLine].count();
 
-        // Don't remove end spaces in lines that wrap
-        if (!(lineProperties[screenLine] & LINE_WRAPPED))
-        {
-            // ignore trailing white space at the end of the line
-            for (int i = length-1; i >= 0; i--)
-            {
-                if (data[i].character == ' ')
-                    length--;
-                else
-                    break;
-            }
-        }
- 
         //retrieve line from screen image
         for (int i=start;i < qMin(start+count,length);i++)
         {

Please restore.

Hi all,

I confirm that Konsole is not adding spaces at the end of lines copied from the history in the newest Mandriva 2010.1 version, this bug has been fixed.
 - Konsole version 2.4.3
 - KDE version 4.4.3

Thanks!
- Stéphane

I've checked 2.4.5 (4.4.5) and issue with mc still not fixed.

Confirming that the problem still exists in KDE 4.4.5 (openSUSE 11.3 RPMs).  I don't use mc; this problem occurs fairly consistently when copying and pasting anything from a konsole window.

The problem reported for mc also happens when copying from jed and minicom, and for some lines copied from emacs -nw and vim. The common factor appears to be the use of colors all the way to the end of the line. This can easily be confirmed by running the following shell commands in a fresh konsole window:
# tput setf 1
# cat /proc/cpuinfo

When copy/pasting the output, the first n lines (where n is the height of the konsole window) will be fine as the color doesn't extend to the end of the line. But new lines that have scrolled in from the bottom have been created with the blue foreground attribute for the whole line. These lines will get the additional spaces when copied.

I guess if someone has a prompt that sets the foreground/background color, they will see this problem "fairly consistently when copying and pasting
anything from a konsole window". Maybe Tristan can confirm/deny something like this is the case for him?

In mutt all colored lines (message headers, quotes) are copied with trailing spaces. This is in 4.5.1.

I have this problem using vim with a colored background in konsole 2.4.2 under kubuntu 10.04.

Start vim in konsole with no colors, past works OK

Add a background color, eg  

:hi normal ctermbg=white

...konsole is now sending trailing whitespace when I copy from it and paste elsewhere.

Problem does not exist using vim under xterm.

Reverting part from comment #53 fixed the problem for me. This code obviously deleted whitespace at the end of line!

http://cvs.pld-linux.org/cgi-bin/cvsweb/packages/kde4-kdebase/kde4-kdebase-bug-188528.patch?rev=1.1

Tried lates konsole from git and copy/paste seems now correct when compared to xterm and gnome-terminal.

That's weird since there is no single commit in konsole repo after 4.6.0 that could be fixing it. Was you able to reproduce in 4.6.0 and not able to reproduce in current git ? (then maybe the commit landed somewhere else, kdelibs or something)

Well, I confirm that the konsole from 4.6 RC2 has the bug, but not the one I just compiled from git.

This bug has a status of resolved, but it is still present in the Konsole version shipped with OpenSUSE 11.4 that just came out today (Konsole version 2.6 on KDE 4.6.00). So, is there a chance that this is ever going to be fixed or should I just definitively switch to a different terminal program that does allow proper copy/pasting?

Well, this will come with KDE 4.7

(In reply to comment #65)
> Well, this will come with KDE 4.7

Why not in KDE SC 4.6.x? This is rather a bugfix not the addition of a new feature.

If anyone has the source and can test this patch, please let me know if it works for you.
It should fix the issue when the fore/background colors are different.

https://git.reviewboard.kde.org/r/100913/

Reopened - I'll see about getting this in 4.6.2 if there are no issues.

(In reply to comment #67)
> If anyone has the source and can test this patch, please let me know if it
> works for you.
> It should fix the issue when the fore/background colors are different.
> 
> https://git.reviewboard.kde.org/r/100913/
> 
> Reopened - I'll see about getting this in 4.6.2 if there are no issues.

Tested, looks good. No spaces, where were previously.
Thanks.

(In reply to comment #67)
> If anyone has the source and can test this patch, please let me know if it
> works for you.

Well, I applied that patch to http://download.opensuse.org/repositories/KDE:/Release:/46/openSUSE_11.4/src/kdebase4-4.6.1-228.1.src.rpm and generated the konsole*.rpm package from it. It seems to behave identically to xterm. However, the original http://download.opensuse.org/repositories/KDE:/Release:/46/openSUSE_11.4/i586/konsole-4.6.1-228.1.i586.rpm also seems to be good.

> It should fix the issue when the fore/background colors are different.

What does it exactly mean?

Anyway I will continue using the newly compiled konsole package in order to check whether it has issues. So far it seems to be OK.

(In reply to comment #69)
> > It should fix the issue when the fore/background colors are different.
> 
> What does it exactly mean?

The issue with the spaces typically appeared when there were actual spaces in the terminal, used for things like drawing coloured areas, such as in text editors, especially the coloured mcedit.  It was less likely to appear with black-and-white apps that don't fill the screen.

Created attachment 58255 [details]
mc screenshot with the patch attached

(In reply to comment #70)
> (In reply to comment #69)
> > > It should fix the issue when the fore/background colors are different.
> > 
> > What does it exactly mean?
> 
> The issue with the spaces typically appeared when there were actual spaces in
> the terminal, used for things like drawing coloured areas, such as in text
> editors, especially the coloured mcedit.  It was less likely to appear with
> black-and-white apps that don't fill the screen.

(In reply to comment #70)
> (In reply to comment #69)
> > > It should fix the issue when the fore/background colors are different.
> > 
> > What does it exactly mean?
> 
> The issue with the spaces typically appeared when there were actual spaces in
> the terminal, used for things like drawing coloured areas, such as in text
> editors, especially the coloured mcedit.  It was less likely to appear with
> black-and-white apps that don't fill the screen.

I attached an mcedit screenshot after applying the patch. It seems to me that this patch just corrupts console. However, this may be because openSUSE (but I don't think so, because their konsole package is an unpatched 4.6.1 version). Can anyone else test it?

(In reply to comment #67)
> If anyone has the source and can test this patch, please let me know if it
> works for you.
> It should fix the issue when the fore/background colors are different.
> 
> https://git.reviewboard.kde.org/r/100913/
> 
> Reopened - I'll see about getting this in 4.6.2 if there are no issues.

OK, after testing on openSUSE 11.4 witk KDE 4.6.1 my conclusion is that this patch is not good, because:

1. It removes the trailing colors from the display too, not only from copied 
text.
2. It also deletes trailing spaces which are actually in the text being displayed!

(I tested it in vim with background: :hi normal ctermbg=blue. Interestingly xterm can display the background color all the way till the end of lines, and despite this it doesn't add unnecessary trailing spaces to the copied text, moreover is doesn't delete trailing spaces from the actual text)

Thanks for testing - there is a new patch if you want to test it.

The vim w/ background set is still an issue.

So, to sum it up, it seems to me (a layman) that currently (in KDE SC 4.6.1) the problem seems to be that when running editors with background color, konsole simly displays the background color by printing spaces onto the default background. This provides good displaying capabilities but it makes impossible to copy the actual text from the display :(

(In reply to comment #73)
> Thanks for testing - there is a new patch if you want to test it.
> 
> The vim w/ background set is still an issue.

Probably. I suffered an injury and am restricted to bed. However, my home netbook is very slow :( Compiling kdebase took more than half an hour. Anyway send me it, the necessary sources are still deployed.

(In reply to comment #75)
> (In reply to 393comment #73)
> > Thanks for testing - there is a new patch if you want to test it.
> > 
> > The vim w/ background set is still an issue.
> 
> Probably. I suffered an injury and am restricted to bed. However, my home
> netbook is very slow :( Compiling kdebase took more than half an hour. Anyway
> send me it, the necessary sources are still deployed.

I'm not sure what are you doing but you can compile just konsole itself and the rest install from KDE Unstable SC repository (http://download.opensuse.org/repositories/KDE:/Unstable:/SC/openSUSE_11.4/).

If I misunderstand something, sorry.

I am using the "White on black" profile and still have the spaces at the end.
Even just selecting text in the window.  No vim or mcedit - just a straight
select.
If you are saying colors are messing up the select then what colors can I use
other then plain "white on black"?

Created attachment 58260 [details]
A file that can be used to test correctness of terminal windows.

Instructions are contained in the file.

The file helps to test the correctness of terminal window software.

Gary

Created attachment 58469 [details]
A file that can be used to test correctness of terminal windows.

This version has tabs and spaces that wrap.

This file also illustrates that most (if not all) terminal windows fudge the last tab on an 80 column window to be 7 spaces, instead of 8.

Created attachment 58606 [details]
Restore removing trailing spaces

I compiled the latest git master konsole against 4.6.1 and I still see trailing spaces in selections.  As I indicated back in comment 53, I believe that I was mistaken in removing that section of code, and I apologize profusely for submitting it.  The attached patch is against current git master and restores that code.  Testing with this I no longer have trailing spaces in my selections.

Yes, I suppose it will erase truly existing trailing spaces at the end of lines, but I feel that is a *lot* smaller issue.

Please get this into the 4.6 branch as well, we really should address this as soon as possible.

(In reply to comment #80)


Suppose? Feel? You suggest to include a patch with an unknown behavior? Don´t do that, please! Will it look like my attachment in comment #71? I don´t think that´s a small issue. Anyway why don´t you think that actual trailing spaces are unnecessary? I´m afraid konsole is more fundamentally broken, so patches of few lines are not enough here :(

That patch (from comment #80) is the same as in comment #60 and is the same thing that was in kde sources before it was removed in #33. Unfortunately #33 wasn't fully correct thing to do.

That patch works here.

The patch does nothing to affect output.  I can't speak to any possible screen corruption.  I think that needs to be a separate bug.

The patch strips spaces from the end of lines when copying it to a stream - i.e. when copying to the paste buffer.  Now, you could imagine a case (I suppose) where a displayed file or output has real spaces at the end of a line that you want to preserve when copy and pasting.  In this case they would be stripped.  I argue that this is much less of a problem than what we have now and is the original behavior of konsole.

I agree with comment #83. Why can't konsole just remember what the line was like, just like xterm did? Neither filling up a line with spaces, nor removing all trailing spaces is ok. What we want is just remember the state "as is", and neither strip nor add to it.

Actually, it seems that in some cases, konsole manages to do it just right.

However, another annoyance is that there is no visual feedback as to the state of the line while selecting.

In xterm, if you move the mouse beyond the last character of a line while selecting, it shows black color til the right screen edge.

Konsole, on the other hand, only shows black until mouse position, even if mouse is already past the last character, just as if there were spaces there.

Yeah, the proper fixes would be to not write the spaces into the screen buffer in the first place.  Not sure where these come from.  vim certainly doesn't put them there.

*** Bug 270216 has been marked as a duplicate of this bug. ***

Hey, this bug had it second birthday few days ago ;-)

If xterm and gnome-terminal do it all right and konsole is broken for two years now. Why not just do it the same way as xterm or gnome-terminal do it?

Konsole really has nice features but with such fundamental things borked its just unusable for many people in many situations.

I could test current git all the time so please let me know when I should try some patch.

Hi!
Sorry for my English.

There is related problem due typo. To reproduce:
create a file in a few lines
cat << EOF > some_file
some text 1
some text 2
some text 3
some text 4
some text 5
EOF

open konsole
Run command 'cat some_file' until konsole does not scroll the screen contents.
Select multiple lines and paste the selection into a text editor.
Note the trailing spaces.

After applying the patch the problem disappears.

diff -Naur konsole-4.6.5.orig/konsole/src/Screen.cpp konsole-4.6.5/konsole/src/Screen.cpp
--- konsole-4.6.5.orig/konsole/src/Screen.cpp   2011-02-28 15:50:53.000000000 +0300
+++ konsole-4.6.5/konsole/src/Screen.cpp        2011-07-23 21:21:24.369999782 +0400
@@ -817,7 +817,7 @@
 
     //if the character being used to clear the area is the same as the
     //default character, the affected lines can simply be shrunk.
-    bool isDefaultCh = (clearCh == Character());
+    bool isDefaultCh = (clearCh == defaultChar);
 
     for (int y=topLine;y<=bottomLine;y++)
     {

*** Bug 213620 has been marked as a duplicate of this bug. ***

Is there a patched version of konsole available in a Debian or Ubuntu repository somewhere, or as a .deb-file, under a different name (fork) if necessary? I do not expect this bug is going to be solved soon. I've waited too long for that. I tried gnome-terminal for some time. In gnome-terminal, copying text works as expected, but it doesn't have the activity-reporting (changing colour of tab name) that konsole gives me. It's the activity reporting I like in konsole. But I hate this bug. If there's a patched version available, I'ld like to use it. If not, I have to apply a patch to the source code myself and compile konsole.

Hello,

On 08/09/2011 09:55 AM, Fred Vos wrote:
> https://bugs.kde.org/show_bug.cgi?id=188528
>
>
>
>
>
> --- Comment #90 from Fred Vos <fred fredvos org>  2011-08-09 07:55:38 ---
> Is there a patched version of konsole available in a Debian or Ubuntu
> repository somewhere, or as a .deb-file, under a different name (fork) if
> necessary? I do not expect this bug is going to be solved soon. I've waited too
> long for that. I tried gnome-terminal for some time. In gnome-terminal, copying
> text works as expected, but it doesn't have the activity-reporting (changing
> colour of tab name) that konsole gives me. It's the activity reporting I like
> in konsole. But I hate this bug. If there's a patched version available, I'ld
> like to use it. If not, I have to apply a patch to the source code myself and
> compile konsole.

Just follow the instructions here to setup your development environment:
http://techbase.kde.org/Getting_Started

Then grab Konsole's sources from here:
https://projects.kde.org/projects/kde/kdebase/konsole/repository

And then give us a patch! :-)

Valentin

I can confirm that the patch from comment #80 does indeed fix the issue.
Valentin, you have the patch, now it's your turn.

Mageia implemented the patch from comment 80 a few days ago, and I can also confirm that this patch fixes the issue. Please apply it upstream to.

Created attachment 63027 [details]
Distinguish between real space and cleaning character

No, it really didn't solve anything, only adds another bug. Did you read the discussion?

Anyway, I'm attaching something to discussion. To my knowledge problem is that Screen::clearImage clears screen buffer by writing spaces to it. Of course it *is* desiderable to write characters to screen ie. to be able to clear screen with some background color as in mcedit.

So lets flag these cleaning spaces (as opposite to actual spaces comming from input) with new special flag RE_CLEAR_CHAR. If you want to see, what I'm calling cleaning spaces, go to Screen::clearImage and temporarily set c to '!' at the begining of function.

Now we need to do a filtering similar to proposed one in patch on comment 80, but not in Screen::copyLineToStream! PlainTextDecoder::decodeLine is IMHO right place. There even was some sort of filtering, but was not working.

At first, it was not working in mc, because _includeTrailingWhitespace was wrongly set, so it wasn't removing empty line ending in mcedit. Secondly, because newline char was at the line end after spaces, it wasn't removing anything.

I'm also looked into problems with tab characters, which are not handled properly. I was only partialy successful, still I'm not fully understanding line wrapping and how to deal with tabs on line end properly, so this is not included.

> At first, it was not working in mc, because _includeTrailingWhitespace was
> wrongly set, so it wasn't removing empty line ending in mcedit. Secondly,
> because newline char was at the line end after spaces, it wasn't removing
> anything.

mc works fine: I applied the patch
https://bugs.kde.org/show_bug.cgi?id=188528#c80
https://bugs.kde.org/show_bug.cgi?id=188528#c88

(In reply to comment #95)
> > At first, it was not working in mc, because _includeTrailingWhitespace was
> > wrongly set, so it wasn't removing empty line ending in mcedit. Secondly,
> > because newline char was at the line end after spaces, it wasn't removing
> > anything.
> 
> mc works fine: I applied the patch
> https://bugs.kde.org/show_bug.cgi?id=188528#c80
> https://bugs.kde.org/show_bug.cgi?id=188528#c88

I was talking about a piece of code in PlainTextDecoder::decodeLine, which is affected by my suggested patch, not about the patch from comment 80. Patch from comment 80 deletes all spaces from end of line without distinguishing whetever they should or slould not be here.

You can test it - make a file with one word, five spaces and linefeed. Cat file in konsole and copy and paste output to editor. In editor you should see word and five spaces. Patch from comment 80 deletes all spaces, so you will se only word alone. Unpatched konsole adds additional spaces to fill whole line. My patch deals with that AFAIK correctly.

(In reply to comment #88)
> Hi!
> Sorry for my English.
> 
> There is related problem due typo. To reproduce:

Well, I doubt the problem is related with or fixed by that patch.

1). Yes, the piece of code looks like a typo
2). But, Character() == defaultChar. Just take a look at the definition of defaultChar and the default parameters of Character::Charcter(...) 

Anyway, that patch does make the code more natural. Thanks for you help.

> 2). But, Character() == defaultChar. Just take a look at the definition of
> defaultChar and the default parameters of Character::Charcter(...) 

may be issue a little subtle:
try commands from comment #88 with and without patch - note the difference
 
> Anyway, that patch does make the code more natural. Thanks for you help.

You are welcome.

*** Bug 269321 has been marked as a duplicate of this bug. ***

(In reply to comment #94)

That patch works(at least for me) , but I think there are some issues:

1). I thinks it is the expected behavior for _includeTrailingWhitespace to be true when copying text. I'm not sure about that statement, though.

2). It is not a good idea to use RE_CLEAR_CHAR to distinguish real (space) characters and place holders. They are really rendered  in the same way in the display. Their difference is meaningful only when decoded into text stream.

I created another patch based upon yours and submitted at https://git.reviewboard.kde.org/r/102906/ . You can also submit your patch on reviewboard so that it can be better tracked, instead of being forgotten.

Testing and feedback are welcome for the patch from comment #94 and this one.

Thank you for your reply and your patch.

1) I don't know, what _includeTrailingWhitespace exactly is. Probably it is true because of blue mcedit background. Therefore you are suppressing nonexisting spaces not only on line ends, but also on line beginings.

This problem is not so easy to spot. When I'm copying from mcedit on my home box, it behaves correctly. But when I ssh to old server which runs old mcedit, this mcedit doesn't print indentation explicitly (only moves cursor) and then such spaces are riped off and copied text looses indentation.

Alternatively you could test it with something like dialog --infobox "qwe" 10 10 and you will see, that it will suppress spaces on the left of dialog.

2) Main reason was not to waste RAM :-)

3) Yes, tab character problem can be solved similar way. You can write tab character normally into screen buffer and it will be copied to clipboard. You just need to filter out padding between tabs when copying to clipboard.

I've been running konsole 4.7.44 patched with https://git.reviewboard.kde.org/r/102906/ under KDE 4.7.2 and noticed that url highlighting seems to be off.  If there is multiple spaces in front of the url, the underline appears starting after the first space.  Anyone else seeing this in any versions?

*** Bug 254628 has been marked as a duplicate of this bug. ***

Any progress here?

(In reply to comment #104)
> Any progress here?

Well, yes and no :)

I made some change to the previous patch and it works with the dialog example in comment #101. Now konsole can copy that whole multi-lines box correctly just as in xterm( the patch in comment #94 seems to have problem when copying the whole box ). No missing leading whitespaces. No extra trailing whitespaces. That is the yes part.

However, nothing has been committed yet. I still don't have total confidence about that _includeTrailingWhitespace. The patch is not as elegant as I have hoped.  My feeling is maybe this is another patch which solves most problems but not all problems. That is the no part.

Considering the age and vote of this bug, I think maybe it is better to commit the patch than doing nothing. I will do it some time later.

Please commit - this is an annoying bug! thanks!

About _includeTrailingWhitespace: Screen::selectedText creates
PlainTextDecoder (_includeTrailingWhitespace is true by default). It calls
Screen::writeSelectionToStream that calls Screen::writeToStream which a)
calls copyLineToStream which calls decoder->decodeLine (history buffer?)  
and b) finally calls decoder->decodeLine.

So the following should make my proposition to comment-out if(!_includeTrailingWhitespace) unneeded. What do you think?

diff --git a/src/Screen.cpp b/src/Screen.cpp
index 1ec2f03..69ee7ea 100644
--- a/src/Screen.cpp
+++ b/src/Screen.cpp
@@ -1150,6 +1204,7 @@ QString Screen::selectedText(bool
preserveLineBreaks) const
     QTextStream stream(&result, QIODevice::ReadWrite);
     
     PlainTextDecoder decoder;
+    decoder.setTrailingWhitespace(false);
     decoder.begin(&stream);
     writeSelectionToStream(&decoder , preserveLineBreaks);
     decoder.end();

Git commit 72a1adc63100b9796795eb9872f53987a7be637d by Jekyll Wu.
Committed on 25/11/2011 at 18:03.
Pushed by jekyllwu into branch 'master'.

Avoid extra trailing whitespace when copying text

A new 'isRealCharacter' flag is added into the 'Character' class
to distinguish real characters and charaters used as place holders.

The code is a bit ugly now. Should improve it later when the idea and
implementation are proven to be feasible and cause no regression.

CCBUG: 188528

M  +14   -2    src/Character.h
M  +5    -2    src/Screen.cpp
M  +30   -2    src/TerminalCharacterDecoder.cpp

http://commits.kde.org/konsole/72a1adc63100b9796795eb9872f53987a7be637d

Unfortunately it needs too big effort to remake all kde distribution in debian testing so I've manually added that commit into source tree of 4.6.5 and found out it helped with spaces but I've got another issue with it: it removes last EOL along with spaces so I cannot copy just two lines with EOLs - it takes first line with EOL but for second line it's only content of the line without EOL. May be it's bug due to incompatibility of that commit with 4.6.5 and it will work well with current Git but I cannot check it right now unfortunately as debian maintainers aren't fast enough. If you can point me how to fix that I would be very glad.

Additional info about losing EOL: EOL isn't lost if copied from plain console (history as well, at least with patch from comment #88) or from less but EOL is lost if copied from slang-based applications such as mutt or mc. Hope you can fix that or at least give a hint where to fix. Thanks.

Yet another bug with this patch. Use case:
1) select a block with empty line(s) in mc;
2) paste it: you get a line with a space char instead of empty line.

(In reply to comment #111)

Thanks for your testing and feedback. 

The problem mentioned in comment #110 also exists on master code. It is my fault to push the commit without noticing that. However, after making some investigation, I think the real problem behind that "EOL of the last line" problem is the previous commit reveals another hidden problem . The faulty logic related with the "EOL of last line", combined with the previously faulty logic related with "trailing non-existent spaces", surprisingly produced the right behavior in regard to the "EOL of the last line" issue. 

I'm make more investigation into that "EOL of the last line" problem, but I prefer not to fix it until the "trailing non-existent spaces" problem is confirmed by users to be really fixed by previous commit. 

I do not quite understand the problem mentioned in comment #111. I do not use mc a lot, so "select a block" is a little confusing to me. I tested mcedit before pushing the commit, and empty lines were pasted as empty lines.

The previous commit should have been contained in the KDE SC 4.8 beta2. More testing and feedback, please :)

Thank you too for making such a useful application. About comment #111 - by "select a block" I meant some text which consists of few lines. Sorry for unclear spelling. Also I've discovered that it not always gets a space on the line and I could not find any pattern for that behavior yet unfortunately. I'll attach a file .joerc here, bug is reproducible for me in next case:
1) run a konsole with 30 rows (only 30 would reproduce a bug);
2) open this file with mcview;
3) scroll one screen down - first viewed line will be empty one;
4) select few lines starting from empty;
5) paste it into any editor - empty line isn't empty but contains 8 spaces.

Created attachment 66552 [details]
the test case for comment #113

Ouch, sorry for incomplete description - (1) should be exactly 30 rows and at least 81 columns to get (3). And you can paste it not only into editor but anywhere (into comments' input field on this page for example).

I built konsole 4.7.90 on F16 with kdelibs 4.7.3.  No extra spaces with vim, my main issue.  URL highlighting looks good too.  Works with "less" too.

I think I've noticed this before with "more", but lines longer than the width of the terminal window get line breaks in the copy/paste when selected.  Is this a product of "more" or can this be fixed?

One more strangeness, which may be an artifact of different kde versions.  I have focus-follow-cursor set.  I have two side by side konsole windows.  If I select text in one and then move the other, the new window does not get focus until I click.  This does not happen with konsole 4.7.3.

Hmm, the focus issue isn't always reproducible.

Created attachment 66565 [details]
konsole joerc explanation

(In reply to comment #113)

> 5) paste it into any editor - empty line isn't empty but contains 8 spaces.

This is probably not so much a bug of konsole, but rather mcview and mcedit behavior. Both these programs apparently does some optimizations while painting screen and mcview actually prints these spaces! Or more preciselly it prints spaces representing a TAB character on the first line in the .joerc file when you first open it and then when you press PgDn, mcview remebers this first 8-character space is visually "empty" so it erases only the rest of the line.

Look at the attached screenshot - the little line (grave accent) character denotes cleaned space (originally space with isRealCharacter = false). You will see that mcview doesn't bother to clean this space.

I have the experience with some more than five years old version of mcedit (cooledit) which prints text in even more unpredictable ways and which for example prints one space after each line of source code iff that line was highlighted by syntax highlighter. (I've actually analyzed bytestream coming from mcedit do konsole and it's not a konsole fault.) Newer mcedit are much less pain in that regard.

Ah, I see, thank you a lot for a good explanation, that explains everything and I'll poke mcview authors then to make that solved. :)
So only bug which is still left of the topic is "EOL of the last line" one. Thanks!

But after some thinking... as you've checked it already... tell me please what mcview/mcedit does anyway? Does it fill rest of line with null chars or with spaces? If those are spaces too then mechanics of patch from comment #108 should not leave any spaces, isn't it? So probably that line after PgDn consist of 8 spaces and rest are null chars?

I'm a spammer here I think. Sorry for that. I've re-read all comments again and I see that line which was rewritten by mcview perhaps consists of 8 spaces which were there before rewrite and other spaces which were written instead of 'real' characters of previous content so in terminal ALL line is plain spaces and I don't see anything wrong with it. I think the patch in comment #108 only partially covers that then as it should cover full line in case of partial rewrite instead of only rewritten part.
Note that due to this bug I'm using konsole from TDE (it's continuation of KDE3 you know) at home and it doesn't have that issue with extra spaces on PgDn on the same host where I run konsole which we testing here. And even worse - before last system upgrade I had konsole from ubuntu karmic (KDE 4.3.2) at home and there was no such bug then too (probably it was with lines in history but never had any problem with lines in mc or mutt) but after upgrade it gone incompatible with rest of KDE unfortunately and that's basically why I was forced to add TDE into my repositories to have working konsole.
And there was mentioned focus... I have some very weird focus issue (with both konsole 2.6.4 and 2.7.2 that are available to test right now) which is present neither in konsole from TDE nor in VTE-based terminals. Assuming that isn't topic here, I probably should open another bugreport if there isn't one already but that issue is still regression and I think about giving up and forgetting of KDE4 for good as I've replaced almost everything with alternate utilities due to numerous regressions. Sorry.

I can confirm this bug in KDE 4.8 When copying text from Konsole, some lines get many tens of spaces appended to the end of the line. I cannot figure out which lines get the spaces and which don't, that is, I don't see the distinction as being determinant.

What application? What text?

> What application? What text?
> 

I can reproduce with almost any text in Konsole: commands in the CLI that are shown on the screen and the CLI output of CLI applications. This includes both those lines that are shown on the screen and those that I must scroll up to see.

Here is an example, note that lines 21-37 have many spaces at the end (you might need to highlight with the mouse to see them):

 - saturn:~$ cat diff.txt 
21,31d20
< 2007-10-19T18:07:04-000019.jpg
< 2007-11-10T14:20:28-000007.jpg
< 2007-11-28T09:00:38-000024.jpg
< 2007-12-14T16:17:50-000042.jpg
< 2008-01-12T12:58:02-000019.jpg
< 2008-01-12T15:39:16-000143.jpg
< 2008-01-12T15:56:44-000162.jpg
< 2008-02-21T15:50:02-000030.jpg
< 2008-12-05T14:15:40-000084.jpg
< 2008-12-07T22:08:24-000002.jpg
< 2008-12-08T11:15:18-000023.jpg
34,44d22
< 2009-04-10T12:32:03-000059.JPG
< 2009-04-20T10:11:15-000157.JPG
< 2009-04-29T15:10:09-000083.JPG
< 2009-05-22T11:18:54-000205.JPG
< 2009-05-27T13:27:59-000035.JPG
< 2009-05-30T14:21:33-000050.JPG
< 2009-05-30T17:48:51-000068.JPG                                                                                                     
< 2009-06-03T09:32:16-000091.JPG                                                                                                     
< 2009-06-03T10:04:44-000112.JPG                                                                                                     
< 2009-06-06T18:59:38-000006.JPG                                                                                                                 
< 2009-06-11T20:03:18-000066.JPG                                                                                                                 
47,76d24                                                                                                                                                       
< 2009-07-03T10:13:27-000075.JPG                                                                                                                               
< 2009-07-10T17:30:46-000006.JPG                                                                                                                               
< 2009-07-13T19:00:59-000031.JPG                                                                                                                               
< 2009-07-16T13:03:23-000062.JPG                                                                                                                               
< 2009-07-18T19:14:07-000114.JPG                                                                                                                               
< 2009-07-23T08:08:51-000136.JPG                                                                                                                               
< 2009-07-24T13:16:42-000274.JPG                                                                                                                               
< 2009-07-31T08:33:47-000291.JPG                                                                                                                               
< 2009-08-05T06:49:56-000003.JPG                                                                                                                                                 
< 2009-08-09T20:57:40-000035.JPG                                                                                                                                                 
< 2009-08-21T08:28:30-000114.JPG                                                                                                                                                 
< 2009-08-22T20:26:25-000123.JPG
< 2009-08-23T00:26:30-000231.JPG
< 2009-08-23T00:30:09-000240.JPG
< 2009-08-23T00:30:13-000241.JPG
< 2009-08-28T20:34:04-000344.JPG
< 2009-08-30T11:04:46-000059.JPG
< 2009-08-30T11:21:50-000111.JPG
< 2009-08-30T12:37:30-000215.JPG
< 2009-09-04T16:37:52-000015.JPG
< 2009-09-20T09:49:30-000043.JPG
< 2009-09-20T09:50:05-000045.JPG
< 2009-10-03T11:53:02-000083.JPG
< 2009-10-03T14:30:25-000135.JPG
< 2009-10-09T16:40:17-000147.JPG
< 2009-10-10T13:23:39-000034.JPG
< 2009-10-10T13:33:41-000052.JPG
< 2009-10-12T12:08:34-000101.JPG
< 2009-10-13T14:15:35-000116.JPG
< 2009-12-13T18:06:54-000213.JPG
79,80d26
< 2010-01-01T21:38:57-000086.JPG
< 2010-01-02T23:46:48-000043.JPG
 - saturn:~$

Note that running the command a second time and then highlighting and middle-mouse-pasting the text shows that the additional spaces are only in the first run of the text, but not the second. This means that the spaces are actually in the console and not added at the time of copy (because they appear the second time that the user copies as well). Also, it means that they were not added by the cat command but rather by Konsole (as the second command does not have them).

 - saturn:~$ cat diff.txt 
21,31d20
< 2007-10-19T18:07:04-000019.jpg
< 2007-11-10T14:20:28-000007.jpg
< 2007-11-28T09:00:38-000024.jpg
< 2007-12-14T16:17:50-000042.jpg
< 2008-01-12T12:58:02-000019.jpg
< 2008-01-12T15:39:16-000143.jpg
< 2008-01-12T15:56:44-000162.jpg
< 2008-02-21T15:50:02-000030.jpg
< 2008-12-05T14:15:40-000084.jpg
< 2008-12-07T22:08:24-000002.jpg
< 2008-12-08T11:15:18-000023.jpg
34,44d22
< 2009-04-10T12:32:03-000059.JPG
< 2009-04-20T10:11:15-000157.JPG
< 2009-04-29T15:10:09-000083.JPG
< 2009-05-22T11:18:54-000205.JPG
< 2009-05-27T13:27:59-000035.JPG
< 2009-05-30T14:21:33-000050.JPG
< 2009-05-30T17:48:51-000068.JPG                                                                                                     
< 2009-06-03T09:32:16-000091.JPG                                                                                                     
< 2009-06-03T10:04:44-000112.JPG                                                                                                     
< 2009-06-06T18:59:38-000006.JPG                                                                                                                 
< 2009-06-11T20:03:18-000066.JPG                                                                                                                 
47,76d24                                                                                                                                                       
< 2009-07-03T10:13:27-000075.JPG                                                                                                                               
< 2009-07-10T17:30:46-000006.JPG                                                                                                                               
< 2009-07-13T19:00:59-000031.JPG                                                                                                                               
< 2009-07-16T13:03:23-000062.JPG                                                                                                                               
< 2009-07-18T19:14:07-000114.JPG                                                                                                                               
< 2009-07-23T08:08:51-000136.JPG                                                                                                                               
< 2009-07-24T13:16:42-000274.JPG                                                                                                                               
< 2009-07-31T08:33:47-000291.JPG                                                                                                                               
< 2009-08-05T06:49:56-000003.JPG                                                                                                                                                 
< 2009-08-09T20:57:40-000035.JPG                                                                                                                                                 
< 2009-08-21T08:28:30-000114.JPG                                                                                                                                                 
< 2009-08-22T20:26:25-000123.JPG
< 2009-08-23T00:26:30-000231.JPG
< 2009-08-23T00:30:09-000240.JPG
< 2009-08-23T00:30:13-000241.JPG
< 2009-08-28T20:34:04-000344.JPG
< 2009-08-30T11:04:46-000059.JPG
< 2009-08-30T11:21:50-000111.JPG
< 2009-08-30T12:37:30-000215.JPG
< 2009-09-04T16:37:52-000015.JPG
< 2009-09-20T09:49:30-000043.JPG
< 2009-09-20T09:50:05-000045.JPG
< 2009-10-03T11:53:02-000083.JPG
< 2009-10-03T14:30:25-000135.JPG
< 2009-10-09T16:40:17-000147.JPG
< 2009-10-10T13:23:39-000034.JPG
< 2009-10-10T13:33:41-000052.JPG
< 2009-10-12T12:08:34-000101.JPG
< 2009-10-13T14:15:35-000116.JPG
< 2009-12-13T18:06:54-000213.JPG
79,80d26
< 2010-01-01T21:38:57-000086.JPG
< 2010-01-02T23:46:48-000043.JPG
 - saturn:~$ cat diff.txt 
21,31d20
< 2007-10-19T18:07:04-000019.jpg
< 2007-11-10T14:20:28-000007.jpg
< 2007-11-28T09:00:38-000024.jpg
< 2007-12-14T16:17:50-000042.jpg
< 2008-01-12T12:58:02-000019.jpg
< 2008-01-12T15:39:16-000143.jpg
< 2008-01-12T15:56:44-000162.jpg
< 2008-02-21T15:50:02-000030.jpg
< 2008-12-05T14:15:40-000084.jpg
< 2008-12-07T22:08:24-000002.jpg
< 2008-12-08T11:15:18-000023.jpg
34,44d22
< 2009-04-10T12:32:03-000059.JPG
< 2009-04-20T10:11:15-000157.JPG
< 2009-04-29T15:10:09-000083.JPG
< 2009-05-22T11:18:54-000205.JPG
< 2009-05-27T13:27:59-000035.JPG
< 2009-05-30T14:21:33-000050.JPG
< 2009-05-30T17:48:51-000068.JPG
< 2009-06-03T09:32:16-000091.JPG
< 2009-06-03T10:04:44-000112.JPG
< 2009-06-06T18:59:38-000006.JPG
< 2009-06-11T20:03:18-000066.JPG
47,76d24
< 2009-07-03T10:13:27-000075.JPG
< 2009-07-10T17:30:46-000006.JPG
< 2009-07-13T19:00:59-000031.JPG
< 2009-07-16T13:03:23-000062.JPG
< 2009-07-18T19:14:07-000114.JPG
< 2009-07-23T08:08:51-000136.JPG
< 2009-07-24T13:16:42-000274.JPG
< 2009-07-31T08:33:47-000291.JPG
< 2009-08-05T06:49:56-000003.JPG
< 2009-08-09T20:57:40-000035.JPG
< 2009-08-21T08:28:30-000114.JPG
< 2009-08-22T20:26:25-000123.JPG
< 2009-08-23T00:26:30-000231.JPG
< 2009-08-23T00:30:09-000240.JPG
< 2009-08-23T00:30:13-000241.JPG
< 2009-08-28T20:34:04-000344.JPG
< 2009-08-30T11:04:46-000059.JPG
< 2009-08-30T11:21:50-000111.JPG
< 2009-08-30T12:37:30-000215.JPG
< 2009-09-04T16:37:52-000015.JPG
< 2009-09-20T09:49:30-000043.JPG
< 2009-09-20T09:50:05-000045.JPG
< 2009-10-03T11:53:02-000083.JPG
< 2009-10-03T14:30:25-000135.JPG
< 2009-10-09T16:40:17-000147.JPG
< 2009-10-10T13:23:39-000034.JPG
< 2009-10-10T13:33:41-000052.JPG
< 2009-10-12T12:08:34-000101.JPG
< 2009-10-13T14:15:35-000116.JPG
< 2009-12-13T18:06:54-000213.JPG
79,80d26
< 2010-01-01T21:38:57-000086.JPG
< 2010-01-02T23:46:48-000043.JPG
 - saturn:~$

i'll start by noting that i use kde as my primary and pretty much only desktop environment since ~ 2001-2003, i really love konsole and i believe that it is the best terminal application out there.

but i also believe that this issue is what some call "technical wanking" - arguing about minor technical detail while most users live with broken functionality.

i' claim that 99% of users (a figure i just made up) do not care about copying trailing space. they don't give a flying - something. for the remaining 1% it anyway doesn't work properly.

it was reported back in 2009. march. it's been 3 and a half years. i would claim that a proper approach would have been "strip everything as before" (because it was broken for users, no matter what tiny, technical details might have been behind it. see http://h30565.www3.hp.com/t5/Feature-Articles/Linus-Torvalds-s-Lessons-on-Software-Development-Management/ba-p/440 , search for "Breaking the user experience in order to ‘fix’ something is a totally broken concept; you cannot do it."). _if_ anybody cares about copying trailing space, that could be developed and tested separately, without messing with user experience for 3 years...

sorry for the rant. i'd like to express my gratitude to all konsole developers - it is a tool i use for the most pat of my working day, thus this "tiny" bug has caused me thousands of annoyances over the years.

If this bug doesn't bother you, good for you!

However, for the rest of us we _do_ care... or else we wouldn't have reported it.

You consider this "technical wanking". So let me ask you this question: when does a wanker stop rubbing? Right: after he has come!

With bug reports it's the same thing: people will stop commenting as soon as it is fixed. So if this talk about "minor technical detail" bothers you so much, then JUST FIX IT, and I'm sure all the wankers will stop wanking.

Hm, with 4.9 I don't see this problem anymore.

(In reply to comment #127)
this bug bothers me a lot, so i'm not sure whether you did read my comment fully :)
i don't want to repeat it, but short version - most users do not want to copy trailing space, majority probably doesn't what that is. as such, the konsole/kde3 behaviour was better overall.

Or, on a more serious note:

From the outside (without going into the code), it looks as if konsole does not actually remember the length of lines but instead just assumes a height x width array of characters, and then tries to infer line length from the position of the last non-space character.

So that's how we oscillate between "lines are always padded with spaces on copy-paste" and "trailing spaces are always stripped on copy-paste", but never quite reach "copy-paste copies exactly what the application had printed to the terminal". What those nitpicky users want is "konsole should explicitly remember how long each line was when printed, and copy-paste exactly that".

Methinks, there is no way around an explicit character-per-line counter.

Of course, this would still not work for full-screen curses applications (such as vi), but this is known and accepted. What people get worked up about is when it breaks for the non-curses stuff.

(In reply to comment #129)

Sorry if I misunderstood you, but when you said "most users do not care", I mistakenly assumed that you agreed with those users.

(I just re-tested it with a simple test case in 4.8.5: the bug no longer seems to exist... So either it is fixed by now, or it has become much more subtle... So, sorry for jumping into this too quickly)

 (In reply to comment #131)

The bug still exists in 4.8.5 but, as you wrote, it became much more subtle.

When I use mcedit on localhost and select a paragraph, the padding whitespace is not copied.
But when I ssh to a remote Centos 5.7 box and use mcedit there, I hit the bug again - padding spaces are copied to clipboard.
To make the whole issue even more confusing, when I ssh to a Fedora Core 6 PC, copying works as expected - no trailing padding spaces.

It behaves as expected anywhere if I use this patch: https://build.opensuse.org/package/view_file?file=kde4-kdebase-bug-188528.patch&package=konsole&project=home%3Adavid-smid
(AFAIK the same patch as https://bugs.kde.org/attachment.cgi?id=58255)

Sorry, I made a mistake, the patch URL is https://bugs.kde.org/attachment.cgi?id=58606

(In reply to comment #133)
> Sorry, I made a mistake, the patch URL is
> https://bugs.kde.org/attachment.cgi?id=58606

This patch appears to ovverride count() with a reduced count that actually _removes_ spaces that are at the end of the line. If count() doesn't give the actual line length, wouldn't it make more sense to fix count(), rather than trying to use such a kludge (... such kludges may well be the reason why this bug was there in the first place!). Or did I miss something here, like some additional condition where such stripping applies.

Developers, when applying this patch, please make sure to test whether it doesn't actually worsen the situation, by testing it with a simple test case:

1. Make a file with several lines, some of which end in one (or more) spaces, and some of which who don't. Use a variety of longer and shorter lines (in the past, bug only occured when a shorter line followed a longer one)
2. Using a shell running in konsole, cat the file
3. Press return a couple of times until file scrolls off screen.
4. Using scrollbar, scroll back, and then copy-paste file into an editor (indeed, in the past bug sometimes only occured on content that was scrolled off-screen)
5. Check that all lines which had spaces at the end still have, and that those who didn't have any originally, didn't acquire any.

(In reply to comment #134)

I see your point but there's no easy way out.

Some ncurses editors just pad with spaces under certain conditions. I'd like to be able to copy the text without trailing spaces - I know this is not correct but "the correct behaviour" just makes my life worse.

I agree  that konsole should copy the trailing spaces that were printed (as it does in 4.8.5) - that should be the default behaviour. But at the same time I'd like to have an option to force konsole to trim the trailing whitespace.

Maybe a configuration option?(In reply to comment #135)
> I agree  that konsole should copy the trailing spaces that were printed (as
> it does in 4.8.5) - that should be the default behaviour. But at the same
> time I'd like to have an option to force konsole to trim the trailing
> whitespace.

Yes, but indeed, that should be an _option_. Not forced on everybody. Maybe some checkbox in the settings (preferably default off, but as long as it is easy to find, I'm ok with default on too)

i'm offering a beer to whoever gets such an option in :)
(depending on where we might meet, maybe next akademy or sooner...)

I feel responsible, because I have proposed the idea on which current behavior is based. So I'll try to explain it in more detail.

Running applications are issuing commands to konsole using VT102 protocol. In the most simplest mode of operation it is a stream of characters to be written on the screen and that's the way commands such as cat /some/file operates. However applications like mcedit makes greater use of VT102 - they issue commands to move cursor around the screen or erase portions of screen. If you look into Vt102Emulation::processToken function you will see many possibilities to clear a portion of screen - application can request clearToEndOfLine, clearToBeginOfLine, clearEntireLine etc. So mcview and similar application doesn't clear screen by directly writing space char on it, but rather using VT102 clearing commands.

All these clear operations are mapped in konsole source code to Screen::clearImage(int loca, int loce, char c) with space character in c variable (in my screenshot replaced by another character to visualize) used to overwrite previous contents in a screen buffer. Konsole's screen buffer is basically a 2d array of characters visible on terminal window and each character have some additional properties such as color. There is no such thing as an "empty character" or "unexisting character" so ordinary space is used instead.

Even konsole in KDE 4.1.x worked fine (meaning not adding unnecessary spaces when copying in mcedit). I'm guessing that konsole simply deleted all the spaces at the end of lines while copying to clipboard. However around 4.2 release there was some change triggering this solution to be non-working. For example in PlainTextDecoder::decodeLine there was code to remove spaces from the end of lines, but when used for copying to clipboard this funcionality is disabled thanks to _includeTrailingWhitespace = true (see my comment #107). There were some attemps to restore space-on-end-of-line filtering, but mostly done in wrong way or on a wrong place (see attachment "mc screenshot with the patch attached").

However I believe that simply deleting all the spaces at the end of line is not an ideal solution. What if that space actually should be there because it is written in the file? See attachment "A file that can be used to test correctness of terminal windows" not paying attention to tab issue (yet). So I've proposed possible solution in comment #94 for testing(!) which distinguishes between spaces written by application and spaces written by clearImage not copying the latter into the clipboard and Jekyll Wu submitted cleanly written patch based on that idea into git. Actually I've expected more feedback before actually submitting this idea, but well...

Latter it occured, that this approach is still not sufficient in some more subtle situations. To me its not completly clear what these situautions are. I have noticed problems when connected over ssh to a server with very old version of mc, but locally on current mc problem was nonexistent. I have studied stream of commands coming from that old mcedit and it occured that the old mc actually prints some nonexisting trailing whitespace! Please see my comment 118.

Opinions may vary and we need more feedback. It can be viewed as a bug in my idea arguing that mcview/mcedit bahavior is correct saving bandwidth and only rewriting parts of screen which needs to be visually different without considering what is actually in screen buffer and what would be eventually copied into the clipboard. And that it's a konsole resposibility to blindly delete all trailing spaces whetever they are real or not. Or it can be viewed as mcedit/mcview bug that it should print real and exact contents of viewed or edited files with exact indentation (spaces vs. tabs) instead of simply moving cursor and considering only as things appear on screen without considering for example what indentation will be copied into clipboard.

I'm still thinking that it would be better if mc developers change it's behavior (or it is a behavior of ncurses or slang?). Consider a source code file with tab-based indentation. Now when you open such a file in mcedit and copy contents into clipboard (both in konsole and in xterm) all the indentation will be changed into spaces instead of tabs. That's another result of mc behavior of only moving cursor instead of printing indentation.

It would be a great help to investigate, who is responsible for this behavior. Mc? Slang? Ncurses? Could they be fixed? Is there other applications beside mcedit which have similar problems too? But please note, that its not allways easy to make a conclusion. See comment 125. I belive, that this problem that disapeared on subsequent run is caused by some invisible garbage left on the screen by previous application. Good way to visualize whats going on is to change c into something visible in src/Screen.cpp on line 842.

Please note I'm not a konsole developer. I'm just long time user who tried to understand a problem and proposed some solution instead of blaming developers. Richlv, if you want checkbox in configuration to delete all trailing spaces, then fine, write a patch.

> Running applications are issuing commands to konsole using VT102
> protocol. In the most simplest mode of operation it is a stream of
> characters to be written on the screen [...]. However applications like
> mcedit makes greater use of VT102 - they issue commands to move
> cursor around the screen or erase portions of screen.

As someone with an intimate familiarity with VT / ANSI character cell
terminals, writing applications and screen handling libraries under VMS
more than twenty years ago, I feel qualified to comment on the following  ;-}

> Opinions may vary and we need more feedback. It can be viewed as a bug
> in my idea arguing that mcview/mcedit bahavior is correct saving bandwidth
> and only rewriting parts of screen which needs to be visually different [...]
> Or it can be viewed as mcedit/mcview bug that it should print real and exact
> contents of viewed or edited files with exact indentation (spaces vs. tabs)
> instead of simply moving cursor [...]
> I'm still thinking that it would be better if mc developers change it's behavior

Considering that an escape sequence to erase or to reposition a cursor
can be several characters long, it used to be very common for example
to write a couple of spaces in order to erase some previous contents of
a screen, or to reposition a cursor by a few positions by re-writing the
same contents.

One cannot assume that the (insert here an application or a screen library of
your choice) to be the only application behaving this way. It is not realistic to
expect existing applications to adapt to some peculiarity of a (future) VT terminal
emulator, so I think it is only realistic to concede and trim trailing space when
copying to a clipboard.

(In reply to comment #138)

> It would be a great help to investigate, who is responsible for this
> behavior. Mc? Slang? Ncurses? Could they be fixed? Is there other
> applications beside mcedit which have similar problems too?

Yes, I have the problem with using 'cat' on the command line. Only reason I noticed it is when generating CSR's with openssl. Extra spaces are added to the end of  certain lines of the CSR after a simple 'cat'.

e.g.
openssl req -new -newkey rsa:2048 -nodes -keyout hostkey.pem -out csr.pem
cat csr.pem

I need to copy and paste the CSR into a web form and the extra spaces render it invalid. The extra spaces don't exist in the file. It is also not reproducible 100% of the time, but enough of the time to be a nuisance. I switch to xterm and it works as expected.

(In reply to comment #138)
> Even konsole in KDE 4.1.x worked fine (meaning not adding unnecessary spaces
> when copying in mcedit). I'm guessing that konsole simply deleted all the
> spaces at the end of lines while copying to clipboard. 
> 
> However I believe that simply deleting all the spaces at the end of line is
> not an ideal solution.

But this is exactly the point that richlv@nakts.net was trying to make:  Yes, it's a bug to add extra spaces to the end of the line, and it's also a bug to delete all spaces from the end of the line.  But given the choice between the two bugs, the latter is far easier to live with than the former, since it's very rare that whitespace at the end of a line is significant.  Ideally konsole should exhibit neither bug, but evidently figuring out how to do so is difficult.

So WHILE we are all spending years discussing how to make konsole behave perfectly, can we please simply revert to the older and less annoying buggy behaviour?

Created attachment 73571 [details]
New profile option "Trim trailing spaces"

OK, I've added a new profile property "Trim trailing spaces" (checkbox on Advanced tab).

You can find the patches here: https://build.opensuse.org/package/files?package=konsole&project=home%3Adavid-smid

kde4-kdebase-bug-188528.patch is applicable against 4.8.5 (tested)

kde4-kdebase-bug-188528-4.9.0.patch is applicable against 4.9.0 (not tested!)

kde4-kdebase-bug-188528-HEAD.patch is applicable against HEAD (not tested!)

OpenSUSE RPMs for 4.8.5 are here: https://build.opensuse.org/package/binaries?package=konsole&project=home%3Adavid-smid&repository=openSUSE_12.1

Created attachment 73575 [details]
New profile option "Trim trailing spaces"  #2

Fixed compilation errors.
Still not tested (HEAD).

The 4.9.0 patch is now fixed and tested as well.

OpenSUSE RPMs can be found here: https://build.opensuse.org/package/binaries?package=konsole&project=home%3Adavid-smid%3Akde-4.9.0&repository=openSUSE_12.1

Created attachment 73576 [details]
New profile option "Trim trailing spaces"  #3

Fixed more errors.

This patch (kde4-kdebase-bug-188528-HEAD.patch) is now tested as well. It works OK.
Forgot to mention that from 4.9.0 the checkbox is on the "Mouse" tab.

OpenSUSE RPMs are here: https://build.opensuse.org/package/binaries?package=konsole&project=home%3Adavid-smid%3Akde-unstable&repository=openSUSE_12.1

(In reply to comment #140)
> I need to copy and paste the CSR into a web form and the extra spaces render
> it invalid. The extra spaces don't exist in the file. It is also not
> reproducible 100% of the time, but enough of the time to be a nuisance. I
> switch to xterm and it works as expected.

I find xclip ( http://sourceforge.net/projects/xclip/ ) useful for this kind of thing. Of course, it doesn't necessarily help when you realize you want to copy something that's already on the screen.

Created attachment 73578 [details]
konsole-cat-csr-pem-problem.patch

> Yes, I have the problem with using 'cat' on the command line. Only reason I noticed it is when generating CSR's with openssl. Extra spaces are added to the end of certain lines of the CSR after a simple 'cat'. [...]

Marcus, thank you for providing this use case. I was able to replicate this bug and then it wasn't hard to solve the problem. Screen buffer resize (enlargement) wasn't handled properly. Screen buffer is initially created at the size 80x40 characters, but typical konsole window is larger. So this problem with simple cat command manifested itself just in the space between line 41 of console till last line of visible screen. Later on, when konsole scrolls and new lines are added it was ok again. Shortly: bottom lines of newly opened console and in new space after konsole enlargement.

> So WHILE we are all spending years discussing how to make konsole behave
> perfectly, can we please simply revert to the older and less annoying buggy
> behaviour?

This would be so nice please consider this advice!

I'll also second the comments that trailing white space on lines may not be "wanted" but often added for convenience either to pad lines or erase characters.  My guess is that a lot of people will be happy with "no trailing white space" and very few will complain about "it isn't exactly as printed". I wouldn't object to an option.

(In reply to comment #146)
> Created attachment 73576 [details]
> New profile option "Trim trailing spaces"  #3

Is there any chance this feature will appear in KDE 4.10 ?
Is anything wrong with the patch ?

#150 if it helps people  I'm not against putting it in for 4.10.

Obviously it would be nice to fix the real issue but no one has had the time/inclination to do so.

apparently it is also impossible to fix in all cases - see the comments about problems with screen buffers and different ways different ncurses apps repaint screen.

please, provide a "hammer" solution for most users, three and a half years are enough to validate it from user experience point of view ;)

I think it has been shown that there is no correct and robust general solution for this problem. So instead of waiting for one, here is another vote for "please just give us something that works in 95% of all cases".

I appreciate that as good maintainers you'd like to resist feature and option creep. But I think this option should go in.

Git commit ca9dd11bf5e1b3bf5c6c17de34d5d3bf4d3baf38 by Kurt Hindenburg.
Committed on 10/09/2012 at 01:03.
Pushed by hindenburg into branch 'master'.

Allow option to trim spaces at end of line

In the ongoing issue with spaces at the end of lines, this allows for
an option to trim the spaces.  By default, it is disabled so only
people who need this will be affected.

Many thanks to David Smid <david@smidovi.eu>

M  +8    -0    src/EditProfileDialog.cpp
M  +1    -0    src/EditProfileDialog.h
M  +10   -0    src/EditProfileDialog.ui
M  +2    -0    src/Profile.cpp
M  +2    -0    src/Profile.h
M  +27   -10   src/Screen.cpp
M  +13   -7    src/Screen.h
M  +2    -2    src/ScreenWindow.cpp
M  +2    -1    src/ScreenWindow.h
M  +3    -2    src/TerminalDisplay.cpp
M  +16   -0    src/TerminalDisplay.h
M  +1    -0    src/ViewManager.cpp

http://commits.kde.org/konsole/ca9dd11bf5e1b3bf5c6c17de34d5d3bf4d3baf38

if anybody else is looking for this option, it's in profile settings, "mouse" tab and it's "trim trailing spaces" in the "copy & paste" section

Since this is set per profile, is there any way to make it the default via an /etc/kde/konsolerc or similar file?

Thanks for the fix --any chance this will be back ported? I'm on 4.9.5

No, at least not here in (upstream) KDE, the 4.9 branch is no longer maintained and no further 4.9.x release is planned. Please ask the packaging team whose 4.9.5 packages you're using (or if you built it yourself, just apply the patch).

Ok - I've moved to 4.10 since the last message so moot for me.

Thanks- this is a much appreciated fix!

Shouldn't this bug be marked as FIXED now?

No, please don't as the bug is really not fixed. We have a workaround now, so this bug is no longer that annoying for users, but the scope of this bug is much wider and still unsolved.

First still nobody applied my "konsole-cat-csr-pem-problem.patch" which is a fix for a problem in initially opened konsole window as pointed out by comment #140. My idea (reimplemented by Jekyll Wu in comment #108) is a possible first step into fully solving problems demonstrated in "A file that can be used to test correctness of terminal windows".

But we bumped into another problem that mcedit didn't paint its window in a way allowing for correct copying into a clipboard. I've explained this throughfully in comment #138 and Mark Martinec disputed my opinion in the next comment.

While I understand his opinion in general way, my point is little less general and little more subtle: Since mcedit is a text editor application, copying text from an edited file into clipboard is quite frequent use-case. Shouldn't mcedit developers be aware that their users possibly want to preserve formating while copying text (ie. source code) from a text editor and sacrifice (possibly optionaly) some painting optimizations to allow correct indentation in copied text? Since there is no other way to preserve copied text exactly without text editor developers painting it correctly into a terminal in first place.

So for me it would be interesting to know mcedit developer's opinion on this.

I would argue that this particular bug report has outgrown being useful.  I would suggest new reports be opened for any specific problem cases.  My opinion anyway.

Even with the 'Trip trailing spaces' option enabled, konsole part (in yakuake in my case) is still adding trailing spaces at the end of some lines. Typically it's in the middle of a long cut&pasted text.

I just struggling with the same problem, Konsole is adding extra spaces where there is non to begin with. It's just annoying that i copy/paste text from one file to another on server using Konsole and i need to remove spaces.

Selecting "Cut spaces on end" didn't help

Cannot reproduce with Konsole 22.04.0. Can you still reproduce it?

Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!

This bug has been in NEEDSINFO status with no change for at least
30 days. The bug is now closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

Thank you for helping us make KDE software even better for everyone!