Bug 399615 - Mishandling \b at the right margin
Summary: Mishandling \b at the right margin
Status: RESOLVED NOT A BUG
Alias: None
Product: konsole
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Konsole Developer
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-10-10 14:25 UTC by Alexei Podtelezhnikov
Modified: 2021-10-22 07:32 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexei Podtelezhnikov 2018-10-10 14:25:37 UTC
Make sure your window is 80-column wide then

$ printf "%080d\b \n" 0
000000000000000000000000000000000000000000000000000000000000000000000000000000 0

Notice the 79th zero deleted instead of the 80th. According to C99,

\b (backspace) Moves the active position to the previous position on the current line. If the active position is at the initial position of a line, the behavior of the display device is unspecified.

The filled line hardly qualifies as "the initial position" so the behavior is wrong.

The bug also exists in xterm and xfce4-teminal. Gnome terminal behaves correctly.
Comment 1 Egmont Koblinger 2018-10-10 22:26:36 UTC
xfce4-terminal and gnome-terminal uses the very same terminal emulation widget (VTE), they must behave identically.

As far as VTE is concerned, see https://gitlab.gnome.org/GNOME/vte/issues/60 and https://bugzilla.gnome.org/show_bug.cgi?id=731155.

It would be great to know how some popular hardware emulators behaved. Not sure if this info can be found somewhere, e.g. on vt100.net. I'm almost sure they behaved like VTE and Konsole do now.

Carefully reading ECMA-48 would possibly also provide the right answer for the expected (documented) behavior.

C99 is not a reference for the behavior of terminal emulators.

In my opinion the current behavior makes no sense at all, I agree with you that it'd be better if the character at the 80th position was removed. But you can't just change the behavior against the standards and the assumptions of existing software, this would result in potentially plenty of apps breaking.
Comment 2 Egmont Koblinger 2018-10-10 22:38:50 UTC
I've tested all the independent terminal emulator implementations I have on my system.

The following 7 of them erase the 79th character: Linux console, xterm, VTE, konsole, mlterm, st (suckless), terminology.

The following 5 of them erase the 80th character: urxvt, pterm (PuTTY), Kitty, screen, tmux.

Sigh.
Comment 3 Martin Hostettler 2018-10-11 00:29:21 UTC
I think it's somewhat logical in the VTxxx model of cursor movement.
In the VTxxx terminals the cursor does not move past the right most column ever, but instead has a "pending wrap" flag that takes effect only on the next printed character unless it's cleared in between.

Then Backspace is only a alias for cursor left (CUB) which as all cursor movement commands clears the "pending wrap" flag before executing the movement. This results in the somewhat unintuitive result.

I think it's just a fact of terminal programming that the right-most column is deeply special. Some applications use " \b" after text to force wrapping. But in general software needs to know the width of the terminal to correctly handle output when reaching the end of the line. It might be unfortunate that the VTxxx cursor model results in one column less that behaves like the others. But i think it's best to stick to the backwards compatible model. Especially for terminals that claim TERM=xterm where xterm usually tries to be historically accurate to what the original DEC VTxxx terminals do.

I'm trying to document what terminal emulators currently do. See https://terminalguide.netlify.com/printing/ and https://terminalguide.netlify.com/seq/a_c0-h/ for my attempt at documenting this(still work in progress).

(@egmont: Thank for retesting different terminal emulators, i had to fix the implementation specific notes on urxvt in my documentation)
Comment 4 Alexei Podtelezhnikov 2018-10-11 00:51:13 UTC
If I use my keyboard to do the same I will get the 80th zero removed. 40% of modern emulators do the right thing. Old gnome terminal 2.9.3 was ok too.

Good luck with that backward compatibility head in the sand.
Comment 5 Alexei Podtelezhnikov 2018-10-11 04:53:57 UTC
Thomas Dickey pointed me to this:

https://github.com/mattiase/wraptest/blob/master/results.txt

It is just funny how inconsistent the BS handling is, no pun intended
Comment 6 Egmont Koblinger 2018-10-11 09:17:51 UTC
> Good luck with that backward compatibility head in the sand.

In my universe, keeping an eye on not breaking the look of ncurses apps doesn't qualify as putting one's head in the sand.

> If I use my keyboard to do the same I will get the 80th zero removed.

No. In cooked mode (e.g. "cat") you get the exact same behavior as with printf. In raw mode (e.g. "bash") you might press the backspace key, but the shell probably emits totally different sequences towards the emulator. (The cursor is already elsewhere after typing the last letter of a line.)

> 40% of modern emulators do the right thing. Old gnome terminal 2.9.3 was ok too.

"the right thing" and "ok" according to your (and my) personal taste. But also according to the terminal emulator specs?

Let's try to be more constructive, shall we?

There's an ncurses test app attached to the VTE bug I linked earlier (731155). Out of the 5 emulators I could try where printf works as you expect, pterm and kitty fail to render the desired string "50%".

What's interesting and surprising: the remaining three (urxvt, screen and tmux) behave at printf as you expect, while also rendering the ncurses app correctly. How do they do this?

A bit of investigation seems to reveal that they implement two possible modes:

- If the 80th character is printed with autowrapping being turned on (the default mode, "\e[?7h") then a subsequent backspace behaves in your preferred way. I think it just resets the "pending wrap" flag, but leaves the cursor at the 80th position.

- If the 80th character is printed with autowrapping being turned off ("\e[?7l", this is what ncurses switches to for this character) then a subsequent backspace moves back the cursor to column 79.

Not sure if this is well thought out intentional design (or intentional workaround for the crazy situation we're in) in these apps, or just an accidental side effect of not setting the "pending wrap" flag at all in no-autowrap mode. We could perhaps consult the source code of these software to figure this out.

Also not sure if this behavior is correct or not according to ECMA or any other standard.

I also cannot find this dual behavior mentioned in the link you posted in your latest comment.

urxvt, screen and tmux can backspace correctly across a linebreak in cooked mode (e.g. "cat"). Technically this requires this current issue to be addressed, plus reverse wraparound (at least across soft breaks). Maybe with this dual behavior these emulators found the way to implement this, while not breaking ncurses. Or maybe that's how it's all documented, and is exactly why ncurses switches off autowrapping for the last character.

I'd love to have further information on how much this is considered the designed correct behavior, or accidental correct behavior, or a terrible workaround hack, or what exactly.
Comment 7 Alexei Podtelezhnikov 2018-10-15 16:19:21 UTC
Ok. I removed \b from my program and learned to never use it again for tweaking the output. Apparently, anything beyond \n and \r is rather implementation-dependent.

Please close the bug unless predictable \b is worth something.
Comment 8 Justin Zobel 2020-11-13 06:37:41 UTC
As no other users have commented on this I'll mark it as resolved.
Comment 9 Bug Janitor Service 2021-10-21 18:07:23 UTC
A possibly relevant merge request was started @ https://invent.kde.org/utilities/konsole/-/merge_requests/510
Comment 10 tcanabrava 2021-10-22 07:32:05 UTC
Git commit c099e0f2ea0f25b923203d17409eee8bc2831bfd by Tomaz  Canabrava, on behalf of Luis Javier Merino Morán.
Committed on 22/10/2021 at 07:29.
Pushed by tcanabrava into branch 'master'.

Fix regression on cursor controls at last column

Commit 8de0e6c7 mistakenly removed code to cancel the pending wrap flag
(which in konsole is implemented as having the cursor one past the width
of the screen) before the cursor controls CUU, CUD, CUB and BS.

While the behavior may seem counter-intuitive (specially BS and CUB,
where `printf "A\bB"` at the last column results in `BA`), this is the
behavior of VT100, xterm (without reverse-wraparound mode), and, most
importantly, is expected by ncurses.

M  +4    -0    src/Screen.cpp

https://invent.kde.org/utilities/konsole/commit/c099e0f2ea0f25b923203d17409eee8bc2831bfd