Bug 447830 - Reflow broken for strings without newline
Summary: Reflow broken for strings without newline
Status: RESOLVED FIXED
Alias: None
Product: konsole
Classification: Applications
Component: general (show other bugs)
Version: 21.12.0
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Konsole Developer
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-01-02 13:59 UTC by Eric Armbruster
Modified: 2022-02-10 21:36 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In: 22.04


Attachments
how it should be (75.52 KB, image/png)
2022-01-02 13:59 UTC, Eric Armbruster
Details
four characters missing (528.42 KB, image/png)
2022-01-02 14:00 UTC, Eric Armbruster
Details
third (95.74 KB, image/png)
2022-01-02 14:02 UTC, Eric Armbruster
Details
Line wrapping only works if line is terminated with a newline character (48.41 KB, image/png)
2022-01-02 18:23 UTC, youduda
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Eric Armbruster 2022-01-02 13:59:03 UTC
Created attachment 145052 [details]
how it should be

SUMMARY
Konsole reflow is broken for strings without newline

STEPS TO REPRODUCE
1.  echo -n eyJhbGciOiJIUzI1NiJ9.eyJyb2xlcyI6W3siYXV0aG9yaXR5IjoiRElTUEFUQ0hFUiJ9XSwidXNlci1pZCI6IjYxY2MxYjkyODgwYTRlN2E5OGRmNDY2NyIsInN1YiI6ImFzZS1kZWxpdmVyeUBtYWZsbzMyMS5kZSIsImlzcyI6ImFzZTMzcHJvamVjdCIsImlhdCI6MTY0MTEzMDg1NSwiZXhwIjoxNjQxMTQ4ODU1fQ.KCUJwq8HVDFFKQjoUgzJziaDImy6yZywD_a1TccMB-8

(this jwt token does not contain production data) (first.png)
2. Press Ctrl + Left Arrow-> some characters went missing  (second.png). Notice the last four characters are missing aswell.
3. Expand the window to the right again with the mouse -> all characters go missing (third.png)

If you echo with trailing newline the same string and test with steps from above again reflow works as expected.

OBSERVED RESULT
reflow swallows a bunch of chars

EXPECTED RESULT
reflow should not swallow chars


SOFTWARE/OS VERSIONS
Operating System: Arch Linux
KDE Plasma Version: 5.23.4
KDE Frameworks Version: 5.89.0
Qt Version: 5.15.2
Kernel Version: 5.15.12-arch1-1 (64-bit)
Graphics Platform: Wayland
Processors: 16 × AMD Ryzen 7 3700X 8-Core Processor
Memory: 15.6 GiB of RAM
Graphics Processor: AMD Radeon RX 5700 XT
Comment 1 Eric Armbruster 2022-01-02 14:00:07 UTC
Created attachment 145053 [details]
four characters missing

second attachment
Comment 2 Eric Armbruster 2022-01-02 14:02:09 UTC
Created attachment 145054 [details]
third

Sorry for the spam but I did not know how I could add multiple attachments otherwise.

 how it should be = first.png
 four characters missing = second.png
Comment 3 youduda 2022-01-02 18:23:42 UTC
Created attachment 145061 [details]
Line wrapping only works if line is terminated with a newline character

Can confirm this see the attached image.

I entered some text in the console (left) then resized the window (middle). The text is just simply cut-off instead of wrapped into multiple lines. When using echo without '-n' it works as expected (see the right). Tested with bash and zsh.
Comment 4 ninjalj 2022-01-10 01:21:52 UTC
This is probably bash occasionally clearing to the beginning of line when the window is resized (SIGWINCH). On my system, zsh forces a newline, and ksh clears to the beginning of line, but leaves the rest of the line until the end untouched, which helps to see  the real culprit.

Can you confirm whether you are using bash, and whether other shells show different behavior?
Comment 5 Eric Armbruster 2022-01-12 18:55:54 UTC
I have now tested it in both bash and zsh. For me the issue is only present in zsh.
Comment 6 ninjalj 2022-01-12 20:07:21 UTC
Can you try after the following command?

setopt PROMPT_SP PROMPT_CR

which tells zsh to preserve a partial line:


>        PROMPT_CR (+V) <D>
>               Print  a  carriage return just before printing a
>               prompt in the line editor.  This is  on  by  de‐
>               fault  as multi-line editing is only possible if
>               the editor knows where the start of the line ap‐
>               pears.
> 
>        PROMPT_SP <D>
>               Attempt  to preserve a partial line (i.e. a line
>               that did not end with a newline) that would oth‐
>               erwise  be  covered up by the command prompt due
>               to the PROMPT_CR option.   This  works  by  out‐
>               putting  some cursor-control characters, includ‐
>               ing a series of spaces,  that  should  make  the
>               terminal  wrap  to  the next line when a partial
>               line is present (note that this is only success‐
>               ful  if  your  terminal  has  automatic margins,
>               which is typical).
Comment 7 Eric Armbruster 2022-01-13 16:30:04 UTC
I still have the same outcome after applying those options.
Comment 8 ninjalj 2022-01-15 22:12:05 UTC
There is some special-casing in konsole for zsh (https://invent.kde.org/utilities/konsole/-/commit/25a24bf347a3138f7146a220621160fd069eaca7).  AFAICT, combined with  PROMPT_SP PROMPT_CR it will think a previous partial line is a command prompt, which is what youduda at comment 3 is seeing. I'm not sure there is a way to tell a prompt from a previous partial line.

Now, as to the losing characters issue, we'll have to investigate further.
Comment 10 Eric Armbruster 2022-01-31 21:51:12 UTC
Hi ninjalj,
thank you for putting so much effort into this. Sadly, setting PROMPT_EOL_MARK='\n' or PROMPT_EOL_MARK=\n does not seem to help. I tried reading a bit into those links, but they go a bit too deep into terminals for me.

I also decided to give kitty a quick try. I couldn't reproduce the issue with this terminal emulator.
Comment 11 Bug Janitor Service 2022-02-02 23:34:38 UTC
A possibly relevant merge request was started @ https://invent.kde.org/utilities/konsole/-/merge_requests/595
Comment 12 ninjalj 2022-02-03 00:06:58 UTC
That would have been:

PROMPT_EOL_MARK="
"

or

PROMPT_EOL_MARK=$'\n'

but unfortunately zsh prints it always when PROMPT_CR and PROMPT_SP are set, adding an extra newline if the last command executed ended in a newline.

On another order of things, may I ask if your configuration sends semantic prompt markers? They are typically sent from a precmd hook function (which can be listed with "add-zsh-hook -L", and the function shown with "functions <function_name>"), or printed as part of PS1.
Comment 13 Eric Armbruster 2022-02-05 12:18:54 UTC
Tried again, this time with PROMPT_EOL_MARK set as you suggested. This seems to have done the job. Reflow works as expected. That is already a nice workaround :)

Regarding your other question, this is what add-zsh-hook -L returns:

typeset -g -a preexec_functions=( omz_termsupport_preexec _zsh_highlight_preexec_hook )
typeset -g -a precmd_functions=( omz_termsupport_precmd vcs_info _z_precmd _zsh_highlight_main__precmd_hook _zsh_autosuggest_start )

and the following is my PS1

%{$oxide_limegreen%}%~%{$oxide_reset_color%} ${vcs_info_msg_0_}
%(?.%{%F{white}%}.%{$oxide_red%})%(!.#.❯)%{$oxide_reset_color%}
Comment 14 tcanabrava 2022-02-07 11:41:13 UTC
Git commit 8345f24ee9313e494c66fc973051c2fd751d4c87 by Tomaz  Canabrava, on behalf of Luis Javier Merino Morán.
Committed on 07/02/2022 at 11:18.
Pushed by tcanabrava into branch 'master'.

More robust reflow on zsh with semantic prompts

Improve the logic for avoiding doing reflow of zsh prompts (zsh will
repaint the prompt and command line) when there are semantic prompt
markers (OSC 133).

M  +10   -2    src/Screen.cpp
M  +6    -0    src/Vt102Emulation.cpp
M  +1    -0    src/characters/Character.h

https://invent.kde.org/utilities/konsole/commit/8345f24ee9313e494c66fc973051c2fd751d4c87