Bug 371687 - Unicode characters appear cut off and mess up text height for rest of line
Summary: Unicode characters appear cut off and mess up text height for rest of line
Status: RESOLVED FIXED
Alias: None
Product: konsole
Classification: Applications
Component: font (show other bugs)
Version: 15.12.3
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Konsole Developer
URL: http://unix.stackexchange.com/questio...
Keywords:
: 315325 323994 336832 364215 364507 369370 373333 380451 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-10-25 23:47 UTC by flarn2006
Modified: 2017-06-05 02:33 UTC (History)
17 users (show)

See Also:
Latest Commit:
Version Fixed In: 17.04
Sentry Crash Report:


Attachments
Sample image showing bug (7.24 KB, image/png)
2016-10-26 01:24 UTC, flarn2006
Details
Sample image without blank lines (3.37 KB, image/png)
2016-10-26 01:24 UTC, flarn2006
Details
Screenshot showing the effect in vim in line 54+55 (112.79 KB, image/png)
2017-01-11 18:13 UTC, Horst Schirmeier
Details

Note You need to log in before you can comment on or make changes to this bug.
Description flarn2006 2016-10-25 23:47:02 UTC
User-Agent:       Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0
Build Identifier: 15.12.3

Whenever Konsole needs to use Unicode characters that are a different height from the rest of the text (generally because they aren't available in the selected font, so it needs to get them from a different one) they don't appear correctly. Instead of being resized to fit in the line, the characters are simply cropped, so part of it isn't visible. The line height also isn't reset, so the rest of the line (even normal characters) is offset and doesn't line up vertically with the beginning of the line. This can lead to these characters being cut off as well.

Reproducible: Always

Steps to Reproduce:
1. Make sure Konsole's encoding is set to UTF-8.

2. Type 'uudecode' at the shell prompt in Konsole and press Enter.

3. Copy and paste the following text into Konsole, starting with (and including) the "begin" line and ending with (and including) "end".

begin 664 /dev/stdout
M"@H)54Y)0T]$12`]/3T]/3X@XX.]X+R\X+J(V83-G."ZB."\O>^^B2`\/3T]
-/3T@54Y)0T]$10H*"@``
`
end

4. Press Enter if necessary.

5. Note how the characters are cut off, and how the beginning and end of the line are not lined up.
Actual Results:  
https://i.stack.imgur.com/GkZdn.png (the very bottom of the image)

Expected Results:  
https://i.stack.imgur.com/ba35f.png (the very bottom of the image)

This image is from gnome-terminal, which displays text correctly.

Turning off bi-directional text rendering makes the remainder of the line shift up instead of down.

The font used in the example images (also the font I use regularly) is "PxPlus IBM VGA8", available from the following URL: http://int10h.org/oldschool-pc-fonts/fontlist/
Comment 1 Christoph Feck 2016-10-26 00:15:40 UTC
Can you show the screen shots where there is no blank line above the line with the Unicode characters?
Comment 2 flarn2006 2016-10-26 01:24:29 UTC
Created attachment 101788 [details]
Sample image showing bug

See the bottom line
Comment 3 flarn2006 2016-10-26 01:24:48 UTC
Created attachment 101789 [details]
Sample image without blank lines
Comment 4 flarn2006 2016-10-26 01:36:41 UTC
(In reply to Christoph Feck from comment #1)
> Can you show the screen shots where there is no blank line above the line
> with the Unicode characters?

Added to attachments
Comment 5 Christoph Feck 2016-10-26 01:45:55 UTC
Also for GNOME Terminal? I am interested how GNOME developers managed it.
- shrink text size
- increase spacing between the line
- overlap with previous line, or
- truncate to not overlap previous line
Comment 6 flarn2006 2016-10-26 02:11:38 UTC
(In reply to Christoph Feck from comment #5)
> Also for GNOME Terminal? I am interested how GNOME developers managed it.
> - shrink text size
> - increase spacing between the line
> - overlap with previous line, or
> - truncate to not overlap previous line

Here's what it looks like in gnome-terminal: http://i.imgur.com/kiWwYhx.png
Comment 7 Christoph Feck 2016-10-26 02:27:51 UTC
Thanks, so it uses third option. What's interesting is that it manages to render only the foreground color, even if terminals usually have both a foreground and a background color.
Comment 8 Egmont Koblinger 2016-10-27 21:13:19 UTC
Such overflows do happen in gnome-terminal (vte). It's not fully thought/worked out there, I guess it's rather accidental than by design, but it works most of the time. E.g. you can see drawing glitches when you highlight either the text line or the Unicode line with your mouse. It's also imperfect in some cases with plain English text, e.g. see https://bugzilla.gnome.org/show_bug.cgi?id=734740 which I spent a couple of hours on, but could not understand what was going on.

The background is painted on the cell according to the strict grid, not on the area actually covered by the glyph. It seems to me now that the backgrounds are all painted before the foregrounds, that's why you can see the foreground overflowing but not the background.

One thing that vte never does (while apparently konsole does) is moving a glyph somewhere away from its desired location because of other oversized glyps.
Comment 9 Christoph Feck 2016-12-07 19:18:38 UTC
Git commit 7e2f9d839f5af447a0fc7fd178dd5d8f58e489bb by Christoph Feck.
Committed on 07/12/2016 at 19:12.
Pushed by cfeck into branch 'master'.

Render text at primary font's baseline

This only addresses the vertical shift and cutoff of lower
parts of characters, but not horizontal shift (characters
moving to the left or right).

See the linked review request for details.

This commit is for the 17.04 version. If you can test master
builds, please add feedback.

REVIEW: 129281

M  +2    -10   src/TerminalDisplay.cpp

https://commits.kde.org/konsole/7e2f9d839f5af447a0fc7fd178dd5d8f58e489bb
Comment 10 Christoph Feck 2016-12-07 19:20:17 UTC
*** Bug 369370 has been marked as a duplicate of this bug. ***
Comment 11 Christoph Feck 2016-12-07 19:21:00 UTC
*** Bug 336832 has been marked as a duplicate of this bug. ***
Comment 12 Christoph Feck 2016-12-07 19:27:45 UTC
*** Bug 323994 has been marked as a duplicate of this bug. ***
Comment 13 Christoph Feck 2016-12-07 19:35:43 UTC
*** Bug 373333 has been marked as a duplicate of this bug. ***
Comment 14 Christoph Feck 2016-12-07 19:36:24 UTC
*** Bug 364507 has been marked as a duplicate of this bug. ***
Comment 15 Christoph Feck 2016-12-07 19:36:55 UTC
*** Bug 364215 has been marked as a duplicate of this bug. ***
Comment 16 Christoph Feck 2016-12-07 19:44:52 UTC
*** Bug 315325 has been marked as a duplicate of this bug. ***
Comment 17 Horst Schirmeier 2017-01-11 18:11:21 UTC
I built konsole from the current git master (v16.12.0-23-gb7e75cf) on Ubuntu 16.10 with most KDE packages from ppa:kubuntu-ppa/backports, and I still see vertical glyph cutoff/offsetting.  However, I may see a different bug, because I can reproduce it with plain ASCII characters.  The relevant part of my bugreport from the Ubuntu bug tracker (https://bugs.launchpad.net/ubuntu/+source/konsole/+bug/1654875):

Depending on the text-cursor position (e.g., in vim), glyphs in a complete text line in Konsole are cut off vertically. These characters need not be fancy Unicode ones, plain ASCII suffices.

Konsole font is the default one here (Oxygen Mono, 9.0), but due to my Hi-DPI display (14" with 2560x1440px) I set "Force fonts DPI" to 192 in KDE's Settings->Appearance->Font->Fonts.  Also reproducible with the "Hack" font instead of "Oxygen Mono", and/or disabling the fonts setting "Force fonts DPI".
Comment 18 Horst Schirmeier 2017-01-11 18:13:22 UTC
Created attachment 103355 [details]
Screenshot showing the effect in vim in line 54+55

How I created the screenshot: Create text file with sufficient number of lines; open with vim; move cursor down until I'm at the terminal's last line and beyond; move back up until cursor is about in the vertical middle of the terminal.
Comment 19 Christoph Feck 2017-01-11 18:30:23 UTC
That looks like an unrelated issue, because the cell grid is already computed wrongly. Are you using HiDPI scaling?
Comment 20 Christoph Feck 2017-01-11 18:33:14 UTC
See bug 373232.
Comment 21 Horst Schirmeier 2017-01-12 08:38:00 UTC
Yes, HiDPI scaling is at 1.4 here.  Thanks, I added comments to bug 373232.
Comment 22 Christoph Feck 2017-05-11 21:50:54 UTC
There was no feedback to comment #9, closing.
Comment 23 Christoph Feck 2017-06-05 02:33:55 UTC
*** Bug 380451 has been marked as a duplicate of this bug. ***