Bug 451716 - Hindi font display issues: halant cut too short, horizontal line break
Summary: Hindi font display issues: halant cut too short, horizontal line break
Status: REPORTED
Alias: None
Product: konsole
Classification: Applications
Component: font (show other bugs)
Version: 19.12.3
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Konsole Developer
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-03-20 08:18 UTC by Dotan Cohen
Modified: 2022-12-21 14:02 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
namaste using Noto Sans Devanagari (25.39 KB, image/png)
2022-03-20 12:57 UTC, ninjalj
Details
80 columns of namaste with CTL (29.97 KB, image/png)
2022-03-20 15:31 UTC, ninjalj
Details
80 columns of namaste with no CTL (30.11 KB, image/png)
2022-03-20 15:32 UTC, ninjalj
Details
mlterm -V output for comparison (23.50 KB, image/png)
2022-03-20 15:32 UTC, ninjalj
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dotan Cohen 2022-03-20 08:18:05 UTC
The text "नमस्ते" is displayed in Konsole as such:
https://imgur.com/a/xYjUQRv

STEPS TO REPRODUCE
1. echo "नमस्ते"
2. Observe the output

OBSERVED RESULT
1. The halant seems to be cut too short, perhaps because it's exceeding the terminal's line height.
2. The horizontal line between the n and m has a tiny break, which it shouldn't. 

EXPECTED RESULT
Properly-formatted Hindi text

SOFTWARE/OS VERSIONS
Operating System: Kubuntu 20.04
KDE Plasma Version: 5.18.8
KDE Frameworks Version: 5.68.0
Qt Version: 5.12.8
Kernel Version: 5.13.0-35-generic
OS Type: 64-bit
Processors: 4 × Intel® Core™ i7-6700 CPU @ 3.40GHz
Memory: 15.5 GiB of RAM
Comment 1 ninjalj 2022-03-20 12:57:59 UTC
Created attachment 147617 [details]
namaste using Noto Sans Devanagari

Result with what I think is Noto Sans Devanagari.
Comment 2 ninjalj 2022-03-20 15:31:00 UTC
Which font are you using? Using what I think is Noto Sans Devanagari, I get the result in the attachment above. If you are using fontconfig (typical on Linux), a way to get a list of fonts containing a given codepoint is:

fc-list :charset=928

> 1. The halant seems to be cut too short, perhaps because it's exceeding the terminal's line height.

While konsole has some special case code for rendering glyphs which exceed its cell to the right (particularly when followed by a space), it has no provisions for rendering glyphs exceeding the cell's height.

OT: it seems like the usual way to write namaste is with a half-consonant SA, instead of with SA+halant.

> 2. The horizontal line between the n and m has a tiny break, which it shouldn't. 

This depends on font, font size and a konsole profile setting in "Advanced→Bi-directional text rendering" (which should be really called something along the lines of "complex text layout").

When enabled, the profile setting makes konsole send whole runs of text of the same script and attributes (instead of isolated characters) to Qt's renderer, to make sure it can render right-to-left text and complex scripts. With this you shouldn't see the gaps, but the width of runs of text with not perfectly monospaced fonts may be quite off, causing issues.

In the absence of perfectly monospaced fonts, there appears an inherent trade-off between proper rendering (e.g. with no gaps in Indic scripts) and correct cell-aligned width. mlterm has a -V option which enables variable width columns, but I haven't had much success trying it (I still see gaps, and in any case mlterm seems to put the halant and the vowel E about one cell before they should be, at least on my system with my mlterm configuration). 

You can compare the screenshots below: namaste-80-ctl.png (on, no gaps, width off) and namaste-80-no-ctl.png (off, gaps visible between instances of "namaste", width ok).

As for font and font size, YMMV. e.g. Noto Sans Devanagari seems to combine well with about one of every two sizes of Hack on my system.
Comment 3 ninjalj 2022-03-20 15:31:48 UTC
Created attachment 147627 [details]
80 columns of namaste with CTL
Comment 4 ninjalj 2022-03-20 15:32:17 UTC
Created attachment 147628 [details]
80 columns of namaste with no CTL
Comment 5 ninjalj 2022-03-20 15:32:46 UTC
Created attachment 147629 [details]
mlterm -V output for comparison
Comment 6 Bug Janitor Service 2022-08-06 18:51:58 UTC
A possibly relevant merge request was started @ https://invent.kde.org/utilities/konsole/-/merge_requests/722
Comment 7 Kurt Hindenburg 2022-08-27 01:11:04 UTC
Git commit 76f879cd70fb494ab2334d2660b34679546f3d9d by Kurt Hindenburg, on behalf of Matan Ziv-Av.
Committed on 26/08/2022 at 19:24.
Pushed by hindenburg into branch 'master'.

Draw characters in exact positions

QT can't be made to draw monospaced text (if the font does not cooperate),
so avoid combining characters, using a QPainter::drawText() call for each
character.

For bidi text support this change requires konsole to reorder and reshape
the characters. This is done using the ICU library (which QT also uses).

This change allows for some improvements related to text rendering:

- More precise bidi reordering, which is no longer changed by characters'
  attributes and selection.
- underlines drawn separately from the text, allowing for differing
  underline modes (double, curly, dashed, dotted, colored).
- Overriding font for emoji characters.

This commit fixes a few bugs and addresses a lot more:

Feature requests: More standard conforming RTL and various underlines:
Related: bug 403729, bug 387811, bug 416508, bug 452087, bug 425973, bug 430822, bug 442742, bug 441037, bug 430822, bug 440070, bug 450017, bug 445846, bug 453086, bug 381593

Using non-monospace font:



Emoji:

Regression: devanagari rendering

M  +2    -0    CMakeLists.txt
M  +1    -0    src/CMakeLists.txt
M  +47   -25   src/FontDialog.cpp
M  +2    -1    src/FontDialog.h
M  +127  -23   src/Screen.cpp
M  +18   -5    src/Screen.h
M  +28   -13   src/Vt102Emulation.cpp
M  +0    -29   src/autotests/CharacterTest.cpp
M  +0    -1    src/autotests/CharacterTest.h
M  +1    -1    src/autotests/TerminalCharacterDecoderTest.cpp
M  +119  -29   src/characters/Character.h
M  +1    -1    src/characters/Hangul.cpp
M  +5    -4    src/decoders/HTMLDecoder.cpp
M  +1    -1    src/decoders/PlainTextDecoder.cpp
M  +3    -0    src/profile/Profile.cpp
M  +29   -0    src/profile/Profile.h
M  +101  -6    src/terminalDisplay/TerminalDisplay.cpp
M  +7    -0    src/terminalDisplay/TerminalDisplay.h
M  +42   -0    src/terminalDisplay/TerminalFonts.cpp
M  +12   -0    src/terminalDisplay/TerminalFonts.h
M  +542  -269  src/terminalDisplay/TerminalPainter.cpp
M  +40   -13   src/terminalDisplay/TerminalPainter.h
M  +2    -0    src/widgets/EditProfileAdvancedPage.ui
M  +113  -3    src/widgets/EditProfileAppearancePage.ui
M  +61   -3    src/widgets/EditProfileDialog.cpp
M  +6    -1    src/widgets/EditProfileDialog.h

https://invent.kde.org/utilities/konsole/commit/76f879cd70fb494ab2334d2660b34679546f3d9d