Version: (using KDE KDE 3.2.3) Installed from: Unspecified Linux OS: Linux When you type any interactive command in the console (e.g. rm), you will be faced with a question such as: rm: remove regular file `filename'? Now, if you switch to Arabic and type some Arabic letters then BACKSPACE to delete them, you will delete letters from the question itself which is equal to the number of the typed Arabic letters. Looks as if something concerned with Arabic being double byte and the UTF-8 encoding.
Does the arabic run from right to left? That's not handled very well at the moment.
Yes, it runs from right to left. This problem is also in konsole, xterm, mlterm, gnome-terminal. Pablo Saratxaga of Mandrake on http://qa.mandrakesoft.com/show_bug.cgi?id=5645 mentioned: "It is probably tied to libreadline (it isn't related to terminals)" Can any one confirm? Where do we need to file this bug then?
bug-readline@gnu.org according to http://cnswww.cns.cwru.edu/php/chet/readline/rltop.html
I received a response from Mr. Chet Ramey, the maintainer of the GNU Readline library. He told me: "The message is coming from `rm'. I don't have anything to do with the distributions in question, or the `rm' source. It seems to me that the burden of proving that readline is at fault is the distributor -- maybe by demonstrating that `rm' is linked with libreadline and that it is calling readline() when it is getting its response from the terminal. Simply asserting that readline is responsible is insufficient." Please change the status of this bug to REOPENED
KDE does not make distributions and rm is not a KDE applications so there is little point in reopening this bugreport here.
Mr. Waldo, it's not a matter of the rm command only. It's a matter of rm, cp, mv, ... and all commands that ask an interactive questions on konsole. This is why I believe it's konsole bug not a command bug, am I still wrong?
Could this be related to wcwidth again? Although I don't know how arabic terminals are supposed to work, but the symptoms described in the original report remind me of the double-width/single-width cjk bug. Does it happen also if say, at a prompt you type some characters, and then delete them.. does the prompt string get partially deleted as well?
It's unlikely to be a konsole bug if the problem also happens with xterm.
oh right, didn't see that, sorry
>Does it happen also if say, at a prompt you type some characters, and then >delete them.. does the prompt string get partially deleted as well? No. I also want to add that launching konsole without without utf support: LC_ALL=en_US.ISO-8859-1 konsole won't produce this bug. Dosn't this mean the bug is in konsole handling of unicode characters? Can't konsole and xterm have the same bug though?
Another example if I want to install a package with urpmi and it has a dependancy, I will be asked Is this OK? (Y/n) If I typed Arabic text and pressed BACKSPACE part or all of the question will be erased. I hope you can, at least, guide me to which lib this bug belongs.
>Does it happen also if say, at a prompt you type some characters, and then >delete them.. does the prompt string get partially deleted as well? Yes! it happens sometimes in konsole (but not xterm ;) ) That's if I kept pressing any English letter (say k) while switching the keyboard layout to Arabic (using Alt+Shift for example) and English again many times. The prompt will be deleted. Actually the cursor goes to the left then up in a very strange way. I meant to say do something like this: $kkkkkkنننننننننننننkkkkkkkkkkنننننننننننننkkkkkkkkk and see this easter-egg in konsole only today ;)
I will open it again since it's now a konsole-specific bug.
Another way to reproduce this strange bug is: - Open konsole - Press Enter many times (just to make things look clear) - Switch to Arabic - Keep pressing any Arabic letter (say the one on the J key), press Alt and release it. - Don't forget to still keep pressing the Arabic letter until it reaches the edge of the screen. You will see how strangely the cursor moves in a wild manner.
Hi, I don't understand Arabic at all, but I was able to reproduce your behaviour.. Waldo, maybe you can try this: * Enable arabic keyboard in control center * run a konsole in en_US.UTF-8 locale * set locale to en_US.UTF-8 in case your shell overrides it * turn on arabic keyboard * start hitting "alt+j".. actually alt and any key. If you keep hitting that combination, the cursor starts jumping backwards. What I don't know is.. is the alt+key combination mean something specific for arabic input? If you set it on the english keyboard, it doesn't do anything.
no, Alt+ArabicKey doesn't mean any thing specific for Arabic input unless it's defined in xorg keymap to mean something which is not the case.
> ------- Additional Comments From munzirtaha newhorizons com sa > 2004-06-26 18:26 ------- no, Alt+ArabicKey doesn't mean any thing > specific for Arabic input unless it's defined in xorg keymap to mean > something which is not the case. > Well just to make sure, could you set it on english keyboard and type alt+j and verify that nothing happens
Ken, first thanks to the way you reproduced the bug. I just checked it now. It's easier and more direct than my steps. Second, you are right of course regarding alt+j in an english keyboard won't cause any probem.
I guess this has nothing to do with arabic. I could even reproduce it with Hungarian which is a Latin left-to-write language. The bug arises in cooked tty mode whereas the kernel(?) has to be told that UTF-8 is used so that the terminal driver knows to remove more bytes from the buffer when a backspace is pressed. How to reproduce No1: bash$ cat type for example xyzáé, press one or two backspaces and then Enter. The line echoed back clearly shows that each backspace removes only one byte and not a whole multibyte character. How to reproduce No2: bash$ echo -n 'someprompt>' ; cat now type some non-ascii characters (accented letters, euro sign etc.) and then press backspace many times. Backspace erases one character as many times as many bytes (not characters!) were typed at the prompt. AFAIK the kernel maintains a bit for each terminal telling whether that one is in UTF-8 mode. The problem is that konsole doesn't set this bit. There's a patch to stty here: ftp://ftp.ilog.fr/pub/Users/haible/utf8/ and it's also there inside SUSE 9.2 coreutils source rpm. If you apply this patch then "stty -a" shows "-iutf8" which means input utf8 mode is turned off. Type "stty iutf8" to turn it on. The bugs discussed above will disappear then. Please check http://www.cl.cam.ac.uk/~mgk25/unicode.html (Markus Kunh's famous UTF-8 FAQ), read the paragraph from "Any Unix-style kernel"... It says only Linux kernels >= 2.6 have this iutf8 feature, so there may be problems with other kernels and other Unix systems. So IMHO what konsole should do is that #ifdef IUTF8 then try to perform the relevant tcsetattr() (either try to explicitely set or explicitely clear this flag since no-one knows when the kernel's default will change to utf-8 mode). Also please note that this iutf8 flag should also be changed if utf-8 mode is changed run-time in konsole (e.g. start an 8-bit terminal and then print \e%G and \e%@).
Or, another possible workaround, if you cannot set the terminal to utf-8 mode is to make an attempt to set it to non-utf8 mode, and then workaround all these stuffs assuming non-utf8 mode. That is, when I press backspace, only move one character to the left on the screen, but pretend to the real terminal driver that backspace was hit more times. This solution, however, I'm afraid might lead to unpredictable problems over remote ssh or telnet connections, so I'd not recommend this for systems that natively support utf8 input in their tty drivers, such as linux 2.6.
xterm suffers from the same problem. I've reported this to the x.org folks too: http://freedesktop.org/bugzilla/show_bug.cgi?id=1841
CVS commit by waba: Add support for IUTF8 BUG: 83236 M +5 -0 TEPty.cpp 1.88 M +1 -0 TEPty.h 1.29 M +3 -5 TEmulation.cpp 1.61 M +3 -0 TEmulation.h 1.35 M +13 -5 konsole.cpp 1.502 M +2 -0 session.cpp 1.100 --- kdebase/konsole/konsole/TEPty.cpp #1.87:1.88 @@ -94,4 +94,9 @@ void TEPty::setXonXoff(bool on) } +void TEPty::useUtf8(bool on) +{ + pty()->setUtf8Mode(on); +} + /*! start the client program. --- kdebase/konsole/konsole/TEPty.h #1.28:1.29 @@ -49,4 +49,5 @@ Q_OBJECT public slots: + void useUtf8(bool on); void lockPty(bool lock); void send_bytes(const char* s, int len); --- kdebase/konsole/konsole/TEmulation.cpp #1.60:1.61 @@ -206,13 +206,11 @@ void TEmulation::setCodec(const QTextCod delete decoder; decoder = m_codec->makeDecoder(); + emit useUtf8(utf8()); } void TEmulation::setCodec(int c) { - //FIXME: check whether we have to free m_codec - m_codec = c ? QTextCodec::codecForName("utf8") - : QTextCodec::codecForLocale(); - delete decoder; - decoder = m_codec->makeDecoder(); + setCodec(c ? QTextCodec::codecForName("utf8") + : QTextCodec::codecForLocale()); } --- kdebase/konsole/konsole/TEmulation.h #1.34:1.35 @@ -65,4 +65,5 @@ signals: void lockPty(bool); + void useUtf8(bool); void sndBlock(const char* txt,int len); void ImageSizeChanged(int lines, int columns); @@ -84,4 +85,6 @@ public: bool isConnected() { return connected; } + bool utf8() { return m_codec->mibEnum() == 106; } + virtual void setListenToKeyPress(bool l); void setColumns(int columns); --- kdebase/konsole/konsole/konsole.cpp #1.501:1.502 @@ -843,7 +843,14 @@ void Konsole::slotSetEncoding() if (!se) return; + QTextCodec * qtc; + if (selectSetEncoding->currentItem() == 0) + { + qtc = QTextCodec::codecForLocale(); + } + else + { bool found; QString enc = KGlobal::charsets()->encodingForName(selectSetEncoding->currentText()); - QTextCodec * qtc = KGlobal::charsets()->codecForName(enc, found); + qtc = KGlobal::charsets()->codecForName(enc, found); if(!found) { @@ -851,4 +858,5 @@ void Konsole::slotSetEncoding() qtc = QTextCodec::codecForLocale(); } + } se->setEncodingNo(selectSetEncoding->currentItem()); --- kdebase/konsole/konsole/session.cpp #1.99:1.100 @@ -71,4 +71,5 @@ TESession::TESession(TEWidget* _te, cons //kdDebug(1211)<<"TESession ctor() sh->setSize()"<<endl; sh->setSize(te->Lines(),te->Columns()); // not absolutely nessesary + sh->useUtf8(em->utf8()); //kdDebug(1211)<<"TESession ctor() connecting"<<endl; connect( sh,SIGNAL(block_in(const char*,int)),this,SLOT(onRcvBlock(const char*,int)) ); @@ -76,4 +77,5 @@ TESession::TESession(TEWidget* _te, cons connect( em,SIGNAL(sndBlock(const char*,int)),sh,SLOT(send_bytes(const char*,int)) ); connect( em,SIGNAL(lockPty(bool)),sh,SLOT(lockPty(bool)) ); + connect( em,SIGNAL(useUtf8(bool)),sh,SLOT(useUtf8(bool)) ); connect( em, SIGNAL( changeTitle( int, const QString & ) ),
Waldo Bastian, Lots of thanks and keep up the good work! I am Dreaming of the day that all the bugs at http://wiki.arabeyes.org/OpenBugs are fixed.
especially that GNOME bugs seems to be far less, would this mean something? ;)