Bug 245334

$ kate --version
Qt: 4.6.3
KDE: 4.4.92 (KDE 4.4.92 (KDE 4.5 RC2))
Kate: 3.4.92

Ok. On a mixed (cp1251 + utf8) file, kate forces cp1251.

Default is utf8.
Auto-detected us cp1251.

Setting anything but utf8 works, setting utf8 sets cp1251.

Created attachment 49904 [details]
file to reproduce the bug

Can not open this file with utf-8 encoding with auto-detect enabled.

$ kate --version
Qt: 4.6.3
KDE: 4.4.95 (KDE 4.4.95 (KDE 4.5 >= 20100723))
Kate: 3.4.95


Affected.

SVN commit 1160327 by cullmann:

myself:
    debug helper output, show me if a loading try with an encoding failed
    the bug 245334 is invalid
    if you try to open a file which is not utf-8 == contains invalid characters, the encoding detection will kick in
    this is intended behaviour, else you can loose file content
    while i can understand, that people might want to enforce an encoding even if it is wrong, if they have multiple encodings in one file, which is btw. no

CCBUG: 245334



 M  +3 -1      katetextbuffer.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1160327

Ok. I can give you at least two use-cases.

1) It is a broken web page with mixed encodings. This happens sometimes, and when i need to copy (or find) something from utf-8 part of the page, i can't due to this behaviour.

2) The file is in cp1251. I want to save it with utf-8. The file is remote, but kate thanks to kio opens them perfectly, no need to copy the file to the home folder.
What can i do: open file, copy the text (ctrl-a, ctrl-c), select utf-8 encoding, select all text (ctrl-a), paste correct text (ctrl-v), save file.
With auto-detect not allowing the third step, i have to clear the file first, which requires more actions.

Anyway, this is confusing. If user selects utf-8, he wants to view this file in utf-8 (or save in utf-8 overriding all text), but the auto-detections thinks that it knows better what user wants than himself.

If I click on, say, cp1252 (or koi8-r), it applies cp1252 (or koi8-r). If i click on utf-8 (or ucs-2), it does not apply utf-8 (or ucs-2).

I understand, that one-byte encoding and multi-byte encodings are different, but it works somehow without autodetection.
This is a very confusing behaviour from a user point of view.

That it is confusing is perhaps correct, but it is the only way to protect people from killing their data. Setting the encoding from the menu is no difference for kate than trying to use the normal encoding set as standard. If it dosn't work, it will always fallback to auto-detection and if that doesn't work, try the fall-back encoding given in config.

I could change the behaviour of the "Encoding" menu, to not use this heuristic but really enforce the usage of the given encoding.
But that is a feature and no minimal change.

What if user wants to kill that data, as that part is not needed, but he wants to view the other one? I posted two use-cases here.

And, that was not the only way.

The old way is MUCH better: show user a warning (with a "no more" checkbox), and, maybe, block file saving (but allow to unblock it). That was a little annoyng, but at least reasonable.

Also, no text editor at all would more certainly not allow user to kill their data with it.

My opinion is that the list should look like this:
1) User-selected encoding - overrides everything. User might recive a warning (like in 4.4) if there are problems with multi-byte encodings.
2) Auto-detected encoding - overrides default. Auto-detection is optional, but turned on by default.
3) Default encoding - the system one.

When the text editor (used partially by developers) thinks that it is smarter than the user, it's just not right. Maximum that it should give is a warning. Like it was before, in KDE 4.4.

Also, it affects the KWrite.

> kwrite --version
Qt: 4.6.3
KDE: 4.5.00 (KDE 4.5.0)
KWrite: 4.5.00 (KDE 4.5.0)

Maybe this is made to prevent the users from killing their data, but the way how it is done is really awful. I guess any typical user who sees a program not setting just selected encoding would consider such behaviour as buggy; the encoding selection menu will seem to be not working. If such unexpected behaviour is intentional, then the program must give, for example, a message explaining the reason why the user selected encoding was not set.

*** This bug has been confirmed by popular vote. ***

Git commit f2bcfd4fae3120d55c45797eea005a5c147a152b by Christoph Cullmann.
Committed on 01/11/2012 at 15:55.
Pushed by cullmann into branch 'master'.

allow the user to hard set encoding via encoding menu
if the user chooses an encoding there, the triggered reload of the document will use exactly that one
if an error occurs (e.g. file not in that encoding, warning will be shown in editing area but file will be loaded as good as possible with this encoding)

M  +3    -3    part/buffer/katetextbuffer.cpp
M  +2    -1    part/buffer/katetextbuffer.h
M  +2    -2    part/document/katebuffer.cpp
M  +2    -1    part/document/katebuffer.h
M  +8    -2    part/document/katedocument.cpp
M  +9    -0    part/document/katedocument.h
M  +1    -1    part/tests/encoding/kateencodingtest.cpp
M  +2    -2    part/view/kateviewhelpers.cpp

http://commits.kde.org/kate/f2bcfd4fae3120d55c45797eea005a5c147a152b

Thanks! =)

Sorry that this has taken that long, just had to much other stuff to do :) Thanks for your concrete use-case description.