Bug 165044

Created attachment 25631 [details]
Tar file with broken encoding.

*** Bug 167523 has been marked as a duplicate of this bug. ***

This bug also affects full unicode names. I have a folder shared in windows xp with only unicode file names and files with only unicode characters (eg. spanish accented names or chinese/japanese/german named files). Dolphin behaves crashing sometimes (fully randomly) or just refuse to copy/change name the file, even with write permissions in the xp machine.

Also happens with local files, thus making dolphin/konqueror/kde 4.1 fully unusable for international users that need unicode compatibility.

This bug forces me to install dolphin for KDE 3.5.x, which has not this problem. I have a Western Digital NAS that provides samba shares for accessing. For every file/folder with Spanish non-standard character, Dolphin shows this error message (translated from Spanish): "The file does not exist: smb://user@nas_ip/share_name/filename".

When I try to get the properties of a folder or file with this characteristics, Dolphin shows in the Permissions tab "forbidden" for user, group and others sets.

I also randomly get a Dolphin crash when browsing folders with this kind of filenames.

Is this related to bug 165400 ?

Probably.

Kde 4.1.1 comes with the bug included. Seems nobody cares.

J.

jorortega, I can perfectly use special characters (utf-8) system with Dolphin. Locally I mean, haven't tried with samba shares (it would be probably a related bug but on the smb:/ kioslave as well).

However, I can reproduce this issue when navigating with the tar:/ kioslave.

I also can say that trying to open this tar with ark from kde3 can't read those folders either.

Not being able to rename a local file, is a local problem. Hence this is not only related to some tar/smb ioslave.

Yeah the problem is not about utf-8, it's about strange/broken encodings that make apps act strange (not able to access folder, delete it, ...).

I don't think so... kde 3.5.9 is able to manage all those files/folders/directoriers/shares/whatever nicely. Ark works well, konqueror is ok.

kde 4.0 and later have the problem. Seems that something is broken. Any file/directory/share/local/etc that contains:

- accents (spanish)
- Russian characters (Cyrilic)
- Janapense/chinese/korean/hebrew characters

just won't open in dolphin neither konqueror under kde 4 (An error message that indicates that "file "something%20blahblah%20.someext" doesn't exist") in some cases the folder listing will just ignore some files. I'm testing kde 4.1.1 in archlinux and kde 4.1 in kubuntu. Both distros have the same problem. Under Kde 3.5.9 in another machine, the same files works just OK. (Even the files attached to this error report just open ok in kde 3.5.9, not in 4.0)

Regards.

J.

When I said "This bug forces me to install dolphin for KDE 3.5.x", I meant I installed dolphin 3.5.x on my KDE 4.1.1 instance. That is: I use KDE4 and Dolphin4, but when I need to access to a samba share, I must use Dolphin3 (running on KDE4).

I have no problems with local files and special (non-English) characters.

We will no longer support broken encodings in KDE. We have had transition code for several years (at least since 2003) and I think 5 years is enough time for people to finish transitioning to UTF-8 environments.

This bug is about broken-encoded files. *Properly* encoded filenames should be 
working and if they aren't, please open a new bug report on the subject.

You will hate me for this, but this bug is a WONTFIX. 5 years is enough time. If in 5 years you haven't renamed all your files, you should use the terminal to do it. 

More info on why this is close to impossible to implement: http://lists.kde.org/?l=kde-core-devel&m=122025063320264&w=2

Do *not* reopen the bug.

For information, WONTFIX here means more "CANNOTFIX" or "TOO_IMPRACTICAL_TO_FIX"

This is indeed working against users --> bad usability. No matter what the reason is, you screw the users, especially those with least experience and most likely files from others OSs.

Further, since renaming in konsole, i.e. mv "broken-name" "new name" works, makes your "close to impossible to fix" looks very silly.

Read the link to kde-core-devel for the reasons.

Sorry, but you did not read my comment properly. The reasons given might make sense technically, but not usability-wise. As I said, new users, coming from other OSs, with files from those and no experience to work around the problem are lost. Huess what a user will do if he cannot work with his old music etc., blame it on Qt or Windows, or rather KDE and Linux?

If Qt only cares about the technical dimensions, fair enough, but this is KDE's bugzilla and hence WONTFIX not an option for this issue, IMHO.#

As I read in the mails, there are ways around this. So would it be possible to detect that something does not work with the file, although it exists and tell the user so, instead of just telling him that the file he sees does not exist? Further, one could offer the user to rename the file and use another than the "normal" function for renaming, nothing fancy, i.e. no conversion of e.g. é to e, just making the file work.

Despite I am not an encoding expert (Thiago is), I really believe we shouldn't drop our efforts on making legacy encodings working. We already have to do lots of things for caring about something that should have already been achieved (that is, filenames being in a properly encoding).

However, I also understand the user point of view. I wonder if other operating systems (as the newest from MS and Mac) handles this correctly.

Windows and MacOS X are fully Unicode. There's no such thing as broken paths on Windows, since it uses UTF-16; on MacOS X, which is UTF-8, Finder shows as "C%F3digo" (i.e., URL encoding, which we do support), but simple applications like TextEdit crash while trying to use the file.

No, broken encodings are a thing of the past and should be confined to the terminal, where it belongs. Applications for the past 5 years have created files in UTF-8 encoding. Pen-drives use often VFAT, which is also Unicode clean. 

It's not like a recent convert will get those broken-encoded files.

I think a 5-year transition period is more than enough. KDE 4 does not support non-UTF-8 locales anyways.

Excuse me, then my problem is not related with wrong encodings, and I think also of the jorortega one. I will create a new ticket related to samba/cifs encodings and Dolphin4.

Jorortega, I think our problem occurs with file/folders named correctly from systems with full support of utf-8/unicode encoding (KDE 3.5.x/4 and Windows XP). Such created names are not well managed by Dolphin4 through a CIFS share.

I understand that decoding that might be technically infeasible, because of Qt and QString. But can't we at least substitute it with ? so it appears like C?digo or something?

Not recovering the name properly, that I can understand. Giving strange and mysterious errors, and not letting you delete/move the files, that doesn't make any sense.

It reminds me of windows 98, where you created a folder on the command-line, using iso8859-15 characters, and then windows couldnt' access or delete it, so it was a (lame) way of keeping some files, or messing with users by creating something they couldn't delete. And that was very lame and bad.

I am suggesting exactly that.

The question mark doesn't help because there's no way that the QString name (UTF-16) can be converted back to its system encoding. So, even if we showed with a ? in Dolphin, you wouldn't be able to edit or move it, rename it, etc.

Hi!

I have this problem to. I'm just a user so I don't know much about UTF and stuff.

All I can do is describe my problem.

I have a Ubuntu Server 8.04 with a samba share. On the share I have some files and folders with swedish characters. The files looks fine from the Ubuntu Server console. If i browse the samba share from Windows XP they also looks fine.

But if I browse the samba share from Kubuntu with KDE 4.1.1 with Dolphine it says that the file or folder don't exist if the file\folder contains swedish characters. Browse the same share with Krusader works just fine.

There is no problem with Dolphin and local files or folders with swedish characters.
If I copy a local file or folder with swedish characters using Dolhin to the samba share and then browse the share with Dolphin I get the same error.
And agin no proplem from Krusader, the server console or XP.

Looks like there is some problem with Dolphin.

Kind Regards
/Bernt

I think the WONTFIX resolution is pretty ridiculous. It's taking functionality away from a program and leaving an awfully broken mess behind.

I have a file server that I use to store all of my music that I access via SMB. I use KAudioCreator to rip CDs to a local directory ($HOME/Music) and then I move the music to the SMB share (ripping would take too long if I attempted to rip directly to the share).

I recently ripped a CD with KAudioCreator from a Finnish band that has letters with accents on them. I could access the files without a problem in the local directory. I then moved the files to the SMB share, and had issues with moving the files with accented characters in them (although the files appeared to have been successfully moved, as they no longer appeared in the local source directory).

Now, when I load the directory on the share that has these files in it, I'm immediately told the files don't exist, even though they clearly do, as they're showing up in the directory, although they show up as being 0B. I cannot rename them (although for some reason I'm unable to rename anything on the share). I cannot delete them. I cannot copy or move them back.

My question to you, Thiago, then, is: This is expected, normal behavior? I've somehow supposedly had five years to change my file-naming ways, even though another KDE application named the files to begin with? I think you need to re-evaluate how you're looking at this show-stopping bug. I now have no way of accessing these files, or other files that were pre-existing on the SMB shared drive that were perfectly accessible before the upgrade to KDE4. I experience this same issue when trying to access music from Björk, Green Jellÿ, Queensrÿche, Rinôçérôse, and more, which were all ripped under KDE3. If you're going to disable my access to my perfectly good files, you should at least be failing a lot more gracefully than this and providing an option for recovery while assuming I am an end user who doesn't know what a command line is.

Yes, if your SMB server's setup is broken, that's the correct behaviour.

However, a properly setup server (including Windows machines as serves, for which no setup should be necessary) should work fine under Dolphin. If there's a problem with that, please file a new bug report.

This bug report here is about local files.

The can you at least take a look at bug 171819 please?

J.

And how is one to rename broken filenames? Is the solution to convert incoming files with another application first?

Hello,

    I'm just wondering if that issue is a KDE issue or something which was changed in the packaging of my distribution (which is as of today october 15th, 2008 kubuntu 8.10 intrepid beta) because the issue described above looks to be solved. Yesterday, that was not yet working. Of course, I have updated my distribution between then and now, so I just know that something has been updated that corrected that issue but I have no idea of the package name which was responsible for the resolution. I just know that now the dolphin version is 1.1 using KDE version 4.1.2.

    There is also another issue which was solved at the same time it is the detail view which is working now properly when I browse a smb:/ address, Before today when I was clicking on the detail button I could be sure that dolphin would get frozen and the only thing that I could do it is to kill it.

Thanks.

O.D. 

What about forcing an encoding like Nautilus does?

Sorry, the problem is much deeper than that.

The *soonest* I could provide a solution would be in one year, then requiring a change in each KDE application (starting with Dolphin).

*** Bug 175764 has been marked as a duplicate of this bug. ***

@Thiago: I've reproduced this bug just now... shouldn't this report be reopened?

*** Bug 167097 has been marked as a duplicate of this bug. ***

No, it shouldn't be reopened because we don't want to support wrong encoding. It's just far too much work, for very little gain. We don't have the infrastructure to handle this.

Also, note it's closed WONTFIX. That means "I won't fix". If you want to reopen, please reassign it to yourself. We'll be glad to review your patches and integrate them if they are good.

Just to add some water ....

system is opensuse 11.1 with lastest kde 4.1 stable desktop 
Qt: 4.4.3
KDE: 4.1.3 (KDE 4.1.3) "release 69.2"
kde4-config: 1.0

locale on my system give me
 
LANG=fr_CH.UTF-8
LC_CTYPE="fr_CH.UTF-8"
LC_NUMERIC="fr_CH.UTF-8"
LC_TIME="fr_CH.UTF-8"
LC_COLLATE="fr_CH.UTF-8"
LC_MONETARY="fr_CH.UTF-8"
LC_MESSAGES="fr_CH.UTF-8"
LC_PAPER="fr_CH.UTF-8"
LC_NAME="fr_CH.UTF-8"
LC_ADDRESS="fr_CH.UTF-8"
LC_TELEPHONE="fr_CH.UTF-8"
LC_MEASUREMENT="fr_CH.UTF-8"
LC_IDENTIFICATION="fr_CH.UTF-8"

I've create from my terminal session a file named 
étang planté sur le château.txt

I upload it to another server by scp 
remote server is an opensuse 10.3 set to be a utf-8 server and filesystem

on the remote server my locale are the same
full fr_CH.UTF-8
an ls give me back the right name 
étang planté sur le château.txt

Now if I try to look at it with dolphin and sftp:/
it give me this 
étang planté sur le château.txt
I can open it with kwrite the content is right.(also utf-8 encoded)

If I launch nautilus and this it by sftp:/
I see it under it's correct name aka 
étang planté sur le château.txt
But I can lauch the action open it by kate/kwrite 
they tell me that file doesn't exist.

So we are in plain utf-8 env. And bug is there.
So I'm asking you to reopen this bug as your mention above wrong file name doesn't take place here.

It just bother us, and make us unable to work with customer's data as we can do with the 3.5x version ( absolutely no problems with it )
In conclusion, this is a regression !

Regards.

You said sftp://. That means it's not relevant to this bug. You're talking about remote files, while this bug is about local files.

This bug mentiones smb:/ as well in some of the first comments, but if I'm correct there is bug 158639 for smb:/, if that makes any difference. sftp:/ could have the same issue, so if you add your comment there or file a new bug report and post the report number here, people that stumble about encoding issues and find this report can be guided to the correct report.

local files: this report
smb:/: bug 158639

Ok ok ok it could be a new bug entry.

But all what's describe doesn't even are in relation with dolphin a simple
kdialog with the protocol are wrong ...

Sorry but as I've to work (really, with customers and so) I was obliged today to come back on opensuse 11.0 (bluetooth,wifi) and my prefered working desktop 
aka kde 3.5.10 ... 

I'll try this again with the kde 4.5 version. :-((((

(In reply to comment #18)
> Windows and MacOS X are fully Unicode. There's no such thing as broken paths on
> Windows, since it uses UTF-16; on MacOS X, which is UTF-8, Finder shows as
> "C%F3digo" (i.e., URL encoding, which we do support), but simple applications
> like TextEdit crash while trying to use the file.
> 
> No, broken encodings are a thing of the past and should be confined to the
> terminal, where it belongs. Applications for the past 5 years have created
> files in UTF-8 encoding. Pen-drives use often VFAT, which is also Unicode
> clean. 
> 
> It's not like a recent convert will get those broken-encoded files.
> 
> I think a 5-year transition period is more than enough. KDE 4 does not support
> non-UTF-8 locales anyways.

I just encountered another example that proves the above wrong. A music download service offers albums via zip-file. Opening that zip-file with ark shows broken filenames -> ark cannot decompress -> user without konsole experience is screwed.

This music is only one year old, so the zip was put togehter in the last months, if not on the fly when demanding the download, hence this is still reality and no matter whether the supplier of that file (a big electronics market) is at fault or not, users are screwed although the issue is known and Qt is ignoring that reality.

I tried unzip, which of course can extract the files, but dolphin cannot handle them afterwards.

And I bet that the Zip file was created on Windows, where the 8-bit encoding is considered legacy.

Now go back to that website where you got the music zip file and download a title with characters in Japanese, Russian or Czech. Tell me what happens.

What I wanted to state is: Files with broken/legacy/whatever encoding are still created and supplied, today, i.e. not this is not an issue of the past.

The files are supplied by http://www.247entertainment.com/ I do not know what kind of servers they run for their services. The URL only indicates that they are using tomcat, e.g. http://dlc.247base.com/tomcat/filedl4/ONFDL4/[...]

Netcraft cannot tell what OS they use http://searchdns.netcraft.com/?restriction=site+contains&host=247base.com&lookup=wait..&position=limited

I understand your point, that you do not want to support what others messed-up, yet that means that users on KDE will have issues they cannot handle without konsole. And those issues do not stem from files that are 5 years old, but new ones.

*** Bug 189150 has been marked as a duplicate of this bug. ***

*** Bug 192515 has been marked as a duplicate of this bug. ***

*** Bug 194262 has been marked as a duplicate of this bug. ***

*** Bug 183040 has been marked as a duplicate of this bug. ***

Brilliant "no solution", new zip files download from Japan are broken in KDE 4 and my old backups in zip files too.

Five years is a lot of time, well, I have 15 year old zip files with spanish characters, including all my works in the University.

It's time to forget Konqueror and look for a better solution.

Yes, please.

But the fault lies at your Zip extractor, not Konqueror. The Zip extractor should produce locale-encoded (i.e., UTF-8) file names.

In any case, I won't add compatibility code for 15-year-old archives you read once every 2 years. How often do you start WordStar on DOSEmu anyway?

A couple of times a year but I'm not use WordStar, I'm use Kwrite to open my old text files. The problem is that file names are in spanish and Konqueror 4 can't handle it but Konsole, Konqueror 3, Gnome an Windows can.

But you forgot that the problem persist actually with wget, as you can see in bug https://bugs.kde.org/show_bug.cgi?id=183040. An yes, zip still is used to compress files and in systems with EUC-JP and other encodings so, you must open Konsole or Konqueror 3 and manually rename the files.

If Qt 4 QString encoding breaks encoding compatability the logic solution is implement some legacy system using Qt 3 because Konqueror 3 do the job.

Obviously the decision is yours but, clearly, KDE 4 is a encoding nightmare for not English people.

If wget can generate file names that are out of the locale encoding, wget needs to be fixed. Same as the Zip extractor.

And I use KDE 4 in Portuguese. It works just fine.

It might be possible to fix this problem differently, but it's a huge effort for very little benefit. Broken tools should be fixed, good tools shouldn't have to work around their bugs.

And file formats that encode 8-bit human text without encoding should be banned and left behind in the 1990s.

I don't try to begin a discusion, you take your decision and I'll take mine.

So I think that you must want mark https://bugs.kde.org/show_bug.cgi?id=182934 as WONTFIX too.

(In reply to comment #48)
> If wget can generate file names that are out of the locale encoding, wget needs
> to be fixed. Same as the Zip extractor.
> 

I am OK if KDE's position is that my data is broken. For KDE, having a non-utf-8 file name is similar to have a corrupted filesystem. That would be OK for me, but:

* Base system tools allow me to create non-utf-8 filenames, so this is a requirement from KDE, not from my operating system.

* When I have broken data, I have tools to check if my data is corrupted. I have fsck, I have database consistency check tools. But I don't have _any_ tool to check if my filesystem is considered "broken" by KDE. All I have are cryptic error messages shown when I try to handle a non-utf8 filename. This, IMHO, is the worst part about this issue.

Eduardo: you're right, we should offer a tool to help you repair those broken filenames.

Don't expect that tool to be Dolphin's normal filemanagement mode. However, it would make sense to have it as a plugin.

So this bug can't be fixed but the problem can be solved with a workaround. 

For an user point of view, a plugin or a fix is, more or less, the same so, for me, is a valid solution.

*** Bug 194122 has been marked as a duplicate of this bug. ***

*** Bug 178927 has been marked as a duplicate of this bug. ***

Thiago, your approach appears to be not very practical - just closing wontfix does not make KDE a workable GUI for beginners today - the minimum I would expect before this bug is closed would be proper error reporting - ie KDE should not say xyzfile not found, but it should upon detecting files that are not readable due to encoding errors tell the user that KDE has detected a problem with the encoding and suggest how to fix this - ideally this would be done from within KDE. Just saying file does not exist after you listed it is just REALLY REALLY bad program behaviour. In KDE-3 that kind of crap would not have been tolerated as a regression over a previous version.

Reporting bugs has become really annoying over the last 2 years with many valid bugs being closed with arguments that have nothing to do with a good user experieince, which is what KDE should be about (and has been in the past).

Didn't I explain over and over again that the problem is at a much lower level than Dolphin? Those file names cannot be represented inside KDE (Qt) code. There's no way we can even report that they exist and were skipped from the traditional API.

What Dolphin can do -- and I urge you to file this as a suggestion for improvement for it -- is to provide a tool to rename broken filenames. That's a very Unix-specific tool, by necessity (since the problem cannot happen on Mac or Windows in the first place). How one gets told that there is the need to rename, I don't know.

Frankly, I don't care. Tools generating broken-encoding filenames should be fixed. Dolphin and KDE shouldn't be made to work around those tools' bugs. (This includes Zip-extracting tools)

Big problem is that tools that generate broken encodings are from other OS (mac and windows), not precisely *nix. And that since the other OS supports what you call "broken" encondings. They just read fine since for commercial grade software, the thing called "Backward Compatibility" is pretty important (BTW the newer versions of the rest still can read a broken-encoded produced file nicely). Nobody here is telling that produce broken encoding should be supported, just being able to read, and if necesary, convert them on the fly without the user should care about it (dunno, probably will be impossible to do for read-only media, but well).

Talking about broken things... i removed me from this bug report long time ago (not using KDE anymore). somebody knows how to unsuscribe??? at least this is the 4th time i unsuscribe to this and seems not working at all.

Regards

J.

Mac and Windows don't support broken encoding. No, the tools that are broken are Unix tools: I'm thinking here of unzip and tar.

And wget and probably many others command unix tools and don't, care because with the exception of de factastic and ultra cool Qt 4 encodign system and KDE 4, all command programs and desktop environment in linux CAN HANDLE this files.

My congratulations for the mind behind Qt 4 encoding design. Not adding support for broken encoding names is as brilliant like desing a car only for cities and highways, that stops every time you drive for a road.

KDE 4 is an eyecandy desktop, but not a rock solid desktop for production. This bug, an all related to connecting to servers not working in english UTF-8, confirm this point.

I now really understand why Gnome was been using as default desktop in corporative linux distributions. Think in a spanish corporation, with thousands of old zip files and hundreds of workers with this problem.

Yes, by all means, keep posting comments like these.

The apex of offensive, non-constructive and baseless...

Sorry Thiago if reality offends you. Obviouslly KDE an Qt 4 has a lot of good things, but encoding system is not one of these. You was the first person to protest about consider related problems in this post like smb so, if we can't mention bad things obviouslly we don't mention good things.

The reality is there is a problem here and WONTFIX an blame others was your solution. Well, this kind of actions are not offensive? On the other side I told that don't support legacy enconding in KDE 4 is big desing error and I'm an offender.

Your constructive solution for this problem is fix other programs, you don't know how many or witch must be fixed so I simply explain another solution very very easy that works without problems, don't use KDE 4 in production systems. Your dificult solution is good but my easy solution dislikes you.

You think my comment are offensive and non-constructive, well I live with it but, baseless? The case I explain is not real? I can mention several corporations in Spain that will suffer this bug it they install KDE 4.

In my modest opinion, for corporations, KDE 4 is to KDE the same was Vista to Windows and in my job we uninstall KDE 4 and use KDE 3 again. In my home I can live with this and other encoding related problems so I still testing it.

When a plugin that solves this problem will be added to KDE, in my work we reconsider install KDE 4 again.

(In reply to comment #60)
> Yes, by all means, keep posting comments like these.
> 
> The apex of offensive, non-constructive and baseless...

Actually I think Ignacio managed to express exactly how I feel about this bug, on the first two paragraphs of comment #59.

If you (KDE, kdelibs, Qt, whatever) chose to not support those filenames, fine. But that's _your_ design decision[1], don't blame the other tools. I would even find this Qt/KDE design requirement acceptable if I had a tool to "fix" my filenames. But I don't have even that.

I know you don't have any obligation implement that to help your users, but refusing to do so is a great way to send the message that you don't care about your users.


---
[1] If you still disagree with that, try submitting a Linux kernel patch to make it reject non-utf8 filenames, and see what happens.  ;)

Thiago has left the building and we look for another solutions. Please, vote for the next bug:

https://bugs.kde.org/show_bug.cgi?id=204768

As you may notice, I am the original reporter for this bug.

While I agree that this is a problem, I think that further arguing is like beating a dead horse. The problem has been laid and explained, and harassing the developers will not solve this problem.

So, I'd say thank you to all, but I think this is at the stage where the only useful input is a working code implementation, so please refrain from re-discussing this over and over again.

Thiago is Brazilian he is probably more aware than most developers that people do use more than ASCII characters.

(In reply to comment #64)
Ivo, are you reading my bug report?. I'm not trying to continue the discusion, I'm trying another aproach to solve this problem (an related bugs including mine) and, If you have a better idea please, suggestions are welcome.

Thiago unsuscribed from this bug so discusion are ended and he expressed very clearly his point of view about this problem "Frankly, I don't care. Tools generating broken-encoding filenames should be fixed.".

Just for the record windows 7 will still support "broken encondings"... also windows vista supports it. Don't know about macs, since i stopped using mac with osx 8.5.

http://www.walkernews.net/2007/06/17/how-to-run-chinese-program-in-vista-ultimate/

The thingie is, that m$ tends to do things very automatically. The installer won't bother anyone about this, and my installs i include support for asian languages, so if you install support for other countries, the tables are installed automatically for software that don't know anything about unicode.

Just 2 cents, and just for the record... nothing more, nothing less...

Regards

J.

*** Bug 205493 has been marked as a duplicate of this bug. ***

*** Bug 217691 has been marked as a duplicate of this bug. ***

*** Bug 217975 has been marked as a duplicate of this bug. ***

Can anyone attach any pacth for this, please?

Or the users must will use Nautilus/GNOME forever?

*** Bug 218148 has been marked as a duplicate of this bug. ***

Developer says that this bug don't care so don't expect to be fixed.

If you wish you can vote this fixing tool and pray https://bugs.kde.org/show_bug.cgi?id=204768.

And yes, don't uninstall Nautilus.

*** Bug 200047 has been marked as a duplicate of this bug. ***

*** Bug 221705 has been marked as a duplicate of this bug. ***

*** Bug 224557 has been marked as a duplicate of this bug. ***

*** Bug 224767 has been marked as a duplicate of this bug. ***

*** Bug 233484 has been marked as a duplicate of this bug. ***

*** Bug 234659 has been marked as a duplicate of this bug. ***

*** Bug 236447 has been marked as a duplicate of this bug. ***

It should be pointed out that no kde4 application should actually generate any files with invalid encodings themselves. This is not the case, as ark will extract zip archives with invalid encodings from kde itself. See: https://bugs.kde.org/show_bug.cgi?id=240727

*** Bug 248314 has been marked as a duplicate of this bug. ***

Hi,

I confirm this bug when files are created automatically by extracting an archive (Ark) or by ripping CD (Grip).
Some characters with accent are changed into a symbol like ("Jean-Sébastien" / "Jean-S�bastien"). 
Even if the shell or Dolphin can show the file on the screen, noway to rename, erase, move... this kind of files. Same problem with Krusader or Konqueror... Meanwhile I find that Nautilus can manage (rename, move, erase) easily this problem without understanding why. 

Hope this can be useful.

(In reply to comment #82)
> Hi,
> 
> I confirm this bug when files are created automatically by extracting an
> archive (Ark) or by ripping CD (Grip).
> Some characters with accent are changed into a symbol like ("Jean-Sébastien" /
> "Jean-S�bastien"). 
> Even if the shell or Dolphin can show the file on the screen, noway to rename,
> erase, move... this kind of files. Same problem with Krusader or Konqueror...
> Meanwhile I find that Nautilus can manage (rename, move, erase) easily this
> problem without understanding why. 
> 
> Hope this can be useful.

Bug is confirmed but Thiago don't want to fix it so you are loosing your time.

*** Bug 237164 has been marked as a duplicate of this bug. ***

*** Bug 252160 has been marked as a duplicate of this bug. ***

*** Bug 252801 has been marked as a duplicate of this bug. ***

Wow, only a few bugs left and this will be one of the few most reported bugs that is not a crash - looks like this bug is a good candidate for the title "MOST ANNOYING BUG EVER AFFECTING the largest number of users which the KDE developers decided to WONTFIX?

Please read Thiago's comments.  I don't think he's completely right (one of the "broken" tools is smbmount when you don't specify the encoding the Server uses 100% correct and I don't see how that could be fixed) and don't believe either that this is really unfixable in Dolphin/Qt.

But until I've got the time to go and prove him (or myself) wrong by fixing this on my own, I try to keep the noise on this bug low.  Bitching around doesn't help and just fills up the inboxes of innocent bystanders.

@ comment 88: insulting people is certainly helping to keep the noise down ...

(In reply to comment #89)
> @ comment 88: insulting people is certainly helping to keep the noise down ...

Totally agree. Let's keep noise down. Who's with me?

This bug is one of the things that I stopped using KDE when 4.0 version came out.
KDE 4.5 is still a toy that breaks when you least expect it and that slows down the computer.

KDE was dead at the 3.5.
R.I.P. KDE.

That is because KDE developers improved many useless things, but things, that is really needed and bugs, that is really interfere to use KDE, were forgotten. Of course, the main thing — it is to make it all look good on the releases screenshots. "Who cares how good it works?!" — this should be a new KDE slogan.

Guys,

if I insulted anybody with my last comment I apologize.

But please stop your ranting here, it is not productive at all.  On the contrary, you are about to scare potential developers like me away because you make the KDE users sound like an ungrateful bunch of people.  I have a rough idea how this could be fixed but it would probably have to happen in Qt and is a bit of work for which I currently don't have the time.

Essentially my idea boils down to:

* The issue of displaying the invalid filenames (with question marks and stuff) is solved quite easily: QFile and QDir store the raw representation of the filename in an extra QByteArray member and if that string fails to be converted to a UTF-8 QString,  it has a flag/method like isValidFileName and a method to retrieve the representation of the invalid file.

* But, since we can't convert the display file name back to the original one, once this string representation is inside an application, we lose track of the link between the original FS entry and the displayed one.  I'd probably solve this by overloading the QFile/QDir members which return a QString with one which returns a new class of QFileName (and QPathName).  That one could be a rather thin wrapper around QString which adds the QByteArray I mentioned above.  Both could even be thrown into an union to save RAM if anybody is afraid of losing any of that.

* Hopefully, that wouldn't change anything for all existing applications, since they's still request QStrings.  To make use of this representation, Dolphin and friends had to use QFileNames everywhere they use QString to represent filenames now.  So all applications handling files would have to be changed.  Lot of work.

* There are still corner cases, like how to represent these filenames in URLs and stuff I probably didn't think about.  But at least Dolphin should be able to manage these broken files again.  Or not, I don't know until I (or somebody else) tried.  Probably Thiago can point out at least ten flaws in my idea :)

Anyway, I hope this convinces you that there are actually people thinking about this bug and this might be fixed one day.  But until then, please keep the noise on this bug down.

(In reply to comment #93)
Malte, thank you for your efforts to solve this problem and, as far as I can read, you don't insult anything and I think that this user don't understand well English like me because this problem is related only to not English users.

Is good to know that not all KDE developers are like Thiago and they thinking that this is a serious problem for not English users and companies.

Again, thank your for your efforts.

Just to make sure nobody gets disappointed:  I'm not a KDE developer, just somebody with a rough idea how maybe this could be handled cleanly.  Also, I currently don't have the time to work on this and dunno if anybody else does.  So please don't hold your breath.

One more thing:  Maybe it would be a good idea to start a page on TechBase or UserBase describing ways and circumstances how one can stumble upon filenames with broken Unicode and how to work around it (like, using smbmount with all those voodoo iocharset and codepage parameters).

(In reply to comment #95)
Sad news that you are not a KDE developer but I'm not hold my break because as many other KDE 3 users I stop using KDE 4 in my work and I only use in my home for fun.

*** Bug 252812 has been marked as a duplicate of this bug. ***

*** Bug 254541 has been marked as a duplicate of this bug. ***

as of dolphin 1.5, KDE 4.5.5, the problem still exists.

This is just a workaround how to repair filenames with invalid characters using mc (midnight commander):

When you are in a directory, select "encoding" from the menu and switch to the suggested encoding (or try until you get propper file names).

Now you will find the encoding at the end of the current path as if it was a subfolder. And indeed: If you select (INS) all damaged file names, press F6 ("move") and enter ".." (directory up), all selected files will be renamed according to the system encoding! 

It took quite a while until I found out, manual renaming was so much more tireing. Maybe somebody finds the time to look into the code of mc...

convmv is also good for fixing the filenames

*** Bug 268586 has been marked as a duplicate of this bug. ***

Created attachment 58935 [details]
proof-of-concept fix for the issue

Hi, this patch attempts to work around this bug by using the QString as byte buffer: the latin1 encoding constructor does nothing but stores the bytes in two-byte spaces. It makes possible to the bytestream read by readdir() to reach the unlink() unmodified, and when it is displayed, it is converted to the proper unicode encoding.
This patch is in no way complete, it will break if an unicode string containing the byte 47 ("/") is entered, and some url manipulation will threat it as a directory separator. I think we will need a flag in at least UDSEntry and KFileItem to show if they contain "raw" or proper QString. 

I would like to restart the discussion and see if its possible to develop this patch to a workable solution.

I think I could kiss you just for trying.

Created attachment 59000 [details]
updated proof-of-concept fix for the issue

This is an updated version of by previous patch. It's less intrusive (only kio_file.so is affected), it doesn't break the displaying of the UTF8 encoded filenames, it doesn't break launching the files if their names are UTF8 encoded (the legacy encoded filename wont launch), but all the file operations (delete, copy, rename, etc) will work regardless of the filename encoding. However it breaks amarok: you won't be able add files to the playlist if they contain non-ascii characters (UTF8 or otherwise encoded, doesn't matter) - I still investigate this one.
Meanwhile I realized that the directory separator question is non-issue since the unix filenames don't care about encoding, so there can be no byte 47 in the filename anyway.
Please give it a try and report application breakages, I only tested dolphin/plasma directory widget (works) and amarok (breaks).

Created attachment 59002 [details]
A slightly better patch

Copying, renaming, moving will need some more work and thinking, probably in dolpin itself also.

*** Bug 272215 has been marked as a duplicate of this bug. ***

The approach in these patches proved to be dead-end, broke too many things. A different method is the works, where I store the original bytestream if necessary in KUrl. Stay tuned!

Just wanted to suggest, that if anyone's got the knowledge, Qt 5.0 would probably be a good opportunity to fix this upstream.

(In reply to comment #109)
> Just wanted to suggest, that if anyone's got the knowledge, Qt 5.0 would
> probably be a good opportunity to fix this upstream.

I'm not sure how that would go, QString in itself isn't broken, just not suited for the way it's used to pass around bytestreams.

Created attachment 59970 [details]
Preview of the KUrl-based solution

This is the preview of my proposed solution: I hijacked an unused, private pointer in KUrl to store the actual bytestream when necessary. (This way the sizeof(KUrl) don't change, so only kio_file.so, libkio.so.5.6.0 and libkdecore.so.5.6.0 needs to be replaced. If this gets included, a QByteArray member should be used.) Currenly move/rename works in dolphin, copying changes the filename, so it's not ready yet, symlinking also changes the filename and to fix it KIO API change will be necessary (the destionation is KUrl, but the source is QString). Please take a look, any feedback is much appreciated!

*** Bug 276925 has been marked as a duplicate of this bug. ***

*** Bug 283573 has been marked as a duplicate of this bug. ***

*** Bug 290786 has been marked as a duplicate of this bug. ***

*** Bug 188396 has been marked as a duplicate of this bug. ***

KDE 4.8.1 is out and we still have this problem. I can't believe this bug existed for 4 years now and like I read in one of the comments is running for the title "MOST ANNOYING BUG EVER AFFECTING the largest number of users which the KDE developers decided to WONTFIX?"

I had problems with this in Ubuntu years ago and like I had installed gnome too I just used Nautilus and fixed the name of the files. Never crossed my mind that this bug were so old and affected so many of us.

It's a shame that this remains a problem for end-users after all this time. I guess the problem was in the original design of how to manage encoding names when they designed KDE 4, or maybe they thought this was a "minor" problem or something like that. Now I bet this won't have any solution until KDE 5 or something like that is born [hopefully].

I'll try what comment #100 says and see what happens. I don't want to install nautilus and a lot of s**t just to rename files.

Regars

(In reply to comment #116)
> KDE 4.8.1 is out and we still have this problem. I can't believe this bug
> existed for 4 years now and like I read in one of the comments is running
> for the title "MOST ANNOYING BUG EVER AFFECTING the largest number of users
> which the KDE developers decided to WONTFIX?"
> 
> I had problems with this in Ubuntu years ago and like I had installed gnome
> too I just used Nautilus and fixed the name of the files. Never crossed my
> mind that this bug were so old and affected so many of us.
> 
> It's a shame that this remains a problem for end-users after all this time.

Amen.

This bug still exist, not only with Dolphin, also with Krusader and whatever app FolderManager.

I use "convmv" for can open files with wrong encoding, but in others DesktopEnviroments (ej. Gnome) this bug is solved. T_T

Just to be clear. This bug exists for a long time and it won't be fixed. Resolved upstream means just that this bug was closed "wontfix" in upstream. Upstream developer removed himself from cc list a long time ago, so no one available to change anything reads this comments. We talk here with ourselfs here, no one is listening.

This does not affect only krusader or dolphin. It affects all Qt and KDE apps.

This won't be fixed in Qt 5 so it won't be fixed in KDE 5. See comments in http://www.macieira.org/blog/2011/09/qurl-in-qt-5-encoding/

"""no, no one has the intention of ever fixing that in Qt. Broken filename encodings will be forever considered filesystem corruption."""

(I'm just "me too" guy summarizing this bug. There's no reason to argue with me.)

This is very sad, how can i use kde in portuguese language if it cant handle diferent encoding?

i think that kde needs a Every detail Matters round up like gnome to ensure that this kind of frustating bugs disapear.

https://live.gnome.org/EveryDetailMatters

regards,

(In reply to comment #118)
> Just to be clear. This bug exists for a long time and it won't be fixed.
> Resolved upstream means just that this bug was closed "wontfix" in upstream.
> Upstream developer removed himself from cc list a long time ago, so no one
> available to change anything reads this comments. We talk here with ourselfs
> here, no one is listening.
> 
> This does not affect only krusader or dolphin. It affects all Qt and KDE
> apps.
> 
> This won't be fixed in Qt 5 so it won't be fixed in KDE 5. See comments in
> http://www.macieira.org/blog/2011/09/qurl-in-qt-5-encoding/
> 
> """no, no one has the intention of ever fixing that in Qt. Broken filename
> encodings will be forever considered filesystem corruption."""
> 
> (I'm just "me too" guy summarizing this bug. There's no reason to argue with
> me.)

Wow, It's really disappointing to see that the position of Thiago in this still the same.

 "filesystem corruption" = not our problem = we won't do anything about it. 

This is really letting me down of KDE and Qt for now. 

I don't know what to say. 

Regards Guys [I'll stay suscribed just in case something happens]

Corrupt file names can also be the result of unzipping archives that come from another platform.
In such cases, the file system itself can IMO not be consiedered as corrupt file system.

Unless fsck complains and offers fixation. IIRC some disk checkers complain about invalid characters,
but I don't know of any tool that cleans up hybrid encodings (esp. on portable usb drives).

Regards
Ralf

This issue can be triggered via any program that writes non-utf8 filenames to a utf-8 filesystem without proper conversion. Unzip likes to do that one.

Now:
- Hello! Notice I'm the original reporter of this bug.
- It's not true that you can't use other languages correctly. Thiago Macieira, the very often quoted Qt developer that maintains the code and says he's not going to change it, also speaks Portuguese, so he knows that the world isn't all ASCII.
- It's been said other times in this thread but I'll say it again: the issue has been reported, and the developers understand it, how it happens, when, and why. They just think that fixing this bug takes lots and lots of work and code changes, and so don't want to work on it.
- So the thing is, this bug at this point only serves to wait for someone who fixes this and adds a patch. Adding more reasons and angry comments doesn't add anything, and, in fact, is the main reason why the developers don't even read this bug report anymore. (I do though!)

Hey guys right now I'm still using KDE, and to "fix" the encoding problem of this files I just found an a really light file manager: Thunar, is the file manager of XFCE but unlike Nautilus I don't have to install all kind of sh*t and the desktop to use it, just a couple of packages and its done, just right-click, Rename the file and that's it.

Hope it works for you too.

And this is another prove that the problem is inside KDE, or if is not KDE does not know how to handle it. A lot of filemanagers in other desktops environments in a lot of distros can do this but in KDE we just have to "suck it up" because it's too much work in exchange for so little benefit [for the users!!]. I guess something went wrong somewhere.

Greetings

*** Bug 297733 has been marked as a duplicate of this bug. ***

*** Bug 301833 has been marked as a duplicate of this bug. ***

I'm on KDE 4.8.4 (Debian) and today I've found a solution for Dolphin (I have the same problem with Ark while extracting a file with non-ASCII caracters but this method I give to you don't work for it) :

> Edit the file called "kdeglobals" on your home/.kde/share/config
> Add this line in the "[Locale]" paragraph :
Charset=iso8859-15 
I'm French so this is the corresponding iso for my language. This iso (8859-15) works for a lot of languages, especially almost all of european ones.
> Save, close the file, eventually log out and log in but I didn't need to do this to make it work.
Now you will be able to display and modify these files in Dolphin.

Modifying your Locale.gen is useless in KDE.
It works for me, I hope it will work for you too.
Sorry if my english is bad !

In fact, it still doesn't work with the method I said before...I was using in the same time convmv, I thought I have entered a wrong command to convert my files into UTF-8 with it because nothing was happened...but it works (with convmv....!)....

hello

when will RESOLVE this SERIOUS PROBLEM? i have this problem with KDE 4.8.5 with opensuse 12.1..
is resolved then you can explain me how i can resolve information...?....thank you

*** Bug 232608 has been marked as a duplicate of this bug. ***

*** Bug 311932 has been marked as a duplicate of this bug. ***

*** Bug 313842 has been marked as a duplicate of this bug. ***

Calling this filesystem corruption is just an arrogant way to say:
we will not fix real world issues, we insist in the world fixing this.

Given, how easy it is to create such an item, e.g.:

	touch $(printf "snaf\374.txt")

this attitude is ridiculous.

Apart from unzip, we are bitten by a winscp/sshd combination, that doesn't encode properly
(all current versions).

@Thiago: wrongly encoded file names will *never* go away. 

What would you thing, if the kernel would forbit access to files, that you saved in a crypted filesystem container, because its filename has an invalid encoding. Say, this contains an important work of yours. Guess what, you would call this kernel names. 

Next logical step would be to automatically *format* such a media in order to prevent further propagation. Your argumentation would even justify this. But face it: middle age is over, and the world isn't turning around KDE, KDE is actively loosing users, because of this attitude.

@Hans-Peter Jansen
You're absolutely right. Lately I'm thinking of not using KDE for this ridiculous attitude.

I too consider this atitude very wrong...

i left using KDE among other things because of wrong code in files...initially i installed used terminal to change names, but after some time i simply quit...changed to gnome...

You have to resolve this, because the world is not only english :)

I too consider this atitude very wrong...

i left using KDE among other things because of wrong code in files...initially i used terminal to change names, but after some time i simply quit...changed to gnome...

You have to resolve this, because the world is not only english :)

Why blaming KDE for something that is in Qt (even if the respective developer is both a Qt and KDE person)? Read comments #118 and #122, if not everything.

Hans-Peter Jansen: You're wasting your breath ;)

See comment #118 : https://bugs.kde.org/show_bug.cgi?id=165044#c118

Nobody reads this bug report, nobody cares. Thiago removed himself from CC list a loooong time ago, so you can target your comment to him, but he won't read it.

I think this project needs someone like Linus Torvalds. When someone in kernel would make a change breaking userspace program with comment "applications should get fixed", Linus would yell at him about being arrogant stupid ... a*hole (Linus would be using even stronger words)   and he would prevent such change. It's always pity when library does not care about real world.

Am Mittwoch, 20. Februar 2013 schrieb Hans-Peter Jansen
> Nobody reads this bug report, nobody cares. Thiago removed himself from CC list
> a loooong time ago, so you can target your comment to him, but he won't read
> it.

Hello,

we should make more people vote for this bug in order to draw more attention
to it. This bug is indeed annoying, but it is possible to drop dolphin without
dropping KDE. 

What file manager is working with you? I use mc all the time in such cases,
and for copying anyway (because of the time stamp bug).

Kind Regards
Ralf

(In reply to comment #138)
> we should make more people vote for this bug in order to draw more attention
> to it. This bug is indeed annoying, but it is possible to drop dolphin
> without
> dropping KDE. 

Seems you don't understand this bug. It's not a bug in dolphin. It's not a bug in KDE. It's bug in Qt library. All Qt applications have this bug. Because KDE uses Qt library it means also all KDE applications are affected and because Dolphin is KDE application, that's why it is affected. This bug mentions just Dolphin, but all KDE and Qt apps have this bug. The file, you can't work with in Doplhin, can't be opened by any KDE nor Qt application.

Am Donnerstag, 21. Februar 2013 schrieben Sie:
> All Qt applications have this bug. Because KDE
> uses Qt library it means also all KDE applications are affected and because
> Dolphin is KDE application, that's why it is affected. This bug mentions just
> Dolphin, but all KDE and Qt apps have this bug. The file, you can't work with
> in Doplhin, can't be opened by any KDE nor Qt application.

Hi and thanks for clarification. 

1) My approach was to use another file manager in order to
fix broken file names, so just replace dolphin.

2) Yesterday, I had a malformed file name, and in Dolphin it
was not possible to copy it to another folder. However, I
could open that (PDF) file from Dolphin into Okular, then
I saved a copy as new file, and could also remove the
file with broken file name from within Dolphin.

dolphin --version
Qt: 4.8.2
KDE: 4.8.4 (4.8.4)
Dolphin: 2.0

But this was no Umlaut. Just double checked, and still malcoded
Umlauts are displayed like <?> and transscribed like this when
you drag&drop file names to non-KDE applications or Konsole.

Rather than sending plain (flatterned) file names to KDE apps,
it could be considered to sending direct inode references 
within QT/KDE applications, couldn't it?

3) Next test: I installed xfe (+package xfe-i18n) as alternative
file manager. Interestingly, xfe displays "broken" umlauts
correctly (auto detection?), and passes the file name
string correctly to xpdf, so the file opens as it should.

But right you are: No chance to send it to okular.

Accordingly, file names have actually to be fixed before
they will work with KDE. 

Is there a bug, "KDE can't handle file names beyond official encoding"?
There should probably be some legacy switch for this support, as
filenames containing binary code might compromise applications
when parsing them from environment.

Kind regards
Ralf

*** Bug 315715 has been marked as a duplicate of this bug. ***

Created attachment 78460 [details]
Working fix

This attacment is a patch against kde4libs-4.10.1.
It works around the problem like this: in KLocalePrivate::initFileNameEncoding() KDE sets the QFile's encoding/decoding function, to to/fromUTF8() in QString, which in turn calls QUtf8's converter function (QUtf8 is not exported to developers, so I had to use an inefficient method, I think it would be better if we could use the state parameter for error detection). I replaced this with the said functions' copy/pasted version and changed it, so when it encounters an invalid UTF8 string, it will encode it byte by byte, mapping the lower 128 their normal unicode place and the upper 128 to U+18000-U+1807F, and of course the decoder reverses it.
To make this actually work you have to define the KDE_UTF8_FILENAMES enviroment variable (otherwise we would need to patch at QT level). 
So, do the following:
.kde/env/KDE_UTF8_FILENAMES.sh
with this content:
export KDE_UTF8_FILENAMES=yesplease

logout, login, try dolphin on faulty files. (instead of the usual boxed "?" you'll see just boxes) 

HTH

I built kdelibs with the patch, exported the KDE_UTF8_FILENAMES and it seems to work as advertised: strange chars are replaced by boxes on the filenames, but I can rename and delete them sucessfully.

I'll keep using the patch on one of my machines to see if there are any other issues.

Big thanks to Szokovacs Robert, and to all the people that complained loudly for all these years: this is your chance. Test the patch, use it, and if it works, bug your favorite KDE developer, or distribution to include it. Tell your friends to use it. Package and distribute it. Don't just complain and leave. I reported this bug in 2008, and I'm still here :)

P.s.: Applying the patch gave me a whitespace warning:
encoding.patch:173: trailing whitespace.

Maybe the fix would be more acceptable in the mainline if the new behaviour would only apply if the user sets KDE_UTF8_FILENAMES to a specific value, for example "broken_names".

Fixed in http://commits.kde.org/kdelibs/f4269ef3498581964e8a1a13cd0d6d7f19c88762
and 
https://projects.kde.org/projects/kde/kdelibs/repository/revisions/736d5237f822fc72736f75f379c4f86d6bf48098

Thanks for your work!
It seems that for all the complaining, nobody bothered to test your patch, or even say thanks. It's a pity.

(In reply to comment #145)
> Fixed in
> http://commits.kde.org/kdelibs/f4269ef3498581964e8a1a13cd0d6d7f19c88762
> and 
> https://projects.kde.org/projects/kde/kdelibs/repository/revisions/
> 736d5237f822fc72736f75f379c4f86d6bf48098

YES, YES, YES.

Thanks!. I was waiting years for this. (^_^)

Thank you for your work...almost 5 years later...it's not your fault, off course...

but this and other problems made me quit using kde, more of the atitude of the developers in some bugs...

maybe one day i install kde only to see if this is fixed...it was a major bug to me, because i'm portuguese, and some files didn't open because of the wrong encoding...

keep alive and kicking... :)

I did not have such encoding problems since a long time, because I did not work with dual-boot -windows anymore, but nice to know that it works now :-) thanks

Nice to know that !
Thanks :)

Nice to know that !
Thanks :)

(In reply to comment #146)
> Thanks for your work!
> It seems that for all the complaining, nobody bothered to test your patch,
> or even say thanks. It's a pity.

Actually there was some discussion on the reviewboard, and some delay was caused by me - I did not realize that I just need to apply for a developer account and not wait around for someone elso to commit for me :) Also, I forgot to include this bug in the commit, sorry about that.

Thank you, Robert! You're a real hero!
Regards
Ralf

Am Donnerstag, 4. Juli 2013 schrieben Sie:
> https://bugs.kde.org/show_bug.cgi?id=165044
> 
> --- Comment #145 from Szokovacs Robert <szo@szo.hu> ---
> Fixed in
> http://commits.kde.org/kdelibs/f4269ef3498581964e8a1a13cd0d6d7f19c88762
> and 
> https://projects.kde.org/projects/kde/kdelibs/repository/revisions/736d5237f822fc72736f75f379c4f86d6bf48098
> 
>

(In reply to comment #145)
> Fixed in
> http://commits.kde.org/kdelibs/f4269ef3498581964e8a1a13cd0d6d7f19c88762
> and 
> https://projects.kde.org/projects/kde/kdelibs/repository/revisions/
> 736d5237f822fc72736f75f379c4f86d6bf48098

Thank you so much!  And, just to clarify, this works without setting KDE_UTF8_FILENAMES as suggested previously?

Thank you, gracias!

This is a great day for KDE, and you made it, Robert.

Thanks for your patience and I really appreciate, that sanity finally won over arrogance.

BTW, KDEPIM is in deep need of people like you, Robert <wink> ;-)

(In reply to comment #154)
> (In reply to comment #145)
> > Fixed in
> > http://commits.kde.org/kdelibs/f4269ef3498581964e8a1a13cd0d6d7f19c88762
> > and 
> > https://projects.kde.org/projects/kde/kdelibs/repository/revisions/
> > 736d5237f822fc72736f75f379c4f86d6bf48098
> 
> Thank you so much!  And, just to clarify, this works without setting
> KDE_UTF8_FILENAMES as suggested previously?

Yes, if your locale is any kind of UTF8, the fix will kick in.

Thank you will check this as soon as I have a working
system (my HDD died) awaiting a new one.
Will be testing under Fedora 19 KDE :)
Right now online with Fedora 18 KDE live boot :)

Regards,
Monts



On 4 July 2013 16:33, Szokovacs Robert <szo@szo.hu> wrote:

> https://bugs.kde.org/show_bug.cgi?id=165044
>
> --- Comment #157 from Szokovacs Robert <szo@szo.hu> ---
> (In reply to comment #154)
> > (In reply to comment #145)
> > > Fixed in
> > >
> http://commits.kde.org/kdelibs/f4269ef3498581964e8a1a13cd0d6d7f19c88762
> > > and
> > > https://projects.kde.org/projects/kde/kdelibs/repository/revisions/
> > > 736d5237f822fc72736f75f379c4f86d6bf48098
> >
> > Thank you so much!  And, just to clarify, this works without setting
> > KDE_UTF8_FILENAMES as suggested previously?
>
> Yes, if your locale is any kind of UTF8, the fix will kick in.
>
> --
> You are receiving this mail because:
> You are on the CC list for the bug.
>

In Qt 5 the filename encoding callbacks were removed, so we will face this issue again in a few years. Let's enjoy Róbert's work for the time being.

http://qt-project.org/doc/qt-5.1/qtcore/qfile-compat.html#setEncodingFunction

*** Bug 325114 has been marked as a duplicate of this bug. ***

*** Bug 254966 has been marked as a duplicate of this bug. ***

Additional test files are available as attachements to Bug 254966.

*** Bug 237817 has been marked as a duplicate of this bug. ***

*** Bug 349898 has been marked as a duplicate of this bug. ***

*** Bug 349996 has been marked as a duplicate of this bug. ***

I just extracted an archive which contained a file that I could not copy/edit/move/rename/delete with dolphin. The file was simple readme file inside a nested subdirectory called L'$'\351''ame.html'

The error messages in dolphin were extremely vague and did not indicate in an way that KDE and QT are incapable by design of handling files with these encoding. They did not inform the user why the file was causing these problems or even hint that they needed to be renamed in a terminal because of a WONTFIX on this bug.

The only reason I knew what to do is because I encountered this bug years ago when operations on these files actually worked. Any normal user will have absolutely no idea what to do, and higher level systems will not handle these failures gracefully when these directory structures can't be modified because of a simple filename.

The file or folder L'$'\351''ame.html' does not exist.
 - This is _false_ it does exist!
Could not delete file L'$'\351''ame.html'.
 - No reason provided whatsoever. The user will not know how to solve this problem.
Could not remove folder "".
 - All other files deleted, but the nested file was ignored leaving the user confused.

This is ridiculous.

It might be a lot of work, but a solution to this problem would be to store these strings as a tuple, where there is a degraded and user-friendly version of the string and the original unmodified string. If no changes are made to the string, use the original unmodified string. If the user decides to change the string, then they are interacting with the degraded version and deciding to keep that new string.

unsubscribe

> Gesendet: Donnerstag, 18. April 2019 um 00:05 Uhr
> Von: "Nathan Shearer" <bugzilla_noreply@kde.org>
> An: pulvermacher@gmx.de
> Betreff: [kdelibs] [Bug 165044] Dolphin can't handle well files/folders with wrong encoding
>
> https://bugs.kde.org/show_bug.cgi?id=165044
>
> Nathan Shearer <kde-20091112@nathanshearer.ca> changed:
>
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                  CC|                            |kde-20091112@nathanshearer.
>                    |                            |ca
>
> --- Comment #166 from Nathan Shearer <kde-20091112@nathanshearer.ca> ---
> I just extracted an archive which contained a file that I could not
> copy/edit/move/rename/delete with dolphin. The file was simple readme file
> inside a nested subdirectory called L'$'\351''ame.html'
>
> The error messages in dolphin were extremely vague and did not indicate in an
> way that KDE and QT are incapable by design of handling files with these
> encoding. They did not inform the user why the file was causing these problems
> or even hint that they needed to be renamed in a terminal because of a WONTFIX
> on this bug.
>
> The only reason I knew what to do is because I encountered this bug years ago
> when operations on these files actually worked. Any normal user will have
> absolutely no idea what to do, and higher level systems will not handle these
> failures gracefully when these directory structures can't be modified because
> of a simple filename.
>
> The file or folder L'$'\351''ame.html' does not exist.
>  - This is _false_ it does exist!
> Could not delete file L'$'\351''ame.html'.
>  - No reason provided whatsoever. The user will not know how to solve this
> problem.
> Could not remove folder "".
>  - All other files deleted, but the nested file was ignored leaving the user
> confused.
>
> This is ridiculous.
>
> It might be a lot of work, but a solution to this problem would be to store
> these strings as a tuple, where there is a degraded and user-friendly version
> of the string and the original unmodified string. If no changes are made to the
> string, use the original unmodified string. If the user decides to change the
> string, then they are interacting with the degraded version and deciding to
> keep that new string.
>
> --
> You are receiving this mail because:
> You are on the CC list for the bug.

Removing complaining user.

(In reply to Nathan Shearer from comment #166)
> I just extracted an archive which contained a file that I could not
> copy/edit/move/rename/delete with dolphin. The file was simple readme file
> inside a nested subdirectory called L'$'\351''ame.html'
> 
> The error messages in dolphin were extremely vague and did not indicate in
> an way that KDE and QT are incapable by design of handling files with these
> encoding. They did not inform the user why the file was causing these
> problems or even hint that they needed to be renamed in a terminal because
> of a WONTFIX on this bug.
> 
> The only reason I knew what to do is because I encountered this bug years
> ago when operations on these files actually worked. Any normal user will
> have absolutely no idea what to do, and higher level systems will not handle
> these failures gracefully when these directory structures can't be modified
> because of a simple filename.
> 
> The file or folder L'$'\351''ame.html' does not exist.
>  - This is _false_ it does exist!
> Could not delete file L'$'\351''ame.html'.
>  - No reason provided whatsoever. The user will not know how to solve this
> problem.
> Could not remove folder "".
>  - All other files deleted, but the nested file was ignored leaving the user
> confused.
> 
> This is ridiculous.
> 
> It might be a lot of work, but a solution to this problem would be to store
> these strings as a tuple, where there is a degraded and user-friendly
> version of the string and the original unmodified string. If no changes are
> made to the string, use the original unmodified string. If the user decides
> to change the string, then they are interacting with the degraded version
> and deciding to keep that new string.

Hello there -- I'm the original reporter of this bug.

Since this one is marked as fixed, my suggestion would be to open a new bug report, as the issue you saw may be a regression, rather than the same as this bug, and it helps keep the discussion focused.

Also, if possible, my suggestion is to also include an archive that triggers this issue as an attachment, so developers can quickly jump on this one.

Happy KDE'ing and happy easter everyone!

(In reply to Ivo Anjo from comment #169)
> (In reply to Nathan Shearer from comment #166)
> > I just extracted an archive which contained a file that I could not
> > copy/edit/move/rename/delete with dolphin. The file was simple readme file
> > inside a nested subdirectory called L'$'\351''ame.html'
> > 
> > The error messages in dolphin were extremely vague and did not indicate in
> > an way that KDE and QT are incapable by design of handling files with these
> > encoding. They did not inform the user why the file was causing these
> > problems or even hint that they needed to be renamed in a terminal because
> > of a WONTFIX on this bug.
> > 
> > The only reason I knew what to do is because I encountered this bug years
> > ago when operations on these files actually worked. Any normal user will
> > have absolutely no idea what to do, and higher level systems will not handle
> > these failures gracefully when these directory structures can't be modified
> > because of a simple filename.
> > 
> > The file or folder L'$'\351''ame.html' does not exist.
> >  - This is _false_ it does exist!
> > Could not delete file L'$'\351''ame.html'.
> >  - No reason provided whatsoever. The user will not know how to solve this
> > problem.
> > Could not remove folder "".
> >  - All other files deleted, but the nested file was ignored leaving the user
> > confused.
> > 
> > This is ridiculous.
> > 
> > It might be a lot of work, but a solution to this problem would be to store
> > these strings as a tuple, where there is a degraded and user-friendly
> > version of the string and the original unmodified string. If no changes are
> > made to the string, use the original unmodified string. If the user decides
> > to change the string, then they are interacting with the degraded version
> > and deciding to keep that new string.
> 
> Hello there -- I'm the original reporter of this bug.
> 
> Since this one is marked as fixed, my suggestion would be to open a new bug
> report, as the issue you saw may be a regression, rather than the same as
> this bug, and it helps keep the discussion focused.
> 
> Also, if possible, my suggestion is to also include an archive that triggers
> this issue as an attachment, so developers can quickly jump on this one.
> 
> Happy KDE'ing and happy easter everyone!

It is in deed a regression, the fix depended on a QT feature that was removed in 5.x, iirc.

(In reply to Szokovacs Robert from comment #170)
> It is in deed a regression, the fix depended on a QT feature that was
> removed in 5.x, iirc.

Is there a report in Qt tracker asking for bring back this feature removed in 5.x?

https://bugreports.qt.io/browse/QTBUG-59402

Git commit 6738a8b2f71c527f30a624b0b560f79d992715d3 by Christoph Feck.
Committed on 21/05/2019 at 11:05.
Pushed by cfeck into branch 'master'.

[kioslave/file] Add a codec for legacy filenames

UNIX filenames can contain any bytes (except \0 and /).
Qt's QFile::decodeName() calls QString::fromLocal8Bit(), assuming that all
filesystems use the system's locale encoding. For filenames that have been
created with a different encoding, and have not yet been converted (e.g. using
convmv), this creates non-reversible U+FFFD (REPLACEMENT CHARACTER)
code points in the filenames.

For example, some old-style archives might not contain any information about
the encoding of the filenames, and even today archivers extract them without
trying to convert to the locale's encoding.

While full support for those filenames is not needed, Dolphin should at least
be able to delete, rename, and move those files. Since all actual (local) file
handling is done inside the file kioslave, patching Dolphin will not help.

This code is a near verbatim copy of the code we had in kdelibs, written by
Szókovács Róbert. Only minor adaptions to Qt5 were done. It decodes invalid
bytes as U+10FExx from Plane 16 (Supplementary Private Use Area-B) to be able
to encode them later.

Dolphin could detect filenames with those characters, and either mark them
(by color or overlay icon), or even automatically offer to rename them.
Related: bug 204768

TEST PLAN

touch "/tmp/test-"$'\377'".txt"
dolphin /tmp

Copying and deleting a test file worked with this code, failed without.

Reviewers: dfaure, Frameworks, Dolphin

Reviewed by: dfaure

Differential Revision: https://phabricator.kde.org/D18161

M  +1    -1    src/ioslaves/file/CMakeLists.txt
M  +8    -0    src/ioslaves/file/file.cpp
A  +174  -0    src/ioslaves/file/legacycodec.cpp     [License: LGPL (v2+)]
A  +66   -0    src/ioslaves/file/legacycodec.h     [License: LGPL (v2+)]

https://commits.kde.org/kio/6738a8b2f71c527f30a624b0b560f79d992715d3

test plan from comment 173 fails on Arch Linux after upgrade to frameworks 5.59.
Dolphin says "The file or folder /tmp/test-􏹿.txt does not exist."

That's because this file name is using the same private code points that are used to encode not correctly encoded UTF-8 file names. You cannot support both.

Well, now at least I can rename and then delete the file. :)

*** Bug 481720 has been marked as a duplicate of this bug. ***

*** Bug 392317 has been marked as a duplicate of this bug. ***