Bug 271664

Summary: Virtuoso 6.1.3 cause nepomuk encoding error.
Product: [Unmaintained] nepomuk Reporter: Weng Xuetian <wengxt>
Component: generalAssignee: Sebastian Trueg <sebastian>
Status: RESOLVED UPSTREAM    
Severity: normal CC: alejandronova, Aleksey_R, alexvpetrov, bruno, dimti, fedotov.i.f, ismail, jem, jens, linux, mail, mat69, matija, me, plusfabi, prozac, rdieter, trueg, violet.amethyste
Priority: HI    
Version: unspecified   
Target Milestone: ---   
Platform: Arch Linux   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:
Attachments: Patch to Virtuoso 6.1.3 which fixes the issue
Message preview bubble displays with incorrect charset

Description Weng Xuetian 2011-04-25 06:22:25 UTC
Version:           unspecified (using KDE 4.6.2) 
OS:                Linux

KDE's nepomuk will recognize all non-ascii character as "?", and dolphin will not display correct message. Downgrade to 6.1.2 will solve this problem. I'm not sure whether it's an upstream bug or an upstream API change cause this.

Reproducible: Always

Steps to Reproduce:
1. create a folder "测试" and a empty file in it.
2. set option display parent folder in dolphin's information panel.


Actual Results:  
the parent folder for that file will be "??".

Expected Results:  
the parent folder for that file will be "测试".

Package version:
* virtuoso 6.1.3-1
* soprano 2.6.0-1
* kdebase-runtime 4.6.2-1
* strigi git20110405-1
Comment 1 Matthias Fuchs 2011-05-03 12:20:59 UTC
I have the same problem. This causes everything that uses non-ascii signs to fail like tags etc.
I am also using Archlinux.
Comment 2 Sebastian Trueg 2011-05-05 10:13:06 UTC
This is a known bug that was already reported to OpenLink. Sadly they did not find the time to fix it before the 6.1.3 release and I did not find a workaround yet. :(
Comment 3 Matija Šuklje 2011-05-07 14:41:46 UTC
I suspect Bug #272643 might be a duplicate or rather a downstream issue which is actually triggered by this bug.
Comment 4 Matija Šuklje 2011-05-07 15:29:11 UTC
(In reply to comment #3)
> I suspect Bug #272643 might be a duplicate or rather a downstream issue which
> is actually triggered by this bug.

ooooor not. Downgrading to 6.1.2 did not save the bug I mentioned, I do seem to be able to rename the tags in Akonadi Console now to get the non-ASCII characters back.

My bad. Ignore my comment #3 then.
Comment 5 Sebastian Trueg 2011-05-30 10:05:20 UTC
I finally managed to write a test case which allows the Virtuoso devs to easily reproduce the problem. They are on it.
Comment 6 Matija Šuklje 2011-05-30 10:16:58 UTC
(In reply to comment #5)
> I finally managed to write a test case which allows the Virtuoso devs to easily
> reproduce the problem. They are on it.

Nice :]

You rock! :D
Comment 7 violet.amethyste 2011-08-02 09:58:59 UTC
This problem caused this bug too  Bug 275632
Comment 8 Rex Dieter 2011-08-08 12:54:04 UTC
*** Bug 275632 has been marked as a duplicate of this bug. ***
Comment 9 Rex Dieter 2011-08-08 12:58:41 UTC
sebastian, if you've any references to the bug from virtuoso's side and/or any patches to test, I'd be happy to help pass this along to kde-packagers...
Comment 10 Rex Dieter 2011-08-24 18:01:57 UTC
Enough me-too's, marking confirmed.
Comment 11 Ismail Donmez 2011-08-24 19:28:11 UTC
An upstream bug/link reference would be nice for tracking purposes.
Comment 12 Christophe Marin 2011-08-28 15:23:59 UTC
*** Bug 279923 has been marked as a duplicate of this bug. ***
Comment 13 Sebastian Trueg 2011-09-08 15:21:15 UTC
Created attachment 63510 [details]
Patch to Virtuoso 6.1.3 which fixes the issue

This patch (provided by OpenLink) does fix the unicode issues which result in several problems like the one described in this bug.
It is recommended to apply this patch to all binary packages.
Comment 14 Aleksandr Letov 2011-09-16 12:09:54 UTC
Что именно нужно сделать, чтобы применить этот патч?

translate:
What to do to apply this patch?
Comment 15 Elias Probst 2011-09-16 13:01:13 UTC
(In reply to comment #14)
> What to do to apply this patch?

If you don't know how to deal with patches, please wait for updated packages provided by your distribution or notifiy your distribution's packagers of this patch by filing a bugreport in the distribution's bugtracker.
Comment 16 Alejandro Nova 2011-09-16 15:32:02 UTC
Chakra Linux and Fedora have packages.

For Chakra, enable the Testing repository.

Fedora users: follow this Koji link and download the patched Virtuoso from there.
http://koji.fedoraproject.org/koji/buildinfo?buildID=263583

Alternatively, you can wait until your mirrors sync.
Comment 17 Ismail Donmez 2011-09-16 18:44:13 UTC
For openSUSE we fixed this in KDE:Distro:Factory and KDE:Release:47 repositories.
Comment 18 Christophe Marin 2011-09-21 15:22:23 UTC
*** Bug 282482 has been marked as a duplicate of this bug. ***
Comment 19 Sebastian Trueg 2011-09-28 07:53:53 UTC
*** Bug 282950 has been marked as a duplicate of this bug. ***
Comment 20 Sebastian Trueg 2011-09-29 07:31:01 UTC
*** Bug 277537 has been marked as a duplicate of this bug. ***
Comment 21 Swâmi Petaramesh 2011-10-07 13:09:00 UTC
Created attachment 64311 [details]
Message preview bubble displays with incorrect charset

Confirming Bug 275632 (made a duplicate of the present one).

Not sure if this is a Nepomuk encoding issue (thus a duplicate), as it manifests itself only for the preview of actual message contents (not subject, nor other fields etc).

« Non-ASCII characters are displayed incorrectly in the tooltip message preview. » (see attached image, some personal information manually blurred) - seen on 2 different machines today - Ubuntu Oneiric Beta 2 with all latest updates installed.
Comment 22 Elias Probst 2011-10-07 13:14:51 UTC
(In reply to comment #21)
> Created an attachment (id=64311) [details]
> Message preview bubble displays with incorrect charset

See my comment at bug#275632: https://bugs.kde.org/show_bug.cgi?id=275632#c14
Comment 23 violet.amethyste 2011-10-07 13:31:34 UTC
(In reply to comment #21)

> Not sure if this is a Nepomuk encoding issue (thus a duplicate), as it
> manifests itself only for the preview of actual message contents (not subject,
> nor other fields etc).

Disagree. I also had this problem with tags in Dolphin not only in message preview in kmail
Comment 24 Elias Probst 2011-10-07 13:33:59 UTC
(In reply to comment #23) 
> Disagree. I also had this problem with tags in Dolphin not only in message
> preview in kmail

It probably just appears to only affect KMail, as IIRC it only affects files/content indexed since the update to 6.1.3, but not already existing content which was previously indexed using 6.1.2.
Comment 25 Sebastian Trueg 2011-10-07 13:42:39 UTC
(In reply to comment #24)
> (In reply to comment #23) 
> > Disagree. I also had this problem with tags in Dolphin not only in message
> > preview in kmail
> 
> It probably just appears to only affect KMail, as IIRC it only affects
> files/content indexed since the update to 6.1.3, but not already existing
> content which was previously indexed using 6.1.2.

(In reply to comment #21)
> Not sure if this is a Nepomuk encoding issue (thus a duplicate), as it
> manifests itself only for the preview of actual message contents (not subject,
> nor other fields etc).

(In reply to comment #23)
> (In reply to comment #21)
> 
> > Not sure if this is a Nepomuk encoding issue (thus a duplicate), as it
> > manifests itself only for the preview of actual message contents (not subject,
> > nor other fields etc).
> 
> Disagree. I also had this problem with tags in Dolphin not only in message
> preview in kmail

Replying to all of you at the same time: this is exactly what this bug and the attached patch for Virtuoso 6.1.3 is about. It is fixed. No need for further information - unless you are running a patched Virtuoso and have re-indexed the affected data.
The latter may be another issue: automatically updating the affected data although at the moment I have no idea how to do that.
Comment 26 Aleksey 2011-10-07 14:04:39 UTC
(In reply to comment #25)
> Replying to all of you at the same time: this is exactly what this bug and the
> attached patch for Virtuoso 6.1.3 is about. It is fixed. No need for further
> information - unless you are running a patched Virtuoso and have re-indexed the
> affected data.
> The latter may be another issue: automatically updating the affected data
> although at the moment I have no idea how to do that.

It is very important to know how to re-index information. As I understand, the e-mails are indexed only once at the time of their downloading. Will the deleting of nepomuk database help?
Comment 27 Elias Probst 2011-10-07 14:13:12 UTC
(In reply to comment #26)
> It is very important to know how to re-index information. As I understand, the
> e-mails are indexed only once at the time of their downloading. Will the
> deleting of nepomuk database help?

Deleting the Nepomuk database doesn't really help, as it will also delete all other Nepomuk data such as tags, indexed metainformation etc.

Detecting corrupted data in the index is probably not possible at all, so only re-indexing corrupted data wouldn't work.

I hope no distribution shipped 6.1.3 as stable yet and so, only beta-testers and/or developers might be affected who should be able to deal with a database reset/restore from backup.

It looks like the patch to fix this issue didn't make it yet into the Ubuntu 11.10 packages - here's an upstream bug for all affected Ubuntu beta-testers to track the status of this issue:
https://bugs.launchpad.net/ubuntu/+source/virtuoso-opensource/+bug/858970
Comment 28 Sebastian Trueg 2011-10-26 11:52:07 UTC
*** Bug 280036 has been marked as a duplicate of this bug. ***
Comment 29 Sebastian Trueg 2011-11-16 10:03:54 UTC
*** Bug 286399 has been marked as a duplicate of this bug. ***