Bug 332358

Summary: Regression: Baloo for tags messes up categories/groups when syncing with external collections (e.g. owncloud)
Product: [Applications] kaddressbook Reporter: Christian (Fuchs) <kde>
Component: generalAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED FIXED    
Severity: major CC: ach, auxsvr, b-misc, bernhard+kde, cfeck, chrysn, einmaladresse_2, ict, jjm, knut.hildebrandt, m.wege, mauspacker, maystar, meyerm, nate, onno, paulac.bur, sebastian.radish, shadow.walker, stefan.bruens, tokoe, tsforen, wolfgang
Priority: NOR    
Version: 4.13 Pre   
Target Milestone: ---   
Platform: Gentoo Packages   
OS: Linux   
Latest Commit: Version Fixed In:
Attachments: Patch for kdepimlibs
Patch for kdepim

Description Christian (Fuchs) 2014-03-20 15:35:33 UTC
Slightly related to https://bugs.kde.org/show_bug.cgi?id=331896

due to the new usage of baloo instead of nepomuk for tags  (in kaddressbook: called categories) now the akonadi tag is saved in the contact. 

Previously it was saved as: 

BEGIN:VCARD
CATEGORIES:Family
UID:0KqAvlaaZq
VERSION:3.0
END:VCARD


BEGIN:VCARD
CATEGORIES:akonadi:?tag=1
UID:1af9e6d2-f1ec-46fc-bcdc-26831c28b332
VERSION:3.0
END:VCARD

This leads to problems when syncing with other sources, e.g. owncloud and quite probably also other groupware, because they then use that as the group name. 

Reproducible: Always

Steps to Reproduce:
1. Create a new category, e.g.  "Family" 
2. Assign this category to a contact from a groupware ressource, e.g. owncloud instance
3. Let owncloud sync that with other devices, e.g. a mobile phone
Actual Results:  
The contact ends up in the new group  "akonadi:?tag=1" and is displayed as that on all synced devices. 

Expected Results:  
The contact ends up in the group "Family", as it did prior to  4.13 

Marking this as major, since most of these sources do auto-sync, hence after an update it's likely that you sync broken data to a lot of devices
Comment 1 Jonathan Marten 2014-05-07 15:14:44 UTC
Same root cause as bug 331896
Comment 2 wuselwu 2014-09-22 04:30:21 UTC
Is there still any activity on this (and the related) bugs? Kaddressbook categories (and everything depending on it, e.g. group features other software, have not been working for months, making Akonadi unusable for serious address management!
Comment 3 mau 2014-10-13 12:32:25 UTC
The same is true for me, unfortunately, using / trying to use kontact with owncloud to synchronize addresses and calendars.

It seems that there is nobody feeling responsible for the overall quality of the PIM Suite, with problems all over and some of them repeating much too often.
Comment 4 wuselwu 2014-10-13 13:53:51 UTC
Unforunately I have to agree. Since the introduction of Akonadi all KDE PIM stuff has been extremely unreliable for me, and most of the bigger and smaller problems don't seem to get any attention over the years :( . But that's another subject.

I'm just wondering why, in this case, a standardized vcard field is abused by Akonadi, breaking compatibility with all other PIM software/groupware/whatever out there, and nobody seems to be interested at all.
Comment 5 Fabian 2014-10-13 14:23:54 UTC
I agree that this behavior of akonadi/baloo makes no sense at all for me. Even if you just use akonadi based software (which is very unlikely...) the numerical IDs can't be assumed to be equal on different machines as long they do not share the same database. And if the databases are already synchronized or shared there is no need to use something like caldav for synchronization. 
From my point of view akonadi should always use names for categories and other fields when communicating to the outside world.
So please, if there's somebody out there who is familiar with the, probably huge, codebase of akonadi, baloo, korganizer... please join this discussion! This bug is ripe enough ;)
Comment 6 Jonathan Marten 2014-10-17 11:02:07 UTC
*** Bug 340048 has been marked as a duplicate of this bug. ***
Comment 7 mau 2014-10-20 09:33:09 UTC
I don't have a fix (yet), but:
Since Kontact shows the categories correctly, there is already the corresponding conversion "internal tag => tag name" in place. It's just missing where akonadi synchronizes to the outside world.

Where does the conversion take place and where the syncing? With this information, a fix should be easy.
Comment 8 wuselwu 2014-10-21 02:00:21 UTC
Honest question: Is Akonadi still actively maintained? For me the history of trying to use Akonadi is a history of always the same showstopper bugs, seemingly never to be closed...
Comment 9 mau 2014-10-21 08:06:17 UTC
Akonadi is definitively actively maintained, think of e.g. imap or caldav which reveived many bugfixes in the last couple of weeks. Also Kmail & Co. (thanks Laurent).

But who "should" be working on this bug, Laurent as well? If yes, could you please invest 5 minutes to look into this? As I wrote in comment 7, the conversion is already in place, just not applied for the synchronization to the outside world.

It would be very nice to get this fixed for Debian Jessie!
Comment 10 wuselwu 2014-10-21 08:29:54 UTC
Glad to hear that something is going on. CardDAV and CalDAV, as you mention it, is also a pain with Akonadi, making it impossible to use it with ownCloud, eGroupware and so on. I was just wondering as all those errors aren't exactly new, but some showstoppers have been there for years, literally...
Comment 11 mau 2014-10-21 08:46:29 UTC
I'm using akonadi with ownCloud, exactly to synchronize calendars and addresses across several devices; calendars are working fine since the lastest update. Also synchronizing contacts is working to some extent - even changing photos works after patching oc (just changing 2 lines in a php file). This bug is the last one I'm currently facing!
Comment 12 Christian (Fuchs) 2014-10-21 09:02:46 UTC
Talking to the developers at Randa I was told that this was going to be looked into. 

Apparently that still didn't happen, and yes, it's a shame that a bug that can lead to data loss / data corruption manages to stay for over half a year, plus that leaving baloo basically unusable in kaddressbook and korganizer ...
Comment 13 Fabian 2014-10-21 12:31:29 UTC
Nice to see that something is going on here.

I would like to discuss if it make sense at all to store this akonadi url as internal reference. The only benefit I see is the possibility to rename categories without loosing their associated elements. But this can also be achieved with some kind of refactoring. On the downside developers of each export and sync adapter have to care for the correct conversion. 
I think removing the akonadi url at all will be the faster and less error-prone solution. But I have no insight in the architecture of akonadi...
Comment 14 wuselwu 2014-10-21 15:04:47 UTC
Bug still unconfirmed?!
Comment 15 meyerm 2014-10-21 15:27:21 UTC
*** This bug has been confirmed by popular vote. ***
Comment 16 Jonathan Marten 2014-11-10 20:24:41 UTC
*** Bug 340830 has been marked as a duplicate of this bug. ***
Comment 17 wuselwu 2014-11-19 16:25:13 UTC
I don't want to insist, but... I'd like to be able to use KDE again for proper managing my contacts....
Comment 18 Thomas Serries 2014-12-28 17:28:58 UTC
Is there anything I can do to get this fixed? I'd love to manage my contacts for my mobile phone with KDE.
Comment 19 wuselwu 2014-12-28 18:50:14 UTC
Not too much activity here, as it seems. Doesn't help my general distrust in Akonadi. Unfortunately, there are no real PIM alternatives out there for Linux.
Comment 20 Thomas Serries 2015-01-05 17:21:19 UTC
I thought I could use my New Year holidays to have a look at the source to fix this issue; at least for me. (I have to admit that I do not have much experience in programming C++).

At first it seemed to be solvable by implementing new methods in KABC::Addressee that return or take a list or cleartext categories. Conversion of the cleartext categories to/from the tag urls seemed to be available by Akonadi::TagFetchJob, Akonadi::TagCreateJob and Akonadi::Tag.

But then I have seen that Akonadi depends on KABC to be compiled.

So this way is not possible as Akonadi depends ok KABC and KABC would depend on Akonadi.

Is there anybody out there who can help me breaking this cycle?
Comment 21 Thomas Serries 2015-01-05 19:03:35 UTC
Some additions: The new methods on KABC:Addressee could be used by KABC::VCardTool to export categories in cleartext to vcards or read cleartext categories from vcards and convert them to URLs of the (internally used) tags.

If I would find a fix for this issue I'd be happy to post a patch -- if it will not break anything and be "good enough" to be accepted ;-)
Comment 22 Achim Bohnet 2015-01-06 11:15:18 UTC
Hi Thomas, 
> Is there anybody out there who can help me breaking this cycle?

I would suggest to ping dvratil in #akonadi on freenode IRC.   I've talked with him about this bug already and he has some ideas how to fix it but obviously not the time.  I'm sure he can point you to the right direction.

Achim
Comment 23 Knut Hildebrandt 2015-03-16 17:18:22 UTC
Still the same here. 

As soon as I change a contact in Kaddressbook all previously (before the introduction of Baloo) added categories are deleted and akonadi:?tag=1 is assigned.

I'm running KDE 4.14.5 and Akondi 1.13.0 on an up-to-date Chakra Linux.
Comment 24 Christian (Fuchs) 2015-10-07 20:03:35 UTC
The new KF5 based contact seems to have gotten an overhaul with regards to categories of contacts and calendars. 

However, syncing them still doesn't work. 

It would be great if whatever new system that is would respect groups that you have defined in e.g. owncloud and correctly sync them, it has not been possible to correctly use groups in KDE for 1.5 years by now ...
Comment 25 Paulo 2015-11-13 13:46:38 UTC
Hi there,

as I could not manage my google contacts with Kontact, I switched to OwnCloud. Unfortunately, even if it is better than previously, there is still a problem with categories/groups : they are not managed !
Moreover I was wondering if the global interface for contacts management will evolve : I found out that the Evolution's one is better (more visual and easier to edit contacts).

If I can help, I will be glad to!

Opensuse 13.2 / KDE 4
Comment 26 Jonathan Marten 2016-05-06 12:02:30 UTC
I've done some investigation with kdepim/kdepimlibs 4.14, although it appears that the code is the same with the current KF5 version.

Currently there is no export or transfer of categories at all, at least with all of those methods that use vCards.  There is this comment in kdepimlibs./kabc/vcardtool.cpp:
    // Laurent: 31 Jan 2015. Not necessary to export it. When Categories were changed
as AkonadiTag
    // nobody thought that it was break categorie support...
    // => not necessary to export just tag...
Comment 27 Jonathan Marten 2016-05-06 12:12:43 UTC
(Continued from comment #26, accidentally submitted)

Obviously that decision was made on the grounds that it is better to export nothing than something meaningless.

In order to export user visible categories it would be necessary to do a mapping here from Akonadi tag URLs to category names.  Unfortunately KABC cannot use Akonadi to do this as it would result in a circular dependency.  One approach that I have therefore investigated is to generate a mapping between Akonadi IDs and category names at the start of the export process, and pass that all the way down to the actual vCard generation.  This means that the vCard generation only needs to use the map and doesn't need to know about or call any Akonadi stuff (although it does know about the format of an Akonadi tag URL).  It's not pretty or elegant, but it works - at least for vCard export which I have tried so far.  I'd expect that a similar approach could be taken with vCard import, although there is the complication that new Akonadi tags may have to be generated if they are not known already.

The patches following are against the KDE/4.14 branch.  I'm not submitting them for review at this stage, because current PIM effort is concentrating on the KF5 version, but they are just here for the record and in case anyone wants to make use of them.
Comment 28 Jonathan Marten 2016-05-06 12:13:23 UTC
Created attachment 98807 [details]
Patch for kdepimlibs
Comment 29 Jonathan Marten 2016-05-06 12:13:50 UTC
Created attachment 98808 [details]
Patch for kdepim
Comment 30 Rettich 2016-12-08 10:57:36 UTC
Hi,

is there any update on this? This seems to be still an issue for KF5.

The bad thing is, that Akonadi deletes categories from NextCloud/Owncloud when the contact is locally modified. Thus, even if I manage categories only in Owncloud (and devices that support them), modifying the contact from KDE means I need to re-add categories in Owncloud.

@Jonathan: Will your patch also work with KF5?
Comment 31 Rettich 2016-12-08 11:02:11 UTC
Btw: Does importing categories work as well with your patch? What about new tags not yet defined in Akonadi/Baloo?
Comment 32 Jonathan Marten 2016-12-08 20:24:45 UTC
@Rettich: It's unlikely that the patches will work wih the KF5-based kdepim - there has been a lot of moving code around and rewriting.  They will need to be ported; when I get the time and summon up enough courage to use the new kdepim, then I'll look at the fix again.
Comment 33 Bug Janitor Service 2022-01-23 19:29:18 UTC
A possibly relevant merge request was started @ https://invent.kde.org/pim/akonadi-contacts/-/merge_requests/17
Comment 34 Jonathan Marten 2022-05-11 17:53:50 UTC
Git commit 4a0e5e5e4b3f96f9464fc133c85064db22b62d5e by Jonathan Marten, on behalf of Max von Buelow.
Committed on 11/05/2022 at 17:53.
Pushed by marten into branch 'master'.

Store tags names rather than tag IDs in KContacts::Addressee.

Solves the issue that contact tags are represented as IDs in the vCard CATEGORIES field, which leads to loss of tags when using address books across multiple devices. This update uses tag names in favor of IDs and automatically creates them when necessary.

"Old" tag IDs in the database are still interpreted for backward compatibility.

See https://invent.kde.org/pim/akonadi-contacts/-/merge_requests/17

Related PRs in other projects:
* https://invent.kde.org/pim/kaddressbook/-/merge_requests/17
* https://invent.kde.org/frameworks/kcontacts/-/merge_requests/33

M  +22   -2    src/akonadi-contacts/plugins/categorieseditwidget.cpp
M  +4    -0    src/akonadi-contacts/plugins/categorieseditwidget.h

https://invent.kde.org/pim/akonadi-contacts/commit/4a0e5e5e4b3f96f9464fc133c85064db22b62d5e