Bug 283020

This would be an awesome feature.  Zimbra servers use the IMAP flags field for tags and it would be wonderful if I could use the same tags in KDE as well as Zimbra.

Tags will become useful to me when I'm able to easily import/export/sync them between clients.  The IMAP protocol already provides a mechanism for it and it would simplify my workflow if Kontact was able to implement it.

*** This bug has been confirmed by popular vote. ***

*** Bug 253492 has been marked as a duplicate of this bug. ***

Hello, 

this one feature is a dealbreaker for me. As my company uses zimbra and we have over 300 employees in offices around the world it stops me from trying out a KDE desktop for our callcenter agents instead of a windows setup. 

Once KMail has the feature to use and manage the imap tags allready stored on the imap server and create new ones as well we can go forward and try KDE if it is usable for our agents in a pilot project.

Thank you and best regards,
Rai

(In reply to comment #0)
> 
> Thunderbird store those tags as $label1, $label2, etc.
> So will be good to add in Kontact storing tags like thunderbird, and add
> support for thunderbird tags.

Just to be explicit: arbitrary tags should be synced as IMAP keywords. The Thunderbird $label1-5 keywords are a leftover of an earlier label implementation of Thunderbird, I think, or an attempt to create a number of (nonstandard) 'standard' labels (Important, Work, Home, Todo, Later) that could be made language-independent.

Also, when talking  to GMail over IMAP, take into account that GMail labels can be synced either as folders and as labels (see <https://developers.google.com/gmail/imap_extensions#access_to_gmail_labels_x-gm-labels>) in a single folder. Some people will prefer the latter behavior (as it mimicks the GMail webmail client).

This would still be awesome.  I recognize that it may take quite a bit of coding to implement something like this.

I'm pinging this just in case someone who isn't aware of this feature request might find it easier to implement now that we are in transition from Nepomuk to Baloo.

It would also be nice to be able to pull out keywords from IMAP headers display them.

FYI: I saw that Trojita supports IMAP tags; perhaps some code can be borrowed there?

(In reply to David Alston from comment #6)
> I recognize that it may take quite a bit of coding to implement something like this.

I just looked at a mail from an IMAP mailbox in the Akonadiconsole: all the flags are there!! (In the tab ‘Internals’) The tab just to the left of it ‘Nepomuk’ (I’m on 4.12.5 still) contains the Akonadi tags. Also, the so-called IMAP system flags (\Seen, \Answered,...), which are also in this tab, are already exposed in such a way that KMail can get to them and modify them.

So, actually, this shouldn’t be a big coding task: in the same way the system flags are synced, the non-system flags could be synced to Akonadi tags. (I actually tried to solve the same issue for OfflineIMAP; see <http://article.gmane.org/gmane.mail.imap.offlineimap.general/6164> and the amount of code changes needed was adding ~12 lines and modifying ~28 lines.)

I can’t seem to modify data in the ‘Internals’ tab (read-only mode?) to add a custom flag to see if it is synced to the server; the other way around works: flags set in Trojita get synced to Akonadi (via the IMAP server).

My question to the developers is: am I corrent in thinking this is, code-wise, not a big issue? If so, what is holding this back most: lack of time, design decisions (not sure if you want to map IMAP flags to Akonadi tags), something else? What could we do to speed up implementation? (Start a small crowdfuning campaign?) Would implementing a small Akonadi agent that does this syncing (and which we can test) be more feasible than immediate integration into the main IMAP agent?

(In reply to Erik Quaeghebeur from comment #9)
> [...], the
> so-called IMAP system flags (\Seen, \Answered,...), which are also in this
> tab, are already exposed in such a way that KMail can get to them and modify
> them.

And not only those, but also application ones like $WATCHED and $TODO.

Furthermore, *there is synchronization with nepomuk tags happening*: I clicked a bit on the statuses in the ‘Intelligent with clickable status’ message view theme, and at some point a ‘Gewenste post’ (Dutch for Non Junk) appeared in my tags (as viewed in the akonadiconsole), while $NonJunk appeared in the IMAP label list. This behavior was very flakey: contradictory/unsynced tags and labels existed for some time, at some point $Junk, $JUNK, $NonJunk, $NONJUNK were all present as labels... (I'm on 4.12.5 still and Nepomuk indexing/tagging is... not working that well; I hope that my distro moves to >=4.13 soon and test this more deeply.)

So: it seems almost all the functionality that is needed is there, it ‘just’ needs to be extended to user-defined tags. My question for the developers is again: am I correct in my assessment of the situation? Can we non-developers do anything to enable implementation (like a small crowdfunding campaign to pay a developer to create the necessary time needed)?

I ready to support crowdfunding campaign for implementing this feature with some money too.

Even though no developers have been active on this bug, they do seem to work towards is solution. I bumped into a blog post of a year ago where they mention working on this functionality:

http://www.dvratil.cz/2013/11/kde-pim-sprint-report/  “Using Akonadi to store tags”

In the mean time, if I interpret the commit logs correctly, it seems that porting to KF5 has taken the upper hand over adding functionality.

Its been a good 6 months since the last comment on here. Any chance of a status update?

Its been a good 6 months since the last comment on here. Any chance of a status update?

My experience says that some importan't features or bugs kde team simply ignore, 
Using kontact or Kmail accross many computers seems to be not realy important.
I try to vote for it.

(In reply to Vladislav Vorobiev from comment #15)
> My experience says that some importan't features or bugs kde team simply
> ignore, 
> Using kontact or Kmail accross many computers seems to be not realy
> important.
> I try to vote for it.

Oha, it works in  KMail 4.14.10 for me...
Thanks great KDE team!

just tried (also 4.14.10) and see no change. KMail does not show flags added by my Sieve script nor does it store "marks" using IMAP flags. Instead it still adds "Tags" like akonadi:?tag=5.

Just want to express again that this would be a very useful feature.

Implementing this would have a nice side effect or two. Akonadi was meant to be an offline cache for stuff like e-mail. Akonadi tags, however, add completely new functionality. If tags are stored as IMAP tags, however, Akonadi's task would then be reduced to caching again. Also, if Akonadi's db is corrupted for whatever reasons, or I decide to switch db backends, or some faulty IMAP server suddenly presents old mail with new IDs, or whatever, Akonadi would not lose all the old tags. It would only have to re-index them. This approach seems a lot less error-prone to me when compared to the current tag system.

To all reporters: Please remember that you can always make it easier for a developer to work on a bug/request by providing references to relevant standards, making detailed suggestions etc. Noone wants to work on a six year old request if she has to put a week of research into it first. In this case: Are IMAP flags even supposed to be used like this? If so, can you link to the related RFC or some other source?

(In reply to Denis Kurz from comment #19)
> To all reporters: Please remember that you can always make it easier for a
> developer to work on a bug/request by providing references to relevant
> standards, making detailed suggestions etc. Noone wants to work on a six
> year old request if she has to put a week of research into it first. In this
> case: Are IMAP flags even supposed to be used like this? If so, can you link
> to the related RFC or some other source?

Please remember to read through the comments before posting something implying that the other posters have done tool little. See my comments 9, 10, and 12 to realize that the developers are fully up to speed about tags and IMAP flags. Everything needed is in Akonadi and KMail already; the work is just an extension of existing functionality. We are ready to give feedback about any remaining details as needed.

first google hit:

https://serverfault.com/questions/115769/which-imap-flags-are-reliably-supported-across-most-mail-servers

says RFC3501 Section 2.3.2. This lists the mandatory flags (\Seen, \Answered, ...), which all server implementations must implement.

But there are also "user-defined" flags "\*" (which is our topic), which are not implemented by all servers (e.g. M$ doesn't).

Sorry, Eric, I didn't mean to disregard your work. After all those "thanks for your report/feedback" comments I wrote in the past, I thought I'd try a more constructive approach. It seems this wasn't the right bug for it, though.

I've just gotten around to toying with kmail again in Fedora 26 Alpha.

I like it as a general client - but glaring shortcomings such as shared labels / tags are just about deal breakers.

Right now, I use roundcube mail when I'm out and about, k-9 Mail on android, and currently thunderbird on desktops (which I really want to replace with kmail) - and these 3 have no problems with the basic, well agreed tags / labels.

I have hope that one day I can check this off the list of annoyances with kmail :)

I've created a crowdfunding campaign on Bountysource. Let's see if we can get together sufficient funds to entice someone who can add this feature:

https://www.bountysource.com/issues/56301087-store-kmail-tags-in-imap-flags-like-thunderbird

As of 5.7.3, the current state seems to be that Akonadi simply saves the tags in the `pimitemtagrelation` table.

> Everything needed is in Akonadi and KMail already;

Akonadi doesnt suite multiseated environments and cannot be automated via tags set via sieve .. having these tags embedded in IMAP is a MUSTHAVE IMHO.

thanks for support
-c-

(In reply to Erik Quaeghebeur from comment #24)
> I've created a crowdfunding campaign on Bountysource. Let's see if we can
> get together sufficient funds to entice someone who can add this feature:
> 
> https://www.bountysource.com/issues/56301087-store-kmail-tags-in-imap-flags-
> like-thunderbird

there was a neat plugin for Mail.app in IOS (back in the days) 
https://web.archive.org/web/20080413101413/http://www.indev.ca/MailTags.html

it was about 30$ that time .. i was waiting since that days for Kmail to implement such features.

if this feature should one day find its way into production, there maybe some tools needed to handle tags.

1) a way to keep akonoditables in sync via control file(?) on IMAP (holding the tagattributetable and tabtable).

2) initialize a mail-storage-ressource (pop3/imap) , searching/importing tags via a slection-dialouge ? 

RFC

(In reply to Christoph from comment #28)
> 1) a way to keep akonoditables in sync via control file(?) on IMAP (holding
> the tagattributetable and tabtable).
Akonadi already deals with the tags and keywords; IMHO it should just keep the tags and IMAP keywords in sync like it already does with the IMAP system flags (which are special keywords). It is just the interface in kmail that is missing.

> 2) initialize a mail-storage-ressource (pop3/imap) , searching/importing
> tags via a slection-dialouge ?
I do not understand what you are suggesting here?

I've been looking at this issue again. The KMail/Akonadi codebase is still to scary for me in most places, so I tried to circumvent it and add this functionality using SQL triggers directly in Akonadi's database. I'm using the SQLite backend, but this should be similar for MySQL and Postgresql.

*TLDR: Using only SQL triggers, a flag added by another client can be synced via IMAP to an Akonadi tag, but deleting it from another client does not work. Adding and deleting from KMail does not sync to the IMAP server.*

For my experiments, I used DB browser for SQLite <https://sqlitebrowser.org/> to add triggers and look at the database; it is a bit more convenient than the functionality in Akonadiconsole. To test, I created two tags in Akonadi (via KMail's config) and these have ids 3 and 5 in the TagTable. I used one flag, with id 57 in the FlagTable. Flag 57 should be synced to tag 3. Tag 5 was used for deeper investigations. (Obviously, the trigger would need to be made flag/tag-agnostic, but this would require adding a FlagTagRelation table, which would go to far for an initial test. It would also have been a waste of time.)

Then, I created the following triggers:

1. Adding tag 3 when flag 57 arrives in the database:
---
create trigger "insert_tag_from_flag" after insert on "PimItemFlagRelation"
when NEW.Flag_Id is 57
begin
	insert or ignore into "PimItemTagRelation" values (NEW.PimItem_id, 3);
end
---
This works. The tag is added to the mail in KMail if in another client (Trojita, Thunderbird) the tag is added to the mail.

2. Adding flag 57 when tag 3 arrives in the database:
---
create trigger "insert_flag_from_tag" after insert on "PimItemTagRelation"
when NEW.Tag_Id is 3
begin
	insert or ignore into "PimItemFlagRelation" values (NEW.PimItem_id, 57);
end
---
This does not sync. The flag is added to the database, alright, but Akonadi isn't aware of it and does not propagate it to the server. ⇒ This needs to be coded in Akonadi/KMail.

3. Deleting tag 3 when flag 57 is removed in the database:
---
create trigger "delete_tag_from_flag" after delete on "PimItemFlagRelation"
when OLD.Flag_Id is 57
begin
	delete from "PimItemTagRelation" where "PimItem_id" == "OLD.PimItem_id" and "Tag_id" == 3;
	insert into "PimItemTagRelation" values (OLD.PimItem_id, 5);
end
---
This does not persist. For a moment, the tag is removed, but then it is added again, likely from another in-memory data structure. (I was keeping the mail folder containing the mail with removed flag open in KMail.) To verify that indeed the database commit works, the additional insert was added, which confirms it does. ⇒ This needs to be coded in Akonadi/KMail.

4. Removing flag 57 when tag 3 is deleted in the database:
---
create trigger "delete_flag_from_tag" after delete on "PimItemTagRelation"
when OLD.Tag_Id is 3
begin
	delete from "PimItemFlagRelation" where "PimItem_id" == "OLD.PimItem_id" and "Flag_id" == 57;
end
---
This does not sync. The flag is removed from the database, alright, but Akonadi isn't aware of it and does not propagate it to the server. ⇒ This needs to be coded in Akonadi/KMail.

So, while insightful to me, this idea doesn't really bring us closer. Into the code we must go.

Following the requesting/supporting comments the main issue here is the lack of a syncing capabillity for message tags over different devices. This request is not about copying the way a competitive product goes but rather filling a gap which makes kmail hard to use in certain professional setups. 

Mentioned setups are e.g.:
* several employees working on the same IMAP account like two people sharing responsibility for one IMAP account 
* employees which need to switch devices regarding their place of work (stationary computer in the office and tablet for travels)
* switching responsibility for a certain IMAP account from time to time (while the next responsible person has to rely on correct/ complete tagging of the mails)  

Therefore I humbly propose to switch the title of this bugreport to better reflect the actual problem.