Bug 283020 - Store Kmail Tags in IMAP flags, like Thunderbird
Summary: Store Kmail Tags in IMAP flags, like Thunderbird
Status: CONFIRMED
Alias: None
Product: kmail2
Classification: Applications
Component: message list (show other bugs)
Version: 5.5.1
Platform: Ubuntu Linux
: NOR wishlist with 492 votes (vote)
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
: 253492 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-09-29 08:58 UTC by Murz
Modified: 2022-03-20 22:21 UTC (History)
30 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Murz 2011-09-29 08:58:40 UTC
Version:           Git (master) (using KDE 4.7.1) 
OS:                Linux

At now Kmail Tags are stored only locally in Nepomuk tags database, so if I look mail in other computer, I lost all tags.

Thunderbird have tags system too, and it store them in IMAP flags list - http://kb.mozillazine.org/Tags

In Akonadi we can also add flags to mail objects, that it upload to IMAP correctly.

Thunderbird store those tags as $label1, $label2, etc.
So will be good to add in Kontact storing tags like thunderbird, and add support for thunderbird tags.

For do this we must sync Nepomuk tags with IMAP Flags.

Reproducible: Always



Expected Results:  
Mail flags will stored in IMAP server, so it will synced with different computers and other mail clients (thunderbird, web interface, etc).
Comment 1 David Alston 2011-11-22 06:24:09 UTC
This would be an awesome feature.  Zimbra servers use the IMAP flags field for tags and it would be wonderful if I could use the same tags in KDE as well as Zimbra.

Tags will become useful to me when I'm able to easily import/export/sync them between clients.  The IMAP protocol already provides a mechanism for it and it would simplify my workflow if Kontact was able to implement it.
Comment 2 Marcos Marado 2012-03-25 22:09:22 UTC
*** This bug has been confirmed by popular vote. ***
Comment 3 Luigi Toscano 2012-08-20 20:30:14 UTC
*** Bug 253492 has been marked as a duplicate of this bug. ***
Comment 4 Rai 2013-08-27 09:10:14 UTC
Hello, 

this one feature is a dealbreaker for me. As my company uses zimbra and we have over 300 employees in offices around the world it stops me from trying out a KDE desktop for our callcenter agents instead of a windows setup. 

Once KMail has the feature to use and manage the imap tags allready stored on the imap server and create new ones as well we can go forward and try KDE if it is usable for our agents in a pilot project.

Thank you and best regards,
Rai
Comment 5 Erik Quaeghebeur 2013-10-11 15:04:19 UTC
(In reply to comment #0)
> 
> Thunderbird store those tags as $label1, $label2, etc.
> So will be good to add in Kontact storing tags like thunderbird, and add
> support for thunderbird tags.

Just to be explicit: arbitrary tags should be synced as IMAP keywords. The Thunderbird $label1-5 keywords are a leftover of an earlier label implementation of Thunderbird, I think, or an attempt to create a number of (nonstandard) 'standard' labels (Important, Work, Home, Todo, Later) that could be made language-independent.

Also, when talking  to GMail over IMAP, take into account that GMail labels can be synced either as folders and as labels (see <https://developers.google.com/gmail/imap_extensions#access_to_gmail_labels_x-gm-labels>) in a single folder. Some people will prefer the latter behavior (as it mimicks the GMail webmail client).
Comment 6 David Alston 2014-03-15 22:59:51 UTC
This would still be awesome.  I recognize that it may take quite a bit of coding to implement something like this.

I'm pinging this just in case someone who isn't aware of this feature request might find it easier to implement now that we are in transition from Nepomuk to Baloo.
Comment 7 Stephan Diestelhorst 2014-06-19 14:19:32 UTC
It would also be nice to be able to pull out keywords from IMAP headers display them.
Comment 8 Erik Quaeghebeur 2014-11-11 15:53:42 UTC
FYI: I saw that Trojita supports IMAP tags; perhaps some code can be borrowed there?
Comment 9 Erik Quaeghebeur 2014-11-12 10:57:20 UTC
(In reply to David Alston from comment #6)
> I recognize that it may take quite a bit of coding to implement something like this.

I just looked at a mail from an IMAP mailbox in the Akonadiconsole: all the flags are there!! (In the tab ‘Internals’) The tab just to the left of it ‘Nepomuk’ (I’m on 4.12.5 still) contains the Akonadi tags. Also, the so-called IMAP system flags (\Seen, \Answered,...), which are also in this tab, are already exposed in such a way that KMail can get to them and modify them.

So, actually, this shouldn’t be a big coding task: in the same way the system flags are synced, the non-system flags could be synced to Akonadi tags. (I actually tried to solve the same issue for OfflineIMAP; see <http://article.gmane.org/gmane.mail.imap.offlineimap.general/6164> and the amount of code changes needed was adding ~12 lines and modifying ~28 lines.)

I can’t seem to modify data in the ‘Internals’ tab (read-only mode?) to add a custom flag to see if it is synced to the server; the other way around works: flags set in Trojita get synced to Akonadi (via the IMAP server).

My question to the developers is: am I corrent in thinking this is, code-wise, not a big issue? If so, what is holding this back most: lack of time, design decisions (not sure if you want to map IMAP flags to Akonadi tags), something else? What could we do to speed up implementation? (Start a small crowdfuning campaign?) Would implementing a small Akonadi agent that does this syncing (and which we can test) be more feasible than immediate integration into the main IMAP agent?
Comment 10 Erik Quaeghebeur 2014-11-14 17:06:07 UTC
(In reply to Erik Quaeghebeur from comment #9)
> [...], the
> so-called IMAP system flags (\Seen, \Answered,...), which are also in this
> tab, are already exposed in such a way that KMail can get to them and modify
> them.

And not only those, but also application ones like $WATCHED and $TODO.

Furthermore, *there is synchronization with nepomuk tags happening*: I clicked a bit on the statuses in the ‘Intelligent with clickable status’ message view theme, and at some point a ‘Gewenste post’ (Dutch for Non Junk) appeared in my tags (as viewed in the akonadiconsole), while $NonJunk appeared in the IMAP label list. This behavior was very flakey: contradictory/unsynced tags and labels existed for some time, at some point $Junk, $JUNK, $NonJunk, $NONJUNK were all present as labels... (I'm on 4.12.5 still and Nepomuk indexing/tagging is... not working that well; I hope that my distro moves to >=4.13 soon and test this more deeply.)

So: it seems almost all the functionality that is needed is there, it ‘just’ needs to be extended to user-defined tags. My question for the developers is again: am I correct in my assessment of the situation? Can we non-developers do anything to enable implementation (like a small crowdfunding campaign to pay a developer to create the necessary time needed)?
Comment 11 Murz 2014-11-17 17:08:33 UTC
I ready to support crowdfunding campaign for implementing this feature with some money too.
Comment 12 Erik Quaeghebeur 2014-11-20 12:59:15 UTC
Even though no developers have been active on this bug, they do seem to work towards is solution. I bumped into a blog post of a year ago where they mention working on this functionality:

http://www.dvratil.cz/2013/11/kde-pim-sprint-report/  “Using Akonadi to store tags”

In the mean time, if I interpret the commit logs correctly, it seems that porting to KF5 has taken the upper hand over adding functionality.
Comment 13 Steven Haigh 2015-06-11 16:42:03 UTC
Its been a good 6 months since the last comment on here. Any chance of a status update?
Comment 14 Steven Haigh 2015-06-11 16:42:25 UTC
Its been a good 6 months since the last comment on here. Any chance of a status update?
Comment 15 Vladislav Vorobiev 2016-01-06 03:11:36 UTC
My experience says that some importan't features or bugs kde team simply ignore, 
Using kontact or Kmail accross many computers seems to be not realy important.
I try to vote for it.
Comment 16 Vladislav Vorobiev 2016-01-06 03:18:12 UTC
(In reply to Vladislav Vorobiev from comment #15)
> My experience says that some importan't features or bugs kde team simply
> ignore, 
> Using kontact or Kmail accross many computers seems to be not realy
> important.
> I try to vote for it.

Oha, it works in  KMail 4.14.10 for me...
Thanks great KDE team!
Comment 17 marvin24 2016-01-06 09:45:33 UTC
just tried (also 4.14.10) and see no change. KMail does not show flags added by my Sieve script nor does it store "marks" using IMAP flags. Instead it still adds "Tags" like akonadi:?tag=5.
Comment 18 Philipp Verpoort 2016-05-07 11:30:41 UTC
Just want to express again that this would be a very useful feature.
Comment 19 Denis Kurz 2017-05-19 11:00:17 UTC
Implementing this would have a nice side effect or two. Akonadi was meant to be an offline cache for stuff like e-mail. Akonadi tags, however, add completely new functionality. If tags are stored as IMAP tags, however, Akonadi's task would then be reduced to caching again. Also, if Akonadi's db is corrupted for whatever reasons, or I decide to switch db backends, or some faulty IMAP server suddenly presents old mail with new IDs, or whatever, Akonadi would not lose all the old tags. It would only have to re-index them. This approach seems a lot less error-prone to me when compared to the current tag system.

To all reporters: Please remember that you can always make it easier for a developer to work on a bug/request by providing references to relevant standards, making detailed suggestions etc. Noone wants to work on a six year old request if she has to put a week of research into it first. In this case: Are IMAP flags even supposed to be used like this? If so, can you link to the related RFC or some other source?
Comment 20 Erik Quaeghebeur 2017-05-19 11:19:34 UTC
(In reply to Denis Kurz from comment #19)
> To all reporters: Please remember that you can always make it easier for a
> developer to work on a bug/request by providing references to relevant
> standards, making detailed suggestions etc. Noone wants to work on a six
> year old request if she has to put a week of research into it first. In this
> case: Are IMAP flags even supposed to be used like this? If so, can you link
> to the related RFC or some other source?

Please remember to read through the comments before posting something implying that the other posters have done tool little. See my comments 9, 10, and 12 to realize that the developers are fully up to speed about tags and IMAP flags. Everything needed is in Akonadi and KMail already; the work is just an extension of existing functionality. We are ready to give feedback about any remaining details as needed.
Comment 21 marvin24 2017-05-19 11:25:57 UTC
first google hit:

https://serverfault.com/questions/115769/which-imap-flags-are-reliably-supported-across-most-mail-servers

says RFC3501 Section 2.3.2. This lists the mandatory flags (\Seen, \Answered, ...), which all server implementations must implement.

But there are also "user-defined" flags "\*" (which is our topic), which are not implemented by all servers (e.g. M$ doesn't).
Comment 22 Denis Kurz 2017-05-19 11:46:20 UTC
Sorry, Eric, I didn't mean to disregard your work. After all those "thanks for your report/feedback" comments I wrote in the past, I thought I'd try a more constructive approach. It seems this wasn't the right bug for it, though.
Comment 23 Steven Haigh 2017-05-31 06:41:35 UTC
I've just gotten around to toying with kmail again in Fedora 26 Alpha.

I like it as a general client - but glaring shortcomings such as shared labels / tags are just about deal breakers.

Right now, I use roundcube mail when I'm out and about, k-9 Mail on android, and currently thunderbird on desktops (which I really want to replace with kmail) - and these 3 have no problems with the basic, well agreed tags / labels.

I have hope that one day I can check this off the list of annoyances with kmail :)
Comment 24 Erik Quaeghebeur 2018-03-21 16:38:56 UTC
I've created a crowdfunding campaign on Bountysource. Let's see if we can get together sufficient funds to entice someone who can add this feature:

https://www.bountysource.com/issues/56301087-store-kmail-tags-in-imap-flags-like-thunderbird
Comment 25 quazgar 2019-03-14 10:37:23 UTC
As of 5.7.3, the current state seems to be that Akonadi simply saves the tags in the `pimitemtagrelation` table.
Comment 26 Christo 2019-07-05 23:26:33 UTC
> Everything needed is in Akonadi and KMail already;

Akonadi doesnt suite multiseated environments and cannot be automated via tags set via sieve .. having these tags embedded in IMAP is a MUSTHAVE IMHO.

thanks for support
-c-
Comment 27 Christo 2019-07-05 23:36:37 UTC
(In reply to Erik Quaeghebeur from comment #24)
> I've created a crowdfunding campaign on Bountysource. Let's see if we can
> get together sufficient funds to entice someone who can add this feature:
> 
> https://www.bountysource.com/issues/56301087-store-kmail-tags-in-imap-flags-
> like-thunderbird

there was a neat plugin for Mail.app in IOS (back in the days) 
https://web.archive.org/web/20080413101413/http://www.indev.ca/MailTags.html

it was about 30$ that time .. i was waiting since that days for Kmail to implement such features.
Comment 28 Christo 2019-11-12 12:13:51 UTC
if this feature should one day find its way into production, there maybe some tools needed to handle tags.

1) a way to keep akonoditables in sync via control file(?) on IMAP (holding the tagattributetable and tabtable).

2) initialize a mail-storage-ressource (pop3/imap) , searching/importing tags via a slection-dialouge ? 

RFC
Comment 29 Erik Quaeghebeur 2019-11-12 17:33:48 UTC
(In reply to Christoph from comment #28)
> 1) a way to keep akonoditables in sync via control file(?) on IMAP (holding
> the tagattributetable and tabtable).
Akonadi already deals with the tags and keywords; IMHO it should just keep the tags and IMAP keywords in sync like it already does with the IMAP system flags (which are special keywords). It is just the interface in kmail that is missing.

> 2) initialize a mail-storage-ressource (pop3/imap) , searching/importing
> tags via a slection-dialouge ?
I do not understand what you are suggesting here?
Comment 30 Erik Quaeghebeur 2022-03-20 22:21:31 UTC
I've been looking at this issue again. The KMail/Akonadi codebase is still to scary for me in most places, so I tried to circumvent it and add this functionality using SQL triggers directly in Akonadi's database. I'm using the SQLite backend, but this should be similar for MySQL and Postgresql.

*TLDR: Using only SQL triggers, a flag added by another client can be synced via IMAP to an Akonadi tag, but deleting it from another client does not work. Adding and deleting from KMail does not sync to the IMAP server.*

For my experiments, I used DB browser for SQLite <https://sqlitebrowser.org/> to add triggers and look at the database; it is a bit more convenient than the functionality in Akonadiconsole. To test, I created two tags in Akonadi (via KMail's config) and these have ids 3 and 5 in the TagTable. I used one flag, with id 57 in the FlagTable. Flag 57 should be synced to tag 3. Tag 5 was used for deeper investigations. (Obviously, the trigger would need to be made flag/tag-agnostic, but this would require adding a FlagTagRelation table, which would go to far for an initial test. It would also have been a waste of time.)

Then, I created the following triggers:

1. Adding tag 3 when flag 57 arrives in the database:
---
create trigger "insert_tag_from_flag" after insert on "PimItemFlagRelation"
when NEW.Flag_Id is 57
begin
	insert or ignore into "PimItemTagRelation" values (NEW.PimItem_id, 3);
end
---
This works. The tag is added to the mail in KMail if in another client (Trojita, Thunderbird) the tag is added to the mail.

2. Adding flag 57 when tag 3 arrives in the database:
---
create trigger "insert_flag_from_tag" after insert on "PimItemTagRelation"
when NEW.Tag_Id is 3
begin
	insert or ignore into "PimItemFlagRelation" values (NEW.PimItem_id, 57);
end
---
This does not sync. The flag is added to the database, alright, but Akonadi isn't aware of it and does not propagate it to the server. ⇒ This needs to be coded in Akonadi/KMail.

3. Deleting tag 3 when flag 57 is removed in the database:
---
create trigger "delete_tag_from_flag" after delete on "PimItemFlagRelation"
when OLD.Flag_Id is 57
begin
	delete from "PimItemTagRelation" where "PimItem_id" == "OLD.PimItem_id" and "Tag_id" == 3;
	insert into "PimItemTagRelation" values (OLD.PimItem_id, 5);
end
---
This does not persist. For a moment, the tag is removed, but then it is added again, likely from another in-memory data structure. (I was keeping the mail folder containing the mail with removed flag open in KMail.) To verify that indeed the database commit works, the additional insert was added, which confirms it does. ⇒ This needs to be coded in Akonadi/KMail.

4. Removing flag 57 when tag 3 is deleted in the database:
---
create trigger "delete_flag_from_tag" after delete on "PimItemTagRelation"
when OLD.Tag_Id is 3
begin
	delete from "PimItemFlagRelation" where "PimItem_id" == "OLD.PimItem_id" and "Flag_id" == 57;
end
---
This does not sync. The flag is removed from the database, alright, but Akonadi isn't aware of it and does not propagate it to the server. ⇒ This needs to be coded in Akonadi/KMail.

So, while insightful to me, this idea doesn't really bring us closer. Into the code we must go.