Bug 73758

Summary: Filter rules do not act on incoming messages in IMAP folders
Product: [Applications] kmail Reporter: Carsten Schlipf <carsten.schlipf>
Component: IMAPAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED FIXED    
Severity: wishlist CC: c.hamacher, gassauer, medisoft
Priority: NOR    
Version: 1.6   
Target Milestone: ---   
Platform: unspecified   
OS: Linux   
Latest Commit: Version Fixed In:

Description Carsten Schlipf 2004-01-29 14:51:29 UTC
Version:           1.6 (using KDE 3.2.0 RC1, compiled sources)
Compiler:          gcc version 3.2.2 20030222 (Red Hat Linux 3.2.2-5)
OS:          Linux (i686) release 2.4.20-18.9

I have an IMAP Inbox for incoming Mail and have set up an rule, that should move incoming mails from the IMAP Inbox to a local folder. This works, when manually applying the filter, but it does not automatically act on incoming new messages.

NOTE: I have found several bugreports that want to move Mail to an IMAP folder. This one is different. I want a filter to act on new messages in the IMAP Inbox folder to do something, e.g. move the message to a local folder.
Comment 1 Ingo Klöcker 2004-01-29 18:32:18 UTC
Sorry, but this is definitely a wish and no bug.

Furthermore, note that you can use a disconnected IMAP account for this. All new mail that arrives in the Inbox of a disconnected IMAP account is filtered.

Having reread wish 50997 I agree that this isn't a duplicate because 50997 is about downloading all messages from an IMAP account and filtering them into local folders (i.e. POP3 behavior for IMAP accounts). I bet that many people voting for 50997 didn't realise this.
Comment 2 Ryan Saunders 2004-02-04 05:02:54 UTC
It seems to me that this *should* be classified as a bug, since, even though the messages remain on the server with IMAP, the concept of an "incoming mail" still makes sense. So then, the "Apply to incoming messages" checkbox should behave similarly for POP and IMAP.
Comment 3 Carsten Schlipf 2004-02-04 20:40:21 UTC
Well, I also do not understnad why this is a wish. Incoming Mail is incoming Mail, why differ between IMAP and POP3?

I do not understand that while I can apply the filter rules manually to the IMAP inbox, I should not be able to apply the rules automatically. Maybe You, Ingo, could give me a good reason for that and maybe it's just a user missunderstanding from my side.

"Disconnected IMAP account"??? What is this? A IMAP account that is available offline? If so, why does it make sense, that filters are automatically applied to an disconnected IMAP account and not to a normal IMAP account?

Sorry, I just don't get it. Maybe I'm just too stupid to get it ;-). But I would like to learn the reasons for that.
Comment 4 Andreas Gungl 2004-02-04 21:00:58 UTC
Subject: Re:  Filter rules do not act on incoming messages in IMAP folders

On Mittwoch, 4. Februar 2004 20:40, carsten@yeap.de wrote:
> "Disconnected IMAP account"??? What is this? A IMAP account that is
> available offline? If so, why does it make sense, that filters are
> automatically applied to an disconnected IMAP account and not to a normal
> IMAP account?

To make it short, IMAP is an online protocoll. Your messages reside on the 
server. Now imagine you have a notebook. You leave a place with online 
access to your IMAP server, let's say you travel by plain. You would like 
to have copies of messages locally, you would like to read them or move 
them around.
Disconnected IMAP allows you to do so because all messages have copies on 
your client machine.

> Sorry, I just don't get it. Maybe I'm just too stupid to get it ;-). But
> I would like to learn the reasons for that.

Well, it's all about history and development capacity. KMail has got 
different implementations for IMAP and Disconnected IMAP. The developers 
are working to migrate to a better code base, it just takes some time.

Thanks for understanding and patience.

Comment 5 Carsten Burghardt 2004-02-04 21:05:56 UTC
Subject: Re:  Filter rules do not act on incoming messages in IMAP folders

On Wednesday 04 February 2004 20:40, carsten@yeap.de wrote:
> Well, I also do not understnad why this is a wish. Incoming Mail is
> incoming Mail, why differ between IMAP and POP3?

If you download mails with POP you have the complete email locally. With IMAP 
you only have the header and load the email on demand. That's why it's 
different: you can't operate on the complete message because you don't have.

Comment 6 Carsten Schlipf 2004-02-04 21:14:20 UTC
OK, all right. I will try disconnected IMAP.

I was just wondering, because it already works with clients like Mozilla or... ehm... Outlook (sorry for having to post that word and shamelessly comparing this with Kmail ;-) )

I know the differences between POP and IMAP. My point was just, why don't apply filters automatically to emails on a IMAP account to move them to a local folder to make them available offline, which would make "disconnected IMAP" somehow obsolete.

BTW: Thank you, Guys, for the best desktop and the best mail client, which just got even better. (For more kudos see http://www.kde-look.org/poll/index.php?poll=64)

Comment 7 Carsten Schlipf 2004-02-04 21:16:28 UTC
OK, all right. I will try disconnected IMAP.

I was just wondering, because it already works with clients like Mozilla or... ehm... Outlook (sorry for having to post that word and shamelessly comparing this with Kmail ;-) )

I know the differences between POP and IMAP. My point was just, why don't apply filters automatically to emails on a IMAP account to move them to a local folder to make them available offline, which would make "disconnected IMAP" somehow obsolete.

But OK... I see your points and it's OK with me to leave it as WISH.

BTW: Thank you, Guys, for the best desktop and the best mail client, which just got even better. (For more kudos see http://www.kde-look.org/poll/index.php?poll=64)
Comment 8 Hans Paijmans 2004-02-05 15:21:15 UTC
To make a long story short, I will not use Kmail as long
as it can not automatically filter spam in an IMAP account.
And I am relatively certain that many KDE users will feel
the same.

As I have said on many occasions, this is a Bad Thing (TM),
because instead I will use Mozilla, and then why bother 
with Konqueror and other good KDE apps? Worse:
I will also miss the integration of Kmail and other
KDE apps, all because the Kmail developers bluntly
refuse do move with the times.

Paai
Comment 9 Carsten Burghardt 2004-02-05 18:50:23 UTC
Subject: Re:  Filter rules do not act on incoming messages in IMAP folders

On Thursday 05 February 2004 15:21, Hans Paijmans wrote:
> Worse:
> I will also miss the integration of Kmail and other
> KDE apps, all because the Kmail developers bluntly
> refuse do move with the times.

THIS is not correct. We don't refuse to implement this but our capacity is 
limited. Did you ever consider that we're doing this stuff in our spare time?
Apart from that it's on the feature list for 3.3


Carsten

Comment 10 Carsten Schlipf 2004-02-10 10:38:54 UTC
I also think that the developers are working hard in their free time. I also would like to contribute, but simply cannot afford it. So thank you very much.

Back to the topic: Disconnected IMAP does not really work for me. Altough it allows to subscribe/unsubscribe to folders these settings are ignored and I see always all folders (since I use IMAP to access an Exchange box, there are a lot of folders that do not make sense, like Calendar, Tasks, Contacts,...).

However since Kmail does not indicate, to which folders new mail was delivered (NEW mail, not UNREAD mail), I begin to like the manual filtering as an feature. I see all new mails in the IMAP inbox folder and when reading or ignoring new mails, I simply press ALT+A to select all messages and ALT+J to apply the filters, to sort all messages in their folders.

However it still would be better to have an indicator that shows, where new mail has gone to (see #54754), and automatically apply filters to the IMAP inbox.
Comment 11 Ferdinand Gassauer 2004-06-14 09:42:49 UTC
this sort of filtering should be done on the IMAP server. Else identical filter rules have to be set up on _all_ access points (Company, Home, ...)
Comment 12 Carsten Burghardt 2004-06-14 10:00:30 UTC
*** Bug 83336 has been marked as a duplicate of this bug. ***
Comment 13 elbows 2004-06-14 18:59:42 UTC
Is there a standard way to install filters on an IMAP server? If so, KMail should provide an interface to it (perhaps transparently).
If not, KMail ought to filter incoming IMAP messages, since the server may not support it.
Comment 14 Carsten Schlipf 2004-06-14 21:58:59 UTC
Hi Ferdinand,

filters on the IMAP server are a different story. These filters can be used to move mail to another IMAP folder on the same server. So you can have your mails sorted before you download it.

If you read the whole thread, this defect here deals with filters between folders on the IMAP server and local folders. E.g. if you have a notebook with no network connection, you may want to set up a filter to move/copy all your mails to a local folder, so that these mails are available offline.
Comment 15 Johannes 2004-07-28 14:51:37 UTC
I strongly comply with comment #6. As said there, why don't apply filters automatically to emails on a IMAP account to move them to a local folder to make them available offline, which would make "disconnected IMAP" somehow obsolete. The affected mails could still reside on the server, but in addition, I get a local copy to the destination folder of my filter.
Comment 16 Andreas Gungl 2004-07-28 15:33:48 UTC
On Wednesday 28 July 2004 14:51, Johannes wrote:
> I strongly comply with comment #6. As said there, why don't apply
> filters automatically to emails on a IMAP account to move them to a local
> folder to make them available offline, which would make "disconnected
> IMAP" somehow obsolete. The affected mails could still reside on the
> server, but in addition, I get a local copy to the destination folder of
> my filter.

If you think you can't live without that feature, why don't you come up with 
a patch?
It was said that there is a lack of man power to implement this 
functionality (comment #9) and I really don't understand why people can't 
accept that reason.


IMHO you don't understand the philosophy behind disconnected IMAP. DIMAP is 
like a local snapshot of your messages on your IMAP server. You can copy or 
move messages between the "virtual" IMAP folders. As soon as you sync 
again, your changes will be propagated to the server.

Example: You initially sync your DIMAP account, then you leave the office. 
In the train there you read those two messages. You decide to move one into 
the trash (actually located on the IMAP server) and to move the other from 
"your projects main" folder to the subfolder "your projects change 
requests". After you have arrived in the subsidary office you sync your 
DIMAP account again. The new positions of these two messages (trash and 
project subfolder) are sent to the server. Now you can use any online IMAP 
client to see the messages at their new postion in the folders you moved 
them into.

DIMAP is not at all about filtering but usually working offline and sync 
with the server from time to time.

In contradiction to the scenario described above, if you move the messages 
to local folders you can't see them on the server any longer. You have them 
on your single machine while with DIMAP you only have a copy of the message 
(and ever have it on the server too).
As a question remains why using IMAP when following this workflow.

Comment 17 Johannes 2004-07-28 16:30:47 UTC
The scenario which I am talking about is as follows: I read emails from 
different locations: notebook while travelling, public computer via web while 
on conferences and from my office using KMail. In no case I want to use POP3 
because many mails have to be accessible from office and home, for example. 
But to keep my inbox clean, I have a multitude of mail filters. In Mozilla, 
these would be applied automatically to incoming messages which means they 
are moved to local folders and thereby deleted from the server which is 
completely ok. Isn't it possible to have this behaviour in KMail, too? I 
don't want to change to Mozilla because I like Kontact and KDE in general 
very much.

Comment 18 Don Sanders 2004-07-29 12:12:34 UTC
As Andreas said we have limited man power.

I am working on this feature and it's at the top of my commercial TODO 
list ( http://kontact.org/shopping ) but it's not possible for me to 
complete it before 3.3 now that we are in feature freeze.

Don.

Comment 19 Andreas Gungl 2004-07-29 15:57:17 UTC
On Wednesday 28 July 2004 16:30, Johannes wrote:
> The scenario which I am talking about is as follows: I read emails from
> different locations: notebook while travelling, public computer via web
> while on conferences and from my office using KMail. In no case I want to
> use POP3 because many mails have to be accessible from office and home, for
> example. But to keep my inbox clean, I have a multitude of mail filters. In
> Mozilla, these would be applied automatically to incoming messages which
> means they are moved to local folders and thereby deleted from the server
> which is completely ok. Isn't it possible to have this behaviour in KMail,
> too? I don't want to change to Mozilla because I like Kontact and KDE in
> general very much.

If you really want to filter into *local* folders (and *not* into IMAP 
folders), you can use a DIMAP account. Filtering from (only) the IMAP inbox 
folder into local folders works. I'm using it daily.

Filtering into other IMAP folders does not work. Filtering of messages in 
other folders then the IMAP inbox also does not work.

Regards,
Andreas

Comment 20 Don Sanders 2005-04-12 10:01:27 UTC
I have written a patch that I think implements the functionality specified in this bug/wish and attached it to bug 50997.

Please see bug 50997 for more information. I plan to close this bug when the patch is committed to CVS.

Thanks very much to everyone who has supported the commercial improvement system and hence supported the development of this patch.

Don Sanders
http://kontact.org/shopping/
Comment 21 Don Sanders 2005-04-29 04:13:53 UTC
CVS commit by sanders: 

Enhance the Kmail filtering system so that filters can be applied to
incoming messages from IMAP accounts, that is messages that are delivered
into an IMAP inbox folder.

Sundry details:
  I've modified these files:

  kmfilterdlg.h kmfilterdlg.cpp
   To add the advanced tab that allows per account filters

  kmfilter.h kmfilter.cpp
   I've added some new functions to KMFilter
     setApplicability( AccountType )
     AccountType applicability()
     setApplyOnAccount( uint id, bool aApply )
     bool applyOnAccount( uint id )
   and an enum
     enum AccountType { All, ButImap, Checked }

  kmfiltermgr.h kmfiltermgr.cpp
    I modified KMFilterMgr::process
    to take two optional parameters
      bool account = false
      uint accountId = 0
    and changed the implementation so that when these
    params are set per account filtering is applied.

  kmaccount.cpp
    Trivially to use the new per account filtering.

  kmacctimap.h kmacctimap.cpp
    For 3 main reasons
    1) In the case (it checks) that a filter applies to this
       imap account, then incoming mails headers are
       downloaded when mail checking occurs. Rather than waiting
       until the folder is selected in the GUI.
       
       Because it's really silly to have to select the inbox 
       folder to manually force filtering to kick in.

    2) To load/save a list of unfiltered messages in the 
       constructor/destructor respectively.

    3) To filter messages (I've sort of duplicated a little code 
       from kmheaders.cpp for this, my apologies).

  kmfolderimap.cpp
    1) To tell mAccount to filter new mail as the headers arrive.

This is basically the patch I've sent to bug:50997 several times and
has been available for testing for some time.

I'll now focus on allowing IMAP folders to be the target of the 
Move Into Folder filter action, and less blocking filtering
(useful for piping into a spam tool).
CCMAIL:73758-done@bugs.kde.org


  M +1 -1      kmaccount.cpp   1.123
  M +123 -1    kmacctimap.cpp   1.194
  M +14 -0     kmacctimap.h   1.93
  M +65 -2     kmfilter.cpp   1.73
  M +44 -0     kmfilter.h   1.35
  M +154 -28   kmfilterdlg.cpp   1.122
  M +12 -2     kmfilterdlg.h   1.42
  M +6 -3      kmfiltermgr.cpp   1.76
  M +5 -1      kmfiltermgr.h   1.34
  M +5 -0      kmfolderimap.cpp   1.258
Comment 22 Carsten Burghardt 2005-05-21 13:07:04 UTC
*** Bug 43705 has been marked as a duplicate of this bug. ***
Comment 23 Mats Ahlgren 2006-01-03 05:57:53 UTC
Has the patch been applied to the latest release? I'm still experiencing this problem with the current release in the Ubuntu apt-get packages.
Comment 24 Andreas Gungl 2006-01-03 06:41:42 UTC
Am Dienstag, 3. Januar 2006 05:57 schrieb Mats Ahlgren:
> Has the patch been applied to the latest release? I'm still experiencing
> this problem with the current release in the Ubuntu apt-get packages.


The patch has been applied. There have still been some minor problems with 
external filter programs in KDE 3.5.0 (KMail 1.9) which has been fixed for 
KDE 3.5.1 (KMail 1.9.1).
If it doesn't work at all for you, please double check your configuration.