Bug 125084 - Do not move/re-upload messages where no filter applies
Summary: Do not move/re-upload messages where no filter applies
Status: RESOLVED WORKSFORME
Alias: None
Product: kmail
Classification: Applications
Component: IMAP (show other bugs)
Version: 1.9.1
Platform: unspecified Linux
: NOR grave with 220 votes (vote)
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
: 119553 121230 128036 134802 144988 145162 148010 171563 172111 (view as bug list)
Depends on:
Blocks:
 
Reported: 2006-04-07 01:42 UTC by Daniel Hahler
Modified: 2009-12-26 23:24 UTC (History)
18 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
A screenshot, displaying the re-uploaded mails (but not the described end result) (79.74 KB, image/png)
2006-04-07 02:11 UTC, Daniel Hahler
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Hahler 2006-04-07 01:42:32 UTC
Version:           1.9.1 (using KDE 3.5.2, Kubuntu Package 4:3.5.2-0ubuntu1 dapper)
Compiler:          Target: i486-linux-gnu
OS:                Linux (i686) release 2.6.16

When I get huge mails (~6MB), with pictures for example, it takes a lot of time, until KMail is finished filtering those, because they first get downloaded (the whole mail!) and then re-uploaded again to "inbox".

I think that KMail should not download the whole mail, if
 - there are only header-type filters (as with me)
 - or if the mails are bigger than a defined max-value

Even if the whole message would get downloaded to check for a "body-type" filter, it should not get re-uploaded to the same folder again. Why not just leave the original one there?

I often also experience that when viewing my inbox through Webmail, that there are all the last mails appearing two times: once "normal" and again with a strike-through. I think this is a related "problem".
Comment 1 Daniel Hahler 2006-04-07 01:43:26 UTC
Sorry, forgot the "header": it's an IMAP online account
Comment 2 Daniel Hahler 2006-04-07 02:09:37 UTC
The "mails-get-doubled" effect is now just happening with those huge mails:
KMail is downloading them all again and re-uploading them, but leaving the old one also there.

This time, out of 4 mails I got 12 - each being ~7MB.

While I've prepared a screenshot of it, KMail kept downloading and re-uploading the mail with subject "bilda1" in the screenshot, where I know have 6 of. "bildas" have been two mails originally (with the same subject), but one of them has been three-plicated. "bilda2" and "bilda3" are only present once.

I've enabled the "Filter log" during this process, but it always says "0 =" for every rule, which I think means "0 matches"?
Comment 3 Daniel Hahler 2006-04-07 02:11:18 UTC
Created attachment 15501 [details]
A screenshot, displaying the re-uploaded mails (but not the described end result)
Comment 4 Daniel Hahler 2006-10-31 23:07:48 UTC
I've just had happening it again. A mail with a ~2MB attachment got downloaded and uploaded endlessly.
Before this happend, I had about 250 duplicated small mails, which I deleted and after KMail did not show some new mails, restarted it (and the issue with the ~2MB mail appeared).

I had to close KMail and remove the index files with:
  find ~/.kde/share/apps/kmail/imap \( -name *.index -o -name *.index.ids \)  -print0 | xargs -0 /bin/rm
to fix it.

So, there are actually two separate bugs in this report:
- KMail should not download and re-upload files, if there are no body type filters
- Somehow the filtering mechanism gets into an endless loop and keeps duplicating existing/new mails!
Comment 5 YeHaa 2006-11-15 16:53:53 UTC
I'd modify your wishes like this:
- KMail should only download the whole mail if there are body-type filters
- KMail should _never_ delete the mail if no filter applies
- KMail should delete the mail only if it is filtered to a local folder but not if it's filtered to another folder on the server (is that even possible?)

The current behaviour keeps me from using KMail at home as well as at work. That's because the reuploading will cause the "other" machine to think this mail is new. So it will run all filters on it again. Thereby reuploading the mail. Now the first machine thinks it's new, and so on. 

Setup: - Courier IMAP-Server
       - KMail 1.9.5
       - Filters: only the rules for using spambayes as spam-filter
Comment 6 mchugh19@yahoo.com 2006-11-17 19:17:22 UTC
*** This bug has been confirmed by popular vote. ***
Comment 7 Daniel Hahler 2007-04-09 20:52:38 UTC
Another issue which seems to be related is that a filtered mail, which does not get moved anywhere else (and therefor stays in the inbox) gets removed at the current location in the message list and moved to the top of the list - at least when ordering by "Date (Order of Arrival)".
Comment 8 Thomas McGuire 2007-05-05 22:23:11 UTC
*** Bug 121230 has been marked as a duplicate of this bug. ***
Comment 9 Thomas McGuire 2007-05-05 22:23:28 UTC
*** Bug 144988 has been marked as a duplicate of this bug. ***
Comment 10 Thomas McGuire 2007-05-05 22:25:59 UTC
Raising Severity due to duplicate bug 144988: Canceling filtering does not re-upload the message and thus it is lost (same issue basically)
Comment 11 Thomas McGuire 2007-05-07 22:30:06 UTC
See also bug 134802.
Bug 95064 is probably also a consequence of this.
Comment 12 Bram Schoenmakers 2007-05-24 09:54:38 UTC
*** Bug 119553 has been marked as a duplicate of this bug. ***
Comment 13 Thomas McGuire 2007-07-09 19:03:07 UTC
Could also be the same as bug 128036.
Comment 14 Daniel Hahler 2007-07-09 20:45:27 UTC
Yes, bug 128036 is the same.
Comment 15 Thomas McGuire 2007-07-10 17:04:16 UTC
*** Bug 128036 has been marked as a duplicate of this bug. ***
Comment 16 Thomas McGuire 2007-07-19 21:41:46 UTC
*** Bug 148010 has been marked as a duplicate of this bug. ***
Comment 17 Thomas McGuire 2007-07-19 21:43:09 UTC
*** Bug 134802 has been marked as a duplicate of this bug. ***
Comment 18 Thomas McGuire 2007-07-19 21:46:19 UTC
*** Bug 145162 has been marked as a duplicate of this bug. ***
Comment 19 Thomas McGuire 2007-08-12 13:07:39 UTC
See also bug 121866, might be the same.
Comment 20 Lari Temmes 2007-10-04 19:12:06 UTC
I really hope this gets fixed soon.
Comment 21 Jaka Kranjc 2008-03-02 10:17:09 UTC
Indeed, this behaviour is highly annoying.
Comment 22 Marton Drotos 2008-10-30 11:50:29 UTC
I think the duplication is because of the synchronization/locking problem described in the other bug report (https://bugs.kde.org/show_bug.cgi?id=95064), and it arises in the following condition:

slow up/download speed, automatic check for new mail in relatively short periods:

Large message is downloaded, deleted on the server, filtered, then uploaded again (I'm not sure when exactly the deletion takes place). Before the mail is effectively deleted a new check starts, starting the new procedure again. The bandwidth demand duplicates, making the problem even worse, and this continues infinitely.

Another annoying, related behavior:

When unfiltered/uncached mails are found in an IMAP folder, they start disappearing from the list as KMail downloads and deletes them. They become visible again only when the whole filtering process is finished. (I think this is a KMail for KDE4 issue only).

System: Debian unstable/experimental, KMail 4.1.2-1 (Debian versioning)
Comment 23 Adam Porter 2008-10-30 16:46:06 UTC
"When unfiltered/uncached mails are found in an IMAP folder, they start disappearing from the list as KMail downloads and deletes them. They become visible again only when the whole filtering process is finished. (I think this is a KMail for KDE4 issue only)."

No, I've experienced that on KDE3 KMail for years.
Comment 24 Henk Spaaij 2008-10-31 04:18:00 UTC
Yep, its the case in KDE-3.5 as well. IT has prohibited the use of kmail as messages from shared mailboxes kept disappearing.
Comment 25 Wyatt Best 2009-01-14 21:43:46 UTC
This download-upload behavior has has caused me to lose an entire folder's contents (it just had to be 'Archive'). KMail was happily looping and duplicating when I finally got fed up and started pressing cancel on every single item in in the queue pop-up. KMail crashed and the contents of the folder vanished in the meantime.

This whole concept of download-reupload is idiotic at best. I was pleasantly surprised at KMail + KDE4's nice interface and was 'trying it out' as a Thunderbird replacement and added my IMAP accounts. Now I have to try and get email backups from a shared host company. An email app with a bug this serious should not be in a production release.
Comment 26 Wyatt Best 2009-01-14 23:51:06 UTC
(In reply to comment #25)
I apologize for the false alarm. Turns out I did not actually lose data; I had already moved the archived mails to a local machine for safekeeping. However, I did experience the download/upload/duplicate loop until I removed all message filters.
Comment 27 Alex 2009-02-02 08:31:35 UTC
The reupload behaviour is very annoying. It seems, kmail adds one or two headers to each Mail? Is there some way to disable this? I even do not like it very much, if a Mailprogram alters existing Mail. Who need stuff like X-Length, if the length could be determined at display time?
Comment 28 Murz 2009-05-05 08:22:16 UTC
Confirm on Kmail version 1.11.2 with KDE 4.2.2.
After entering a folder of a new IMAP account with many emails, I spend a lot of time when Kmail load the folder!
1. Kmail download all messages with body and attachments
2. After that I see that all messages removes from the list on display one-by-one.
3. Kmail uploads all messages again to account and I see that all messages return to list one-by-one.

I can't found how I can disable this. I remove all filters and try to disable Anti-spam bogofilter (how can I totally disable it?) but it isn't help.
Comment 29 Björn Ruberg 2009-12-12 01:14:38 UTC
*** Bug 172111 has been marked as a duplicate of this bug. ***
Comment 30 Björn Ruberg 2009-12-12 01:15:35 UTC
*** Bug 171563 has been marked as a duplicate of this bug. ***
Comment 31 Björn Ruberg 2009-12-12 01:17:38 UTC
This issue long kept me away from using IMAP in kmail. It has been fixed later in the 4.2 line (4.2.3?). I'm sure it is fixed because I use kmail for IMAP for over half a year from now.
Comment 32 Björn Ruberg 2009-12-26 23:24:05 UTC
Please reopen this bug when you can reproduce this problem - from my experience it is fixed.