Whenever I send a mail the copy that gets stored in the "Sent Items" folder is missing the last line. Reproducible: Always Steps to Reproduce: 1. Send a mail 2. Open the copy in the "Sent Items" folder Actual Results: Last line is missing Expected Results: The mail should be an exact copy of the mail that was actually sent. This only happens with one of my (online) IMAP accounts though. If I configure the identity to use a sent mail folder from another account or a local mixedmaildir resource this does not happen. The mail is sent correctly, the truncation occurs when appending the mail to the online IMAP folder. This only happens in the case of sending mails AFAICT. If I copy/move mails manually, this DOES NOT happen. I noticed this first in November, with KMail 4.11.3 and Akonadi 1.10.3. The relevant part of the imap log: C: A000083 APPEND "Sent Items" (\Seen) {446} S: + Ready for literal data C: From: Wolfgang Bauer <wbauer@tmo.at> To: wbauer1@a1.net Subject: test Date: Thu, 06 Feb 2014 11:44:22 +0100 Message-ID: <2163859.AFpIymfEBe@amiga> X-KMail-Identity: 832371678 Disposition-Notification-To: Wolfgang Bauer <wbauer@tmo.at> User-Agent: KMail/4.12.2 (Linux/3.11.6-4-desktop; KDE/4.12.2; x86_64; ; ) MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" hello kind regards, wolfi S: A000083 OK APPEND completed C: A000084 SELECT "Sent Items" S: * 47 EXISTS S: * 0 RECENT S: * OK UID validity status [ UIDVALIDITY 1348918618 ] S: * OK Predicted next UID [ UIDNEXT 236 ] S: * FLAGS ( \Answered \Flagged \Deleted \Draft \Seen ) S: * OK Permanent flags [ PERMANENTFLAGS ( \Answered \Flagged \Deleted \Draft \Seen ) ] S: A000084 OK SELECT completed [ READ-WRITE ] C: A000085 UID SEARCH HEADER Message-ID <2163859.AFpIymfEBe@amiga> S: * SEARCH S: A000085 OK UID SEARCH completed C: A000086 UID STORE +FLAGS (\Seen) S: A000086 BAD Bogus attribute list in UID STORE I set the severity to Critical since data loss is involved (although it's only the copy of the sent mail). Sorry, if that is wrong. Feel free to adjust please.
I want to add, that it's the full mail in the imap.log. But the actually stored mail misses the last line ("wolfi" in this case).
Another note: The truncation is apparently caused by that "UID SEARCH HEADER" command. If I disable that, by commenting "d->tags << d->sessionInternal()->sendCommand( command, parameters );" in StoreJob::doStart() (line 248 in kdepimlibs/kimap/storejob.cpp), the mail is NOT truncated. I think it's strange that this search wouldn't find the mail that got appended just before anyway.
(In reply to comment #0) > This only happens in the case of sending mails AFAICT. If I copy/move mails > manually, this DOES NOT happen. I just noticed, that this in fact DOES also happen when copying/moving mails to that account manually, but apparently not for every mail. At least when I use a sent mail folder on a different account (where the truncation does not occur), it does get truncated as well when I copy/move it to that account manually. > I noticed this first in November, with KMail 4.11.3 and Akonadi 1.10.3. Before that I used a folder from a local mixedmaildir resource for storing sent mail because of bug#278082, so I can't really tell if it would have happened before. I found out now that this apparently only happens if the last line doesn't include a line feed. I.e. apparently all the content after the last line feed is dropped.
Thanks for the detailed report. C: A000085 UID SEARCH HEADER Message-ID <2163859.AFpIymfEBe@amiga> S: * SEARCH S: A000085 OK UID SEARCH completed C: A000086 UID STORE +FLAGS (\Seen) S: A000086 BAD Bogus attribute list in UID STORE What we see here is that the server apparently returns an empty uidset from the search command, and kimap apprently doesn't recognize this and then tries nevertheless to UID STORE without UID (which is obviously invalid). So the imap resource should abort the process if the UID SEARCH delivers an empty result set. That doesn't explain the truncation however, and given we can see the complete message on the wire (almost on the wire), to the server, this looks like a server problem apart from that. Of course the UID SEARCH should usually also deliver a result.
Git commit 5d15dde0d9cebc191ae6805f110eabe8ca07f7d5 by Christian Mollekopf. Committed on 19/02/2014 at 16:30. Pushed by cmollekopf into branch 'master'. Protect call to last() from an empty list. M +1 -1 resources/imap/moveitemstask.cpp http://commits.kde.org/kdepim-runtime/5d15dde0d9cebc191ae6805f110eabe8ca07f7d5
Git commit 4f2fcd4cd653bd81ba475093c883acd4d25c7eff by Christian Mollekopf. Committed on 19/02/2014 at 16:50. Pushed by cmollekopf into branch 'master'. Check if the uid set is empty before passing it to STORE. M +8 -0 kimap/storejob.cpp http://commits.kde.org/kdepimlibs/4f2fcd4cd653bd81ba475093c883acd4d25c7eff
Not that this necessarily fixes the problem, but it should at least fail cleaner. For the truncated message I suggest you check the actual payload on the server. The only other thing I can think of is that the transmitted message size (446) is not big enough: A000083 APPEND "Sent Items" (\Seen) {446} Feel free to reopen if you know more.
(In reply to comment #4) > Thanks for the detailed report. Thank you for looking into this. (In reply to comment #7) > For the truncated message I suggest you check the actual payload on the > server. Sorry, I can't. This is a server from my provider ( t-mobile.at), which I have no control over. > The only other thing I can think of is that the transmitted message size > (446) is not big enough: A000083 APPEND "Sent Items" (\Seen) {446} > > Feel free to reopen if you know more. I'll leave this at WAITINGFORINFO for now. I will try with your patches tomorrow and report back. Although I don't really think they will solve the problem. As I already wrote in comment#2, removing the "UID SEARCH HEADER" command did help. I tried to remove the "UID STORE" command before that, but that still resulted in the truncation. Maybe a timing problem? (although that would point to a server problem I think) Or maybe the last line should have a line feed as well in any case? (as I said, apparently only the text after the last line feed is lost) I'm not sure what the RFC's say regarding that... >The only other thing I can think of is that the transmitted message size (446) is not big enough: A000083 APPEND "Sent Items" (\Seen) {446} I thought about this as well, but it works fine with the other servers I use. And it doesn't really seem to be related to the exact size, rather to the last line feed in the message. Anyway, I will try to do more experiments.
> > For the truncated message I suggest you check the actual payload on the > > server. > Sorry, I can't. > This is a server from my provider ( t-mobile.at), which I have no control > over. > Ok, no problem. > > The only other thing I can think of is that the transmitted message size > > (446) is not big enough: A000083 APPEND "Sent Items" (\Seen) {446} > > > > Feel free to reopen if you know more. > I'll leave this at WAITINGFORINFO for now. > > I will try with your patches tomorrow and report back. > > Although I don't really think they will solve the problem. > > As I already wrote in comment#2, removing the "UID SEARCH HEADER" command > did help. Since SEARCH is a readonly operaton and the server acknowledged the APPEND, this can only be a server problem it seems. > I tried to remove the "UID STORE" command before that, but that still > resulted in the truncation. > > Maybe a timing problem? (although that would point to a server problem I > think) > I can't imagine it since it's all sequential. > Or maybe the last line should have a line feed as well in any case? (as I > said, apparently only the text after the last line feed is lost) > I'm not sure what the RFC's say regarding that... > No newline required. > >The only other thing I can think of is that the transmitted message size (446) is not big enough: A000083 APPEND "Sent Items" (\Seen) {446} > I thought about this as well, but it works fine with the other servers I use. > And it doesn't really seem to be related to the exact size, rather to the > last line feed in the message. > It's supposed to be the exact number of bytes that are part of the transferred data. > Anyway, I will try to do more experiments. Thanks