Bug 44378

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wednesday 26 June 2002 14:37 matthias.grossmann@epost.de wrote:
> I noticed that KMail appends a line to the mails in my sent-mail
> folder (mbox) which contains the last letter of the mail one or more
> times. The boundary between two mails in ~/Mail/sent-mail then looks
> like:
>
> -----------------------------------------------------------------
>  Matthias
> sssFrom matthias.grossmann@info...rt.de Mon Jun 10 13:28:08 2002
> -----------------------------------------------------------------
[snip]
> deleting .sent-mail* and restarting KMail. The mails are displayed
> correctly thereafter but KMail reinserts those extra letters
> probably when appending new mails. This firstly affects only the most
> recent mail (the one directly preceeding the newly appended mail)
> but on subsequent additions of new mails more and more old mails are
> affected. Sometimes the extra letters do not directly proceed the
> "From" but appear on a seperate line. In any case the blank line
> between the two mails is removed.
>
> I also removed all mails from sent-mail and deleted the index-files
> but I still have this problem after sending a few mails.
>
> I experienced this problem using KDE 2.2.2 too.

Do you compact your folders on program exit? Do you wait until KMail 
finished folder compaction before you logout? Is only the sent-mail 
folder affected? Do any other programs access the sent-mail file?

As workaround you could convert your sent-mail folder from mbox to 
maildir. (This conversion has to be done by hand.)

Regards
Ingo

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE9GkNnGnR+RTDgudgRAsIvAJ9s471jwq/CuE260Fbt+YUsL+PJNgCeMxm/
k4RTVtcmreXwlfRPZwm3png=
=SVKZ
-----END PGP SIGNATURE-----

> Do you compact your folders on program exit?=20

Yes.

> Do you wait until KMail finished folder compaction before you logout?=20

I wait until KMail has closed its window is this long enough?

> Is only the sent-mail folder affected?=20

Several months ago a folder I created myself was also affected probably=
=20
after moving mails from sent-mail to this folder but I was not able to=20
reproduce this. Other folders are not affected.

> Do any other programs access the sent-mail file?

I tried once to access my mail with Outlook and KMail (Linux/i386) via IMAP=
=20
but this was long before I started to investigate this problem and cleared=
=20
the sent-mail folder.

I'm editing the sent-mail file from time to time to remove those extra lett=
ers=20
as I wrote in the bug report.

> As workaround you could convert your sent-mail folder from mbox to
> maildir. (This conversion has to be done by hand.)

Ok I'll give this a try.

Thanks
 Matthias

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thursday 27 June 2002 09:09 Matthias Grossmann wrote:
> > Do you wait until KMail finished folder compaction before you
> > logout?
>
> I wait until KMail has closed its window is this long enough?

No. KMail still compacts the folder when the windows are closed. This 
can take up to several minutes if very large folders (sent-mail is 
often a large folder) have to be compacted.

The next version of KMail will display a dialog while folder compaction 
takes place.

Regards
Ingo

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE9G53QGnR+RTDgudgRAvHKAJ9Znu+SuEuuGOQYLzfkgyGnrpVPkwCfVZAE
OElqb8PdfAa7lPxKpoKkKZw=
=VAqi
-----END PGP SIGNATURE-----

> No. KMail still compacts the folder when the windows are closed. This
> can take up to several minutes if very large folders (sent-mail is
> often a large folder) have to be compacted.

My sent-mail folder currently contains four messages (~6KB).

I made some additional tests today. This bug is definitly related to append=
ing=20
mails to the folder I was able to reproduce it without compacting. I can=
=20
send copies of the sent-mail folder and a more detailed description of what=
 I=20
did if that helps.

Regards
 Matthias

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Friday 28 June 2002 10:11 Matthias Grossmann wrote:
> > No. KMail still compacts the folder when the windows are closed.
> > This can take up to several minutes if very large folders
> > (sent-mail is often a large folder) have to be compacted.
>
> My sent-mail folder currently contains four messages (~6KB).
>
> I made some additional tests today. This bug is definitly related to
> appending mails to the folder I was able to reproduce it without
> compacting. I can send copies of the sent-mail folder and a more
> detailed description of what I did if that helps.

Yes. Please do so.

Regards
Ingo

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE9HL4TGnR+RTDgudgRAthuAJsEQd6u29oH6Kfm+OsOuGX+WzdLmwCdGy+Z
RaozCgWoeBVuJdOXzJ2nVx8=
=tH75
-----END PGP SIGNATURE-----

> Yes. Please do so.

I started this test with an empty sent-mail folder. I sent three mails=20
without compacting folders or leaving KMail in between. After sending the=
=20
first mail the sent-mail folder contains:

---------------------------------------------------
From matthias.grossmann@informatik.uni-stuttgart.de Fri Jun 28 09:36:46 2002
Content-Type: text/plain;
  charset=3D"us-ascii"
From: Matthias Grossmann <matthias.grossmann@informatik.uni-stuttgart.de>
To: Matthias Grossmann <grossmms@assun6>
Subject: Test
Date: Fri 28 Jun 2002 09:36:46 +0200
X-Mailer: KMail [version 1.4]
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Message-Id:=20
<200206280936.47070.matthias.grossmann@informatik.uni-stuttgart.de>
Status: RO
X-Status: S

Ein Test

---------------------------------------------------=20

After sending the second mail:

---------------------------------------------------
From matthias.grossmann@informatik.uni-stuttgart.de Fri Jun 28 09:36:46 2002
Content-Type: text/plain;
  charset=3D"us-ascii"
From: Matthias Grossmann <matthias.grossmann@informatik.uni-stuttgart.de>
To: Matthias Grossmann <grossmms@assun6>
Subject: Test
Date: Fri 28 Jun 2002 09:36:46 +0200
X-Mailer: KMail [version 1.4]
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Message-Id:=20
<200206280936.47070.matthias.grossmann@informatik.uni-stuttgart.de>
Status: RO
X-Status: S

Ein Test

From matthias.grossmann@informatik.uni-stuttgart.de Fri Jun 28 09:38:15 2002
Content-Type: text/plain;
  charset=3D"us-ascii"
From: Matthias Grossmann <matthias.grossmann@informatik.uni-stuttgart.de>
To: Matthias Grossmann <grossmms@assun6>
Subject: Test 2
Date: Fri 28 Jun 2002 09:38:15 +0200
X-Mailer: KMail [version 1.4]
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Message-Id:=20
<200206280938.15980.matthias.grossmann@informatik.uni-stuttgart.de>
Status: RO
X-Status: S

Zweiter Test
---------------------------------------------------

This still seems to be ok but I noticed that there is no blank line at the=
=20
end of the second mail. After sending the third mail the last letter of th=
e=20
second mail is duplicated:

---------------------------------------------------
From matthias.grossmann@informatik.uni-stuttgart.de Fri Jun 28 09:36:46 2002
Content-Type: text/plain;
  charset=3D"us-ascii"
From: Matthias Grossmann <matthias.grossmann@informatik.uni-stuttgart.de>
To: Matthias Grossmann <grossmms@assun6>
Subject: Test
Date: Fri 28 Jun 2002 09:36:46 +0200
X-Mailer: KMail [version 1.4]
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Message-Id:=20
<200206280936.47070.matthias.grossmann@informatik.uni-stuttgart.de>
Status: RO
X-Status: S

Ein Test

From matthias.grossmann@informatik.uni-stuttgart.de Fri Jun 28 09:38:15 2002
Content-Type: text/plain;
  charset=3D"us-ascii"
From: Matthias Grossmann <matthias.grossmann@informatik.uni-stuttgart.de>
To: Matthias Grossmann <grossmms@assun6>
Subject: Test 2
Date: Fri 28 Jun 2002 09:38:15 +0200
X-Mailer: KMail [version 1.4]
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Message-Id:=20
<200206280938.15980.matthias.grossmann@informatik.uni-stuttgart.de>
Status: RO
X-Status: S

Zweiter Test
t
From matthias.grossmann@informatik.uni-stuttgart.de Fri Jun 28 09:41:25 2002
Content-Type: text/plain;
  charset=3D"us-ascii"
From: Matthias Grossmann <matthias.grossmann@informatik.uni-stuttgart.de>
To: Matthias Grossmann <grossmms@assun6>
Subject: Test 3
Date: Fri 28 Jun 2002 09:41:25 +0200
X-Mailer: KMail [version 1.4]
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Message-Id:=20
<200206280941.25459.matthias.grossmann@informatik.uni-stuttgart.de>
Status: RO
X-Status: S

Dritter Test

---------------------------------------------------

Regards
 Matthias

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Monday 01 July 2002 09:05 Matthias Grossmann wrote:
> > Yes. Please do so.
>
> I started this test with an empty sent-mail folder. I sent three
> mails without compacting folders or leaving KMail in between. After
> sending the first mail the sent-mail folder contains:
>
> ---------------------------------------------------
[snip]
> X-Status: S
>
> Ein Test
>
> ---------------------------------------------------

Same here. To be precise I didn't enter a newline after "Ein Test". 
Therefore KMail automatically added two line feeds.

> After sending the second mail:
>
> ---------------------------------------------------
[snip]
> X-Status: S
>
> Ein Test
>
> From matthias.grossmann@informatik.uni-stuttgart.de Fri Jun 28
> 09:38:15 2002
[snip]
> X-Status: S
>
> Zweiter Test
> ---------------------------------------------------

> This still seems to be ok but I noticed that there is no blank line
> at the end of the second mail.

Here there is a blank line at the end of the second message.

> After sending the third mail the last letter of the second mail is
> duplicated:
>
> ---------------------------------------------------
[snip]
> X-Status: S
>
> Zweiter Test
> t
> From matthias.grossmann@informatik.uni-stuttgart.de Fri Jun 28
> 09:41:25 2002
[snip]
> X-Status: S
>
> Dritter Test
>
> ---------------------------------------------------

Here everything is o.k. And the source code also looks correct. I fear 
that your problem is caused by a strange incompatibility of Solaris. As 
I don't have access to a Solaris machine I can't track this bug down.

You have two alternatives:
1.) You compile KMail yourself and try to hunt down this bug. The 
relevant code is in kmfoldermbox.cpp in the function 
KMFolderMbox::addMsg(...)

2.) You convert your sent-mail folder (and just to be on the safe side 
also all other folders) from mbox format to maildir format. 
Unfortunately this has to be done manually. The FAQ describes how to do 
it.

Regards
Ingo

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE9ILPyGnR+RTDgudgRAimsAKCzz437XGZNzjmVvJeD763lonNXMACfb9hW
Tyy3O3L0wGeu/UOt70EftwM=
=C6cb
-----END PGP SIGNATURE-----

Matthias, are you still seeing this with kmail 1.5.2? 

Subject: Re:  sent-mail folder corruption when appending mails

> Matthias, are you still seeing this with kmail 1.5.2?

Sorry, but I'm still using kmail 1.4. The installation of kmail I discovered 
the bug in is the one I'm using at work, so I have to wait until the 
infrastructure department upgrades to a new version.

Regards,
 Matthias Grossmann

Subject: Re:  sent-mail folder corruption when appending mails

> ------- Additional Comments From till@adam-lilienthal.de  2003-06-07 21:06
> ------- Matthias, are you still seeing this with kmail 1.5.2?

Ok, I' now using KDE 3.1.2 and KMail 1.5.2 and I'm still seeing this bug. I 
had a look at the source (kmfoldermbox.cpp) and noticed some comments about 
problems with NFS. I didn't mention in my previous mails that my mail folders 
are actually located on a NFS share, so this may be a NFS problem.

Regards,
 Matthias Grossmann

Matthias, thanks for following up on this. I'll add the keywords solaris and nfs to the description, which is all we can do, at present, since without access to that kind of setup, the bug is not reproducible for us. If you have time to investigate this, we'd appreciate it.

I just reviewed Debian's BTS, and I think there is a similar problem reported there against recent builds:

http://bugs.debian.org/280287

Ok. Here's the connection from the debian bug.
First I have to admit that the last version I tested was the
Version from kdepim 4:3.3.1-2 (i386 source all)
created on [2004-11-21]
http://packages.qa.debian.org/k/kdepim.html

My temporary fix works ok for now. First thing I'll test on monday
is if problem exists in current debian package:
[2005-01-19] Accepted kdepim 4:3.3.2-1 (i386 source all)

Any ideas how I can help beside testing this?

Problem still exists in 
kdepim 4:3.3.2-1 :-(


But tested serveral things. Findings:

Problem exists only if "Send Now" is selected.
Temporarly storing and sending afterwards ("Send Later")
fixed the problem.

So maybe the bug is inside these lines:

kmsender.cpp
------------------
send(.... : 116
------------------
  //Ensure the message is correctly and fully parsed
  kmkernel->outboxFolder()->unGetMsg( kmkernel->outboxFolder()->count() - 1 );

  if (sendNow && !mSendInProgress) rc = sendQueued();
  else rc = TRUE;
  kmkernel->outboxFolder()->close();

  return rc;
------------------------


checked unGetMsg and sendQueued but haven't found
anything yet. Any ideas?

*** Bug 109663 has been marked as a duplicate of this bug. ***

Hi,

Any updates on this bug? Bug 109663 is closed. Why should not this too? Sorry, but I can't test it really without having nfs and Solaris.

Just ping.

> Any updates on this bug? Bug 109663 is closed. Why should not this too?
> Sorry, but I can't test it really without having nfs and Solaris.
>
> Just ping.


I'm not affected by this bug anymore, as I've switched my mail folders to 
maildir and I'm now using Linux/i386. However, I still do have access to 
Solaris boxes, and I did a test today. I can still reproduce this bug with 
the latest version installed on our Solaris boxes (KDE 3.5.2 / KMail 1.9.1).

Created attachment 24513 [details]
Test program reproducing character duplicating bug

The attached file kmailtest.cpp contains a code snippet from 
KMFolderMbox::addMsg (kmfoldermbox.cpp) with some debug output. Test run on 
Linux:

-----------------------------------------------------------------
matthias@kermit:~/Programmieren/c++/test$ echo Test > test.mbox
matthias@kermit:~/Programmieren/c++/test$ ./kmailtest
1: 5
2: 3
3: 5 740a
4: 6
matthias@kermit:~/Programmieren/c++/test$ cat test.mbox
Test

matthias@kermit:~/Programmieren/c++/test$
-----------------------------------------------------------------

This works as expected. Test run on Solaris:

-----------------------------------------------------------------
matthias@statler:~/Programmieren/c++/test$ echo Test > test.mbox
matthias@statler:~/Programmieren/c++/test$ ./kmailtest 
1: 5
2: 3
3: 5 740a
4: -8184
matthias@statler:~/Programmieren/c++/test$ cat test.mbox 
Test
t

matthias@statler:~/Programmieren/c++/test$ 
-----------------------------------------------------------------

Shows the last-char-duplicated-behavior. The result of the last call to 
ftell() looks invalid. I think the reason is that a fread() on mStream is 
directly followed by fwrite(), and that Solaris requires a file positioning 
function between them as described in the fopen man page. Enabling the no-op 
fseek in kmailtest.cpp makes the test program work on Solaris as expected.

I ran the tests in local directories as well as NFS mounted ones. The results 
are the same, so this problem does not appear to be NFS specific.

This has been fixed in trunk since r772888.