Bug 243569

So, some more information:

A) Kmail activity

The situation is bad no matter if mails are loaded or not. (A possible reason could be imap because this is not stabel either.)

1. When I switch of nepomuk/strigi the situation does not change

2. When I switch to "offline mode" the situation does not change

3. When I restart KMail the situation does not change

4. When I restart KMail in offline mode the situation improves!

5. When I switch on online mode again, the situation will decline again (same as before)

B) Kmail configuration

- I use KMail with 12 accounts, one of it is imap

- There are a lot of filters defined but noone starting a script or something complicated.

Regards, Martin

*** Bug 225602 has been marked as a duplicate of this bug. ***

Hi again,

I installed KDE 4.5 RC2 (Kubuntu Lucid packages 64bit) and I still have the same problem... Using only two disconnected IMAP accounts -- no filters.

I do not know if it helps, but I have noticed that clicking on the system tray icon (thus restoring kmail's window) the CPU goes down to normal levels.

Still happens in KDE 4.5.0...

Hello *,

not sure if this is the same issue : in my case the high CPU load is caused by a "check for mail" on a disconnected imap account, so it is not "without any reason"

using "connected" imap, the check of the 500+ folders lasts <10s, using the disconnected imap account type, the update lasts ~10min, with hich (75%) CPU usage by kmail. I therefore exclude a global issue with the imap server (dovecot, ubuntu 10.04). It looks however as if the disconnected imap variant was doing more/different things, which extremely slow down the access.

this happens as well on first as on further checks, so it is not related to some kind of initial setting of data structures/refresh of information.

clicking on the status progress bar (the blue widget in the bottom right corner of the kmail window), it shows that ~1 folder/s is refreshed. the displayed messages are "Retrieving message list", "Checking for validity" and "synchronization done"

using dovecot's rawlog feature, I see that the disconnected imap uses following commands to refresh one folder :
2 NAMESPACE
3 LIST "" ""
4 LIST "" "Mailing_lists.XYZ"
5 SELECT "Mailing_lists.XYZ"
6 LIST "" "Mailing_lists.XYZ.%"
7 NOOP
8 UID FETCH 1:* (FLAGS RFC822.SIZE)


wheras the "connected" imap account issues :
2 NAMESPACE
3 LIST "" ""
4 LIST "" "Mailing_lists.XYZ"
5 SELECT "Mailing_lists.XYZ"
6 NOOP
7 UID FETCH 4927:* (UID RFC822.SIZE FLAGS ENVELOPE BODY.PEEK[HEADER.FIELDS (REFERENCES)])

I think the main difference is the last line : disconnected imap fetches ALL messages, which results in higher effort per folder.

Could somebody check why this is implemented like that ?

Thanks

I have exactly the same problem. When kmail starts it checks for emails on my disconnected IMAP account (which has a filter to forward all emails to my proper kmail inbox) and the CPU usage increases roughly linearly (up to about 50%) with time until a few minutes later when it drops back down to idle usage.

I should mention that some of my email folders have thousands of emails in so if something decides it's a good idea to scan these folders it might be there for a long time.

This is on KDE 4.4.5.

Ralph

*** This bug has been confirmed by popular vote. ***

I have disabled all filters and set the email accounts to manual checking.

When I check _just_ my POP3 account, kmail goes mental and the CPU usage climbs and climbs as does hard drive activity. My local sent-mail and some other folders have thousands of emails in each (my inbox only has a few emails). Clearly kmail is doing some trawling through the local folders that I have and this is what is causing the slow down. Is this anything to do with Nepomuk?

It's really annoying!

No changes? No solutions available? Please fix it!

(In reply to comment #9)
> No changes? No solutions available? Please fix it!

I'm fairly certain I know what's causing this.

It's the mail compacting process. 

If you have loads of emails it just chugs away trying to "compact" (whatever that means - sounds like something from the bygone days of Outlook Express!) your email. As I have thousands of emails it just takes ages. 

Is it worth putting a wish in for the user to be able to disable this compacting feature which seems to slow the computer down so much?

Is message compacting even needed in this day and age where hard drive space is so cheap?

(In reply to comment #10)
> 
> Is it worth putting a wish in for the user to be able to disable this
> compacting feature which seems to slow the computer down so much?
> 

No, KMail 2 (from kdepim 4.6) has no such function. (and doesn't need one)

Am Donnerstag, 26. Mai 2011, um 02:39:24 schrieb Christophe Giboudeaux:
> No, KMail 2 (from kdepim 4.6) has no such function. (and doesn't need one)

Who uses KMail 2???

As I remember I opened this issue and this ist KMail 1.13.6

Just to clarify what version of KMail we talk about! In addition I wonder, if 
opening an incident like that is useful at all - is it or not? Does someone 
work on the old and obviously most often used version of KMail??

And not to forget, for sure my configuration has changed. Now ist is Kubuntu 
11.04 (Natty Narwhal) and KMail 1.13.6 !

(Thanks to all these wonderful developers!)

Martin

Am Sonntag, 4. Juli 2010, um 12:14:04 schrieb Martin L ü c h e m:
> +++ This bug was initially created as a clone of Bug #225602 +++
> 
> Version:           1.13.3 (4.4.4 (KDE 4.4.4), Debian packages)
> Compiler:          cc
> OS:                Linux (x86_64) release 2.6.32-5-amd64

(In reply to comment #13)
> Just to clarify what version of KMail we talk about! In addition I wonder, if 
> opening an incident like that is useful at all - is it or not? Does someone 
> work on the old and obviously most often used version of KMail??

No, KMail 1 bugs that cannot be reproduced in KMail2 are closed. (revisiting them all takes a lot of time,however).

Note that this report and the comments are not really helpful: The cpu usage can be caused by totally different operations.

Without some real debugging (memcheck, massif...) there's nothing that can be done

Hi Christophe,

Am Donnerstag, 26. Mai 2011, um 23:09:14 schrieb Christophe Giboudeaux:
> https://bugs.kde.org/show_bug.cgi?id=243569
> 
> 
> 
> 
> 
> --- Comment #14 from Christophe Giboudeaux <cgiboudeaux gmx com> 
> 2011-05-26 23:09:12 --- (In reply to comment #13)
> 
> > Just to clarify what version of KMail we talk about! In addition I
> > wonder, if opening an incident like that is useful at all - is it or
> > not? Does someone work on the old and obviously most often used version
> > of KMail??
> 
> No, KMail 1 bugs that cannot be reproduced in KMail2 are closed.
> (revisiting them all takes a lot of time,however).
> 
> Note that this report and the comments are not really helpful: The cpu
> usage can be caused by totally different operations.

Oh yes, for sure, everything is possible!

> 
> Without some real debugging (memcheck, massif...) there's nothing that can
> be done

This bug is more than one year old and, yes I sent in the result of debugging 
+ a video. Would you be so kind to look at this:

"Re: [Bug 225602] Kmail is extremely slow and uses CPU without any reason
 Datum: 15.04.2010 15:34
 Von: Martin Lüchem <Heinrich20@gmx.de>
 An: bug-control@bugs.kde.org
 
Hi Thomas,

the situation improved a little bit. Some weeks everything seemed to be ok but 
then the problem reoccured. Now we have CPU usage up to 20%. This occurs in 
cycles!

This is the result of GDB. I hope, what I did is correct and helps:

Program received signal SIGINT, Interrupt.
0x00007fe43e12c743 in *__GI___poll (fds=<value optimized out>, nfds=<value 
optimized out>, timeout=3024)
    at ../sysdeps/unix/sysv/linux/poll.c:87
87      in ../sysdeps/unix/sysv/linux/poll.c
(gdb) backtrace
#0  0x00007fe43e12c743 in *__GI___poll (fds=<value optimized out>, nfds=<value 
optimized out>, timeout=3024)"

and so on!

"Re: [Bug 225602] Kmail is extremely slow and uses CPU without any reason
 Datum: 18.04.2010 14:14
 Von: Martin Lüchem <Heinrich20@gmx.de>
 An: Thomas McGuire <mcguire@kde.org>
 
Hi Thomas,

this vid to show you, how KMail uses CPU without any action.

Regards, Martin"


So, why not, close this bug! May be I am one of the remaining idiots that have 
been used Kmail since 1999. (This is NOT) Very funny! 

Martin

As far as I know Kmail 2 isn't out yet and when it does get released it will be buggy and not adopted by many distros until it proves itself to be reliable.

Hence it is important to fix this bug. I have thousands of emails and I am sure that this fact coupled with Kmail's periodic "compacting" of those emails is what causes the massive slow down.

If you really can't see why fixing this bug now would be useful, please at least test Kmail 2 with tens of thousands of emails in say 20 different folders and check that it doesn't lock up the system when doing main "compacting".

Testing with 20 emails will not show up a problem!

Thanks.

(In reply to comment #16)
> and check that it doesn't lock up the system when doing ***mail*** "compacting".

There were some people that voted for this bug. I sent in a gdb result. You can see that it is a pain and you tell us you will no longer look at bugs like this?

When I look at the list of open bugs I understand what makes you stop working on this.

Nevertheless this shows that kmail came down from a really great and fast tool to a very slow an painful tool - a pity!

Regards, Martin

I met Till Adam from KDAB several weeks ago and we debuged this problem on my system. The (on of the?) reason is, that KMail writes his config-file kmailrc *very* often after every change on its database, folders or imap-storage, because kmailrc is including details about every IMAP-folder. 

This kmailrc is very large => that leads to very much I/O, very much wait-I/O for the CPU and very much CPU-time for calculating this config-file. You can trace this problem with gdb, seeing KMail writing his config-file in 9 out of 10 cases.

KDAB prepared a patch for me, making KMail writing his kmailrc not after each change, but after several seconds/minutes.

I'm running Kmail with this patch for two weeks now without any problems. It's much better know. Maybe there are still other performance problems in KMail, but THIS problem on my system has been solved by this. We're tracing this at the moment, but looks like my patch will go upstream.

You can find my patched version in my OpenSUSE-Repo at http://download.opensuse.org/repositories/home:/pheinlein/openSUSE_11.4. Or try this patch.

Created attachment 60713 [details]
Patch for Kmail, writing his kmailrc not so often

Am Montag, 6. Juni 2011, um 22:10:12 schrieb Peer Heinlein:
> KDAB prepared a patch for me, making KMail writing his kmailrc not after
> each change, but after several seconds/minutes.
> 
> I'm running Kmail with this patch for two weeks now without any problems.
> It's much better know. Maybe there are still other performance problems in
> KMail, but THIS problem on my system has been solved by this. We're
> tracing this at the moment, but looks like my patch will go upstream.
> 
> You can find my patched version in my OpenSUSE-Repo at
> http://download.opensuse.org/repositories/home:/pheinlein/openSUSE_11.4. Or
> try this patch.

klingt ja wie ein böser Nebeneffekt einer an sich harmlosen Ursache. 
KPackagekit jammert bei dem rpm mit einer sinnfreien Fehlermeldung! :-( 
Jetzt frage ich mich nur, wie lange es braucht, bis der Patch transportiert 
wird. Theoretisch sollte das ja schnell gehen... Man kan nur hoffen!

Danke, Peer!

Sorry, wrong language! ;-)

I couldn'tinstall this on kubuntu with the KPackage-kit. Message did not tell 
anything about the reason. Hopefully the patch will be out soon because this 
really is annoying!

Tahnk you Peer!

Am Dienstag, 7. Juni 2011, um 22:25:44 schrieb Martin L ü c h e m:
> https://bugs.kde.org/show_bug.cgi?id=243569
> 
> 
> 
> 
> 
> --- Comment #21 from Martin L ü c h e m <Heinrich20 gmx de>  2011-06-07
> 22:25:41 ---
> 
> Am Montag, 6. Juni 2011, um 22:10:12 schrieb Peer Heinlein:
> > KDAB prepared a patch for me, making KMail writing his kmailrc not after
> > each change, but after several seconds/minutes.
> > 
> > I'm running Kmail with this patch for two weeks now without any problems.
> > It's much better know. Maybe there are still other performance problems
> > in KMail, but THIS problem on my system has been solved by this. We're
> > tracing this at the moment, but looks like my patch will go upstream.
> > 
> > You can find my patched version in my OpenSUSE-Repo at
> > http://download.opensuse.org/repositories/home:/pheinlein/openSUSE_11.4.
> > Or try this patch.
> 
> klingt ja wie ein böser Nebeneffekt einer an sich harmlosen Ursache.
> KPackagekit jammert bei dem rpm mit einer sinnfreien Fehlermeldung! :-(
> Jetzt frage ich mich nur, wie lange es braucht, bis der Patch transportiert
> wird. Theoretisch sollte das ja schnell gehen... Man kan nur hoffen!
> 
> Danke, Peer!

I have similar symptoms, KDE is sometimes, seemingly randomly unbearably slow and takes a lot of CPU.  It is clearly doing something but it is impossible to tell what.  Usually this occurs when kmail has recently started but it can occur during other times as well.  

I get the UI grinding to a halt, slow window updates, very slow responsiveness almost as if the process has been halted, it usually lasts for a couple of minutes and then kmail springs back into life.  During this time kamil runs at 100% CPU in a single thread.  The UI isn't completely dead, just extremely slow.

for what it is worth I have nepomuk disbaled.

Kmail 1.13.6 / kde 4.6.0 release 6 and suse 11.4. If I recall correctly earlier versions were not badly affected.

kmail constantly eats 5-6 % cpu on my laptop. This is quite annoying because it consumes battery quite fast.

I did some quick debugging.

$ kmail --version
Qt: 4.7.3
KDE Development Platform: 4.6.5 (4.6.5)
KMail: 1.13.7
$ cat /etc/system-release
Fedora release 15 (Lovelock)
$ strace -p $(pidof kmail) &> /tmp/kmail.strace ; (Ctrl-C after few seconds)
$ sort /tmp/kmail.strace | uniq -c | sort -n
      1 poll([{fd=3, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=5, events=POLLIN}, {fd=14, events=POLLIN}, {fd=16, events=POLLIN}, {fd=25, events=POLLIN}, {fd=26, events=POLLIN}, {fd=27, events=POLLIN}, {fd=31, events=POLLIN}, {fd=33, events=POLLIN}, {fd=36, events=POLLIN}, {fd=38, events=POLLIN}, {fd=37, events=POLLIN}, {fd=39, events=POLLIN}], 15, 10) = 0 (Timeout)
      1 poll([{fd=3, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=5, events=POLLIN}, {fd=14, events=POLLIN}, {fd=16, events=POLLIN}, {fd=25, events=POLLIN}, {fd=26, events=POLLIN}, {fd=27, events=POLLIN}, {fd=31, events=POLLIN}, {fd=33, events=POLLIN}, {fd=36, events=POLLIN}, {fd=38, events=POLLIN}, {fd=37, events=POLLIN}, {fd=39, events=POLLIN}], 15, 11) = 0 (Timeout)
      1 poll([{fd=3, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=5, events=POLLIN}, {fd=14, events=POLLIN}, {fd=16, events=POLLIN}, {fd=25, events=POLLIN}, {fd=26, events=POLLIN}, {fd=27, events=POLLIN}, {fd=31, events=POLLIN}, {fd=33, events=POLLIN}, {fd=36, events=POLLIN}, {fd=38, events=POLLIN}, {fd=37, events=POLLIN}, {fd=39, events=POLLIN}], 15, 15 <unfinished ...>
      1 poll([{fd=3, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=5, events=POLLIN}, {fd=14, events=POLLIN}, {fd=16, events=POLLIN}, {fd=25, events=POLLIN}, {fd=26, events=POLLIN}, {fd=27, events=POLLIN}, {fd=31, events=POLLIN}, {fd=33, events=POLLIN}, {fd=36, events=POLLIN}, {fd=38, events=POLLIN}, {fd=37, events=POLLIN}, {fd=39, events=POLLIN}], 15, 2) = 0 (Timeout)
      1 poll([{fd=3, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=5, events=POLLIN}, {fd=14, events=POLLIN}, {fd=16, events=POLLIN}, {fd=25, events=POLLIN}, {fd=26, events=POLLIN}, {fd=27, events=POLLIN}, {fd=31, events=POLLIN}, {fd=33, events=POLLIN}, {fd=36, events=POLLIN}, {fd=38, events=POLLIN}, {fd=37, events=POLLIN}, {fd=39, events=POLLIN}], 15, 4) = 0 (Timeout)
      1 Process 2079 detached
     44 poll([{fd=3, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=5, events=POLLIN}, {fd=14, events=POLLIN}, {fd=16, events=POLLIN}, {fd=25, events=POLLIN}, {fd=26, events=POLLIN}, {fd=27, events=POLLIN}, {fd=31, events=POLLIN}, {fd=33, events=POLLIN}, {fd=36, events=POLLIN}, {fd=38, events=POLLIN}, {fd=37, events=POLLIN}, {fd=39, events=POLLIN}], 15, 0) = 0 (Timeout)
     84 poll([{fd=3, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=5, events=POLLIN}, {fd=14, events=POLLIN}, {fd=16, events=POLLIN}, {fd=25, events=POLLIN}, {fd=26, events=POLLIN}, {fd=27, events=POLLIN}, {fd=31, events=POLLIN}, {fd=33, events=POLLIN}, {fd=36, events=POLLIN}, {fd=38, events=POLLIN}, {fd=37, events=POLLIN}, {fd=39, events=POLLIN}], 15, 15) = 0 (Timeout)
     87 poll([{fd=3, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=5, events=POLLIN}, {fd=14, events=POLLIN}, {fd=16, events=POLLIN}, {fd=25, events=POLLIN}, {fd=26, events=POLLIN}, {fd=27, events=POLLIN}, {fd=31, events=POLLIN}, {fd=33, events=POLLIN}, {fd=36, events=POLLIN}, {fd=38, events=POLLIN}, {fd=37, events=POLLIN}, {fd=39, events=POLLIN}], 15, 14) = 0 (Timeout)
    174 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1883, ...}) = 0
    438 read(8, 0x1b94df4, 4096)                = -1 EAGAIN (Resource temporarily unavailable)
$ lsof -n -p $(pidof kmail) > /tmp/kmail.lsof
# select file descriptors from kmail strace
$ for fd in $(sort -u /tmp/kmail.strace | egrep -o 'fd=[[:digit:]]+' | sort -u | sed -e 's/fd=//'); do egrep " $fd(u|r|w)" /tmp/kmail.lsof; done
kmail   2079 xxx   14u  unix 0xffff8801fd17ad80       0t0   24968 socket
kmail   2079 xxx   16u  unix 0xffff8801fd370680       0t0   23184 /tmp/ksocket-xxx/kmailGm2079.slave-socket
kmail   2079 xxx   25u  unix 0xffff8801cf564ac0       0t0   27569 socket
kmail   2079 xxx   26r  0000                0,9         0    4063 anon_inode
kmail   2079 xxx   27u  unix 0xffff8801f67e57c0       0t0   38315 socket
kmail   2079 xxx    3r  FIFO                0,8       0t0   23983 pipe
kmail   2079 xxx   31u  unix 0xffff8801cf55e800       0t0   44507 /tmp/ksocket-xxx/kmailfp2079.slave-socket
kmail   2079 xxx   33u  unix 0xffff8801cf55f840       0t0   44511 /tmp/ksocket-xxx/kmailKU2079.slave-socket
kmail   2079 xxx   36u  unix 0xffff8801f67e71c0       0t0   93677 /tmp/ksocket-xxx/kmailGS2079.slave-socket
kmail   2079 xxx   37u  sock                0,6       0t0  135541 can't identify protocol
kmail   2079 xxx   38u  unix 0xffff8801cf55aa40       0t0   91663 /tmp/ksocket-xxx/kmailug2079.slave-socket
kmail   2079 xxx   39u  unix 0xffff8801f3935480       0t0  135542 socket
kmail   2079 xxx    5u  unix 0xffff88022d8a3a80       0t0   22970 socket
kmail   2079 xxx    8u  unix 0xffff88020393f1c0       0t0   23985 socket
kmail   2079 xxx    9u  unix 0xffff8801fd17de40       0t0   24801 socket

Hope this helps.

For those who wanna know: Kmail2 is fast and stable, Nepomuk enabled. (KDE 4.7.1)

Kmail in KDE SC 4.8.1 is apparently slow again. Can anyone confirm?

I (indirectly) confirm with this Fedora Bugzilla filed during Fedora 17 Beta testing: https://bugzilla.redhat.com/show_bug.cgi?id=814410 .
The symptoms are described in a very similar way, so I suppose they are related.

WONTFIX for kmail 1.x.

Note for the redhat report: valgrind is way more reliable than strace.

Hey great. So you solved this problem in the new version? Which one? This is 
really good news because this is not KMail 1.x but KMail 4.7.3! So I think we 
can keep this issue open - thank you!

By the way, I never had problems like that with the first version of KMail. 
Very stable, quite fast, really good. It all started with the Akonadi stuff!

Oh, this is a little bit confusing: This is version 1? Why the difference in 
Software Revisioning?

Please help and be a bit more precise: What will really happen to help all the 
KMail users that suffer from the instability of this version. We like this 
piece of software, that is why we did not move to Thunderbird.

Martin

The KMail > 1.13.7 bugs are to be filled under the KMail2 product.

Note that 4.7.3 is now considered obsolete. Ask your distribution to provide a recent version.

I think, what I did when opening the issue was using the help menu. So I would 
assume that the revisioning data ist the correct one - for sure!

Can you switch this to KMail2? Can I?

(I will update as soon as I do have DSL again! ;-) )

Martin

KMail 4.8.2, is this KMail 1.x oder 2.x? What about the bug, what will happen 
to it?

Regards, Martin

I'd rather avoid, there are already a couple bugs on the same topic that need to be sorted/merged: 
https://bugs.kde.org/buglist.cgi?query_format=advanced&short_desc=slow&bug_status=UNCONFIRMED&bug_status=NEW&short_desc_type=allwordssubstr&product=kmail2