Bug 131507 - Changing between big maildirs with many unread folders is slow
Summary: Changing between big maildirs with many unread folders is slow
Status: RESOLVED UNMAINTAINED
Alias: None
Product: kmail
Classification: Applications
Component: maildir (show other bugs)
Version: unspecified
Platform: Gentoo Packages Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
: 233134 (view as bug list)
Depends on:
Blocks:
 
Reported: 2006-07-29 12:33 UTC by Christian Janoff
Modified: 2015-04-12 10:27 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christian Janoff 2006-07-29 12:33:40 UTC
Version:            (using KDE KDE 3.5.2)
Installed from:    Gentoo Packages
Compiler:          gcc-3.4.6 
OS:                Linux

The same as bug 69439 from 2003:

"Switching from one mail folder to another, where
 the destination folder has in excess of 10,000
 messages (all folders are Maildir) causes a long
 pause (more than 10 seconds) and kmail is
 unresponsive for the duration of the swich.

 The more messages you have in the folder, the
 longer the pause."

I have 2 different maildir folders on a local ext3
filesystem, about 55000 mails each.

It takes 25 (twenty-five) seconds switching to one
of these folders, everytime. Afterwards I switch
back and forth, the same thing. Processor activity
rises to 100% until switching is finished. Hard disk
activity is 0, so this might not be a filesystem issue.

My machine is a 1600/3000+ AMD Sempron (AMD64), 1 GB
of RAM (533 MHz dual channel). This should be enough.

My last machine (1200 AMD Duron!, also ext3) used an
older version of KDE (I think 3.4). That old system
was a lot faster in switching to the same folders.

Christian
Comment 1 Ingo Klöcker 2006-07-29 15:50:48 UTC
Do you use message threading? If yes, does it help to turn it off completely? Does it help to turn only threading by subject off?
Comment 2 Christian Janoff 2006-07-29 17:02:39 UTC
My configuration:

General Options

[ ] Threaded message list"

Threaded Message List Options

[X] Open threads that contain new, unread or important messages
    and open watched threads."

These were the default settings since starting KMail 1.9.1 for the
first time. The first is unchecked (turned off), the second is
checked (turned on).

I turned off the "Open threads ..." option by selecting "Threads
default to closed" and it's much faster since then. (Thanks for
the hint! :-)

This seems to lead to another unexpected (at least for me ;-)
reason:

It seems that some options in the "Message List" tab have no
effect when changing.

"Threaded messages list" seems to be activated, but it's not
checked. (See above)

I also checked "Display message sizes" (because I missed them
in the message list today), but it has no effect. Even after
restarting KMail I cannot see any sizes (yes, I noticed the
scroll bar at the bottom).

Should I open a new bug report for this?

Christian
Comment 3 Ingo Klöcker 2006-07-29 18:08:24 UTC
The general setting "Threaded message list" can be overridden by the folder specific setting which you can find in the Folder menu.

IIRC then there's already a bug report about "Display message sizes". As workaround try right-clicking on the message list header and then enabling the Size column.
Comment 4 Ingo Klöcker 2006-07-29 18:10:30 UTC
FWIW, I just made a quick test. Switching from a folder with a small number of messages (e.g. the outbox) to my kde-devel folder with about 45,000 messages took about 20 seconds the first time and about 10 seconds the next time. And I'm using my good old Athlon 550 and for the kde-devel folder threading and threading by subject are both enabled and "Open threads that contain ..." is also enabled. Weird.
Comment 5 Christian Janoff 2006-08-01 16:31:31 UTC
Ok, as I mentioned before "threads default to closed" works:
Switching to large folders takes about 1 seconds. That will do
for me. (Thanks again for the hint! :-)

But maybe for others who read high traffic mailing lists and
want to have some threading options enabled, these options are
not fast enough.

Unfortunately I cannot provide really safe info about this
problem because some options don't seem to work on my KDE
installation (see my older mails above), so that I cannot
be *sure* which options are turned activated and which are
not. I will add another bug report about this because I didn't
find it in the bug db.

What I can provide to this "speed problem" is the following:

*****
[X] Always keep threads open:
[ ] Open threads that contain new, unread or important messages
    and open watched threads.

Switching to a folder with >55000 mails for the first time: 28 seconds. 
Switch back and forth to the same folder: about 1 second.

I think that's ok.
*****

*****
[ ] Always keep threads open:
[X] Open threads that contain new, unread or important messages
    and open watched threads.

Switching to a folder with >55000 mails for the first time: 25 seconds.
Switch back and forth to the same folder: 25 seconds.

To be more precise: It *always* takes 25 seconds to open this folder
with this option. KDE 3.5.2, KMail 1.9.1.
*****

Christian
Comment 6 András Manţia 2007-02-16 09:54:04 UTC
I can confirm this slow switching bug. The system is an Athlon 64 3200+, dual channel 1GB memory, so not exactly a slow one:
Switch from inbox (1000messages) to kde-core-devel(21000 messages): 4 seconds
switching back is < 1 seconds. Switching a second time gives me no noticeable difference. Switching to a folder with 12500 messages is about 2 seconds.

This is with Thread Messages and Thread also by subject enabled. With them disabled it takes about 3 seconds to switch to the 21000 messages folder. 

KMail and KDE from around 3.5.6 was tagged.

Comment 7 Thomas McGuire 2007-07-17 14:30:30 UTC
See also bug 88467, which might be the same issue.
Comment 8 András Manţia 2008-02-01 20:02:51 UTC
Kmail trunk is even worse. A folder with 7500 messages from which 2500 unread is unusable (dual core, 2GB memory). Not only the switching, but anything in there (reading, scrolling, etc.). Another one with 14000 messages, all read is also slow, but at least usable.
Comment 9 Thomas McGuire 2008-09-11 22:07:16 UTC
András, for the speed problems in trunk/KDE4, see 163469.
Comment 10 Médéric Boquien 2009-02-21 19:59:30 UTC
Hi. Is this bug still valid? With the new messages list it is much faster.
Comment 11 András Manţia 2009-02-21 21:06:11 UTC
Partly. It is faster, but more annoying. Switching to my kde-commit folder which currently has 7280 messages, 1720 unread is instant, but for 8-9 seconds the message list keep reorganizing itself, it is basically unusable, you scroll somewhere and it scrolls away.
Comment 12 András Manţia 2009-02-21 21:07:26 UTC
I have to say the problem is probably because of the high number of unread messages, switching to another folder that has 15000+ and only 10 unread works just fine.
Comment 13 Thomas McGuire 2009-02-21 21:10:29 UTC
> I have to say the problem is probably because of the high number of unread messages, switching to another folder that has 15000+ and only 10 unread works just fine.

That could well be, because unread messages in threads are expanded automatically, and Qt4 is dead slow when it comes to expanding/collapsing something.
Comment 14 Björn Ruberg 2010-04-02 12:55:54 UTC
So is this in the end just about qt slow expanding of listitems?
Comment 15 Björn Ruberg 2010-04-05 23:33:09 UTC
*** Bug 233134 has been marked as a duplicate of this bug. ***
Comment 16 Thomas Fjellstrom 2010-04-11 16:01:04 UTC
If it helps, it seems that after you disable the auto opening of threads with unread messages, and tell it to only show unread threads, then tell it to expand all threads, its speedy. Could be a hint to whoever eventually looks into fixing this.

Though I have to say, switching into a maildir mailbox with 100k+ messages is STILL pretty slow (not as bad as it was) even with auto opening unread threads turned off. So I'm not sure the entire cause is Qt's slowness when expanding listitems.
Comment 17 David Jander 2010-08-20 09:30:59 UTC
I can confirm this problem is still there in kde-4.3.2.
Opening an IMAP folder (local net) with 35000 messages where most of them are unread in the "Threaded date view" mode takes 2 minutes reorganizing the message list each time before I can scroll somewhere and actually use the folder.
This bug has been with us for so long now, any chance this will be fixed one day? It is one of the primary reasons kmail is mostly unusable with large folders in thread mode, isn't this supposed to be bad enough for kmail developers to really worry?
I realize this fix isn't trivial, but if QT is at fault, what about trying to fix Qt or at least implement a work-around for this problem? IMHO, not using threaded view in large mailing-lists is not very useful!
In the worst case, it Qt's tree widget is not scalable for such use-cases, maybe KDE needs an alternative widget for tree-lists?
Come on, it has been 7 years now! Please start doing something!
Comment 18 Laurent Montel 2015-04-12 10:27:19 UTC
Thank you for taking the time to file a bug report.

KMail2 was released in 2011, and the entire code base went through significant changes. We are currently in the process of porting to Qt5 and KF5. It is unlikely that these bugs are still valid in KMail2.

We welcome you to try out KMail 2 with the KDE 4.14 release and give your feedback.