Bug 279115 - Standard Mailing list doesn't thread properly
Summary: Standard Mailing list doesn't thread properly
Status: RESOLVED FIXED
Alias: None
Product: kmail2
Classification: Applications
Component: message list (show other bugs)
Version: unspecified
Platform: Arch Linux Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-08-02 02:02 UTC by Jonathan M Davis
Modified: 2018-08-16 22:03 UTC (History)
7 users (show)

See Also:
Latest Commit:
Version Fixed In: 5.10.0
Sentry Crash Report:


Attachments
Zip archive of test messages (10.09 KB, application/zip)
2013-06-12 19:41 UTC, Jonathan Marten
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan M Davis 2011-08-02 02:02:33 UTC
Version:           unspecified (using KDE 4.7.0) 
OS:                Linux

Kmail lists itself as 4.7.0 just like KDE, so I don't know what application version number (2.x) it is, hence why I left it as unspecified.

Okay. I'm using the "Standard Mailing List" mode for a folder which has e-mails from a mailing list in it. That makes it select

Grouping: None
Group expand policy: - (grayed out)
Threading: Perfect, by References and Subject
Thread leader: Topmost Message (grayed out)
Thread expand policy: Expand Threads With Unread or Important Messages

in the Groups & Threading tab, and

Fill view strategy: Favor Interactivity

in the Advanced tab

Kmail does manage to put messages in the correct thread, but it frequently threads them under the original message or only one deep. It doesn't usually put them under the message that they were actually replying to. It does sometimes, but usually not. Kmail 1 on another machine of my machines threads much better with the same folder with the same messages. Kmail 2 doesn't really thread them correctly much beyond getting them under the correct parent.

If it matters, I'm using disconnected IMAP via gmx.com, and I'm using the mailing lists for D: http://www.digitalmars.com/NewsGroup.html (primarily digitalmars.D).

Reproducible: Always

Steps to Reproduce:
Have a folder using "Standard Mailing List" for its configuration. Have messages from a mailing list in that folder.

Actual Results:  
Messages are under the correct overall parent but generally don't end up under their own correct parent.

Expected Results:  
Messages are nicely threaded so that they're threaded under the messages that they reply to.
Comment 1 Silver Salonen 2011-08-03 08:35:22 UTC
In my case KMail frequently does not even recognize correct threads even if they are in the exact same thread and should be recognized even by perfect threading (I guess).
Comment 2 Wonko 2011-08-26 23:52:53 UTC
I also noticed the bug. I think this was working fine in KMail1.
Um, why is this assigned to the sieve component? This does not look right.
Comment 3 Zed 2012-11-06 21:16:33 UTC
*** This bug has been confirmed by popular vote. ***
Comment 4 Laurent Montel 2012-11-07 07:59:17 UTC
Need example for testing it.
Comment 5 Jonathan Marten 2013-06-12 19:09:37 UTC
See also bug 136489 and bug 129973 - not sure if those are exact duplicates.
Comment 6 Jonathan Marten 2013-06-12 19:41:48 UTC
Created attachment 80486 [details]
Zip archive of test messages

Test case attached, public messages with no personal information included.  These Bugzilla mails are unchanged apart from deleting irrelevant "Received" headers and removing the "New:" prefix from the thread starters so that the subjects are identical.

To reproduce:  Create a new empty folder in KMail.

From within the zip archive, click on each of the files in turn to open them with KMail View.  Do not do this with the 'thread2-1' message for the moment.

Context menu - Copy Item To - the new folder

The folder should now have five emails in it.  With the aggregation options set to "Standard Mailing List", observe that the 3 messages referring to bug#314981 are threaded but the 2 messages referring to bug#279188 are not.  Even though the thread starter is not present, they should be grouped by subject.

Then copy in the 'thread2-1' message.  Observe that the second set of messages become threaded.

To confirm, delete one of the thread starter messages and observe that the remaining ones become unthreaded again.
Comment 7 Nick Leverton 2013-09-17 11:51:35 UTC
Still an issue in 4.11.1.    For instance if you are added to a BZ (any bug on any bugzilla, not just KDE's) at some point after the bug was opened, kmail will ignore the References header and will show any subsequent mails from that bug as individual at the top level, instead of threading them.  This results in the unthreaded mails flooding out the view of the mailbox.

My threading settings are set to "Perfect, References and Subject"
Comment 8 Denis Kurz 2016-09-24 18:17:00 UTC
This bug has only been reported for versions before 4.14, which have been unsupported for at least two years now. Can anyone tell if this bug still present?

If noone confirms this bug for a Framework-based version of kmail2 (version 5.0 or later, as part of KDE Applications 15.12 or later), it gets closed in about three months.
Comment 9 Denis Kurz 2017-01-07 22:23:27 UTC
Just as announced in my last comment, I close this bug. If you encounter it again in a recent version (at least 5.0 aka 15.08), please open a new one unless it already exists. Thank you for all your input.
Comment 10 Jonathan Marten 2017-10-06 09:47:02 UTC
Confirmed with current KMail Git master (5a385e200).
Subject threading is not working with the same test case.
Comment 11 Jan 2018-08-12 07:29:10 UTC
I figured out why this is happening: KMail only considers prefixed E-Mails for subject based threading. If it doesn't start with a "Re:" for example, it's not a thread. Bugzilla doesn't do that. So in the absence of that, it only uses In-Reply-To/Reference headers. And that doesn't work if the thread starter is missing, which imho is the wrong way to handle that.

And even if it is prefixed, it will not consider messages with a time difference of 120 seconds. So if you managed to have a github conversation that is faster than that, it's not a thread either. This one in particular I really don't understand.
Comment 12 Daniel Vrátil 2018-08-16 22:03:21 UTC
Git commit dc4a7feb10f035be1a712e67d75daf7ebe859f33 by Daniel Vrátil, on behalf of Jan Janssen.
Committed on 16/08/2018 at 22:02.
Pushed by dvratil into branch 'master'.

Thread messages with same reference ID together

Summary:
Thread messages with same reference ID together

Messages from github or bugzilla usually use In-Reply-To and References headers
with an ID that either doesn't actually exist or just isn't around in the
folder. In case of bugzilla, no threading happens in that case because subject
lines don't start with "Re:". github does do that, but oftentimes the subject
based threading is imprecise (by design).

Therefore create a sorted-by-date cache of messages similar to the subject
based cache and put any message we cannot perfectly thread to the very first
message with the same message ID.
What results is a flat but threaded messagelist similar to what thunderbird does.

FIXED-IN: 5.10.0

Reviewers: dvratil

Reviewed By: dvratil

Subscribers: dvratil, bruns, kde-pim

Tags: #kde_pim

Differential Revision: https://phabricator.kde.org/D14883

M  +89   -4    messagelist/src/core/model.cpp
M  +9    -0    messagelist/src/core/model_p.h

https://commits.kde.org/messagelib/dc4a7feb10f035be1a712e67d75daf7ebe859f33