Bug 359603 - Incorrect and useless From header in emails generated by KDE Bugzilla
Summary: Incorrect and useless From header in emails generated by KDE Bugzilla
Status: RESOLVED NOT A BUG
Alias: None
Product: bugs.kde.org
Classification: Websites
Component: general (show other bugs)
Version: unspecified
Platform: unspecified All
: NOR major
Target Milestone: ---
Assignee: KDE sysadmins
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-02-20 16:39 UTC by Pali Rohár
Modified: 2016-06-08 21:32 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
signature.asc (197 bytes, application/pgp-signature)
2016-03-26 18:35 UTC, Pali Rohár
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Pali Rohár 2016-02-20 16:39:06 UTC
Before Oct 2015 all emails generated by KDE Bugzilla when somebody adds comment to bug or change description has From and Sender header in this format:

From: XNAMEX YYY <XEMAILX>
Sender: bugzilla_noreply@kde.org

(where XNAMEX YYY is name and XEMAILX email address).

Since Oct 2015 email there is no Sender header and format of From is:

From: XNAMEX YYY via KDE Bugzilla <bugzilla_noreply@kde.org>

(where XNAMEX YYY is name of who added comment to specific bug).

And now in this week I got emails from KDE Bugzilla which format of From header is:

From: via KDE Bugzilla <bugzilla_noreply@kde.org>

(again without any Sender header).

From header with value "via KDE Bugzilla <bugzilla_noreply@kde.org>" is totally useless, it does not contain truth information not any useful.

From header with value "XNAMEX YYY via KDE Bugzilla <bugzilla_noreply@kde.org>" is incorrect too. For specifying who sent that email there is Sender header. From header should be really in format "XNAMEX YYY" and email address. Not some funny value like "via KDE Bugzilla".

Please fix generating From header for emails sent by KDE Bugzilla. Do not invent new funny format of From header, but instead generate it according to Email standards, like RFC 5322 (Originator Fields): https://tools.ietf.org/html/rfc5322#section-3.6.2

Reproducible: Always
Comment 1 Ben Cooksley 2016-02-20 20:13:11 UTC
The change was made due to DKIM/DMARC issues. You can no longer set "From" to be the person who you are representing - it just does not work.
Comment 2 Pali Rohár 2016-02-20 20:35:16 UTC
Why it does now work? RFC 5322 says:

The "From:" field specifies the author(s) of the message, that is, the mailbox(es) of the person(s) or system(s) responsible for the writing of the message.

So using value "XNAMEX YYY via KDE Bugzilla <bugzilla_noreply@kde.org>" for From header is incorrect and in some conditions can be interpreted as violating RFC document which is standard for email messages.

For specifing via KDE Bugzilla, there is Sender field, RFC 5322 says:

The "Sender:" field specifies the mailbox of the agent responsible for the actual transmission of the message.  For example, if a secretary were to send a message for another person, the mailbox of the secretary would appear in the "Sender:" field and the mailbox of the actual author would appear in the "From:" field.

KDE Bugzilla acts there as "secretary". If KDE Bugzilla provides email support it should send emails according to email standards...
Comment 3 Ben Cooksley 2016-02-20 20:56:21 UTC
Ignore the RFC, it isn't helpful here.

DKIM/DMARC are standards for verifying the email was legitimately sent. In particular, they operate to ensure headers and the message body originate from someone authorised (by the domain owner) to send mail. The originator of the message is determined by the From header.

We aren't authorised to send mail for @yahoo.com, @gmail,com, etc. so therefore we CANNOT set From to be the users address. If we try to do so, our email will not validate, and will be rejected or sent to the spam folder by Yahoo/GMail/other providers.
Comment 4 Pali Rohár 2016-02-20 21:52:42 UTC
But if you ignore RFC 5322 how email structure must looks like, you are breaking email message itself. It means that every email client which is RFC 5322 compliant just show incorrect or less useful data/email to user who is trying to read it.

I cannot ignore RFC 5322 or its older version RFC 2822 or RFC 822. RFC 822 has now "INTERNET STANDARD" status and meaning From and Sender field is there same as in its last RFC 5322 in "DRAFT STANDARD".

In case of correctness RFC 822 is here unambiguous and "INTERNET STANDARD". Standards are here to make interopability between other applications.

If you ignore it, it cause problem in all other applications which users use to read emails. And really modified and useless information in From header cause problems for me and my email softwares. I'm sure I'm not alone who use advanced search or other actions on inboxes and such "fake" information in envelope cause lot of problems...
Comment 5 Nicolás Alvarez 2016-02-20 21:59:17 UTC
Complain to the creators of DMARC. Complain to the email providers following DMARC and sending RFC-compliant messages to the spam folder.

We can't do anything about it.
Comment 6 Pali Rohár 2016-02-20 22:30:21 UTC
I do not agree.

If somebody put correct and RFC-compliant message with meaningful headers suitable for processing and parsing into spam folder it is his problem.

But if server generates incorrect email message and send it to the world, it is problem of server, not client. If server generates something wrong it is bug on server and like other bugs it should be fixed.

Think of user, receiver who configured that he *wants* to receive notifications from KDE Bugzilla. And receive those notifications in form of email message. If that message is incorrect or contains useless informations, it is useless for receiver=user.

If somebody configured his bugzilla account that he want to receive *email* messages, he should get email messages according to email message standard (RFC 822, or new). Not something different. Why otherwise he configure such option?

Receiving these emails correctly formatted is to make it easier, not harder just becase emails comes "broken".
Comment 7 Nicolás Alvarez 2016-02-20 22:36:41 UTC
Will *you* explain to all our GMail and Yahoo user that it is "their problem" if every bugzilla message arrives to their spam folder?
Comment 8 Ben Cooksley 2016-02-20 23:03:53 UTC
Sorry, but we won't be changing this. A large proportion of our user base uses GMail in particular, and users have no control over the automatic bulk foldering of emails here. 

Plus it harms our IP and domain reputation to send mails which are widely regarded in the email community as invalid (regardless of what any RFC might say, if you look like a spammer you will be treated as one)
Comment 9 Pali Rohár 2016-02-20 23:30:11 UTC
@Nicolás: If problem is just on recipient site, then please consider at least per-user option to generate valid RFC 5322 emails. It is really bad to do such thing for all users automatically, without option to disable it or warning users "we will send you incorrect email messages". Why should all users suffer from this, even if they do not have such problems?

I will reopen this bug, because second part of this problem was not discussed nor fixed. Some emails sent to me this week has this email header:

From: via KDE Bugzilla <bugzilla_noreply@kde.org>

It is useless as it does not provide any information about user who is responsible for the change. Name "via KDE Bugzilla" looks like you forgot to include user identifier at the begining.
Comment 10 Ben Cooksley 2016-02-20 23:31:44 UTC
Usually it will say <Name> via KDE Bugzilla. You get a simple "via KDE Bugzilla" when someone hasn't entered in any name at all. I'd be open to fixes to this issue.
Comment 11 Pali Rohár 2016-02-20 23:39:39 UTC
And one more thing... Via kde.org domain are running more mailing list servers. I do not see any discarding of From header in emails send to mailing list. So why are you doing it for bugzilla emails? If somebody has problem with bugzilla, does not it have also with mailing lists?
Comment 12 Nicolás Alvarez 2016-02-20 23:49:22 UTC
People are sending DKIM-signed messages from their respective providers into our mailing lists, and we're forwarding them without modifying anything that would break the DKIM signature. Other subscribers then get messages with From: foobar@yahoo.com and the DKIM signature matches the public key from yahoo.com.

Bugzilla can't auto-generate messages with From: foobar@yahoo.com and DKIM-sign them with Yahoo's key.
Comment 13 Pali Rohár 2016-02-21 00:13:01 UTC
And for exactly such situation there is defined Sender header in RFC 5322 where should be specified kde.org address and signed by kde key.
Comment 14 Nicolás Alvarez 2016-02-21 00:36:50 UTC
"DMARC protects the domain name of the RFC5322:From field against spoofing"
Comment 15 Pali Rohár 2016-02-21 00:45:10 UTC
So that DMARC is broken by design...

And one more reason for per-bugzilla-account option to enable/disable removing From header.
Comment 16 Frédéric Sheedy 2016-02-21 20:41:56 UTC
No such feature exists in Bugzilla.
But you seem to have technical knowledge to develop it.

Do no hesitate to contribute.
Comment 17 Pali Rohár 2016-02-22 22:53:09 UTC
Maybe I have knowledge how email should like, but I have absolutely no knowledge of bugzilla and its code, so I'm not able to develop it...

I can only test implementation if it send correctly formatted emails :-)
Comment 18 Pali Rohár 2016-03-16 13:02:45 UTC
Please do something with this bug! Just having string "via KDE Bugzilla"
in From header without any other information about sender/creator makes
my work with bugzilla really hard. I --- as maintainer of Kopete IM client,
which is part of KDE --- needs some working system, not something like
this.

I really do not want to take Kopete project out of this KDE system, but
if you are not able to fix bug which is regression I would need to start
thinking what to do with it. As I'm really unable to use such broken
system.

It is probably good that users can report Kopete bugs into KDE bugzilla,
but really useless if those who want to fix them cannot easily use such
system. Please thing about it!

Anyway, I do not understand why you have problem with signing DKIM with
@kde.org domain key as lot of emails are normally received by gmail into
INBOX (not SPAM!) which has different DKIM key as domain specified in
email address in From: header. So there is probably broken also
something else...

If you need some other assistance, I can help, but I really od not like
this that nobody care about this my reported bug and there is just
silence...
Comment 19 Nicolás Alvarez 2016-03-16 23:24:25 UTC
What email client are you using? I see "Pali Rohár via KDE Bugzilla" in the notifications about your comments.
Comment 20 Pali Rohár 2016-03-16 23:39:59 UTC
Nicolás: This is independent to email client. Email notification from this bug contain "From: NAME via KDE Bugzilla <bugzilla_noreply@kde.org>" header, but email notifications from e.g. bug 360384 contain only "From: via KDE Bugzilla <bugzilla_noreply@kde.org>".
Comment 21 Nicolás Alvarez 2016-03-16 23:48:16 UTC
Then maybe we should tweak Bugzilla to use the email address if there is no name.
Comment 22 Pali Rohár 2016-03-17 14:07:01 UTC
Nicolás, thank you! Please at least do that.

Anyway, I still do not understand, why problem is with Bugzilla and not
with (KDE) mailing lists?? I see that KDE mailing lists servers set
DKIM-Signature email header with "d=kde.org" value even if From: header
contains @gmail.com domain. And such emails are normally received to
gmail users into INBOX, not to spam.

So again, why bugzilla do not set correct From: and Sender: headers?

If problem would be really with mismatching domain in (d=) in
DKIM-Signature header and domain (after @...) in From header, then also
emails sent by mailing lists would go to spam...

Also I see others (non KDE) emails with different domain in
DKIM-Signature and From. And they also go to gmail INBOX, not spam.

So I would suspect, that problem is not in DKIM header/signature, but on
different place... Please can you look at it?
Comment 23 Ben Cooksley 2016-03-18 23:05:27 UTC
Probably better would be to tweak the name to either lack the "via KDE Bugzilla" (simpler) or say "Anonymous via KDE Bugzilla" / "Unknown via KDE Bugzilla" when we don't know the name.

In terms of changing the From address itself, that can't be done due to the previously mentioned DMARC issues. Doesn't exist with mailing lists as the authorised sender actually sent it.

We attach our signature to mail that passes through KDE.org lists for legacy reasons - that setup predates the existence of DMARC. It doesn't harm (or help) things though.

Comments?
Comment 24 Pali Rohár 2016-03-18 23:09:42 UTC
Ben, but there is no difference between email sent by Bugzilla and email sent by kde.org list (both with d=kde.org DKIM signature). In both cases that email is sent by outgoing kde.org SMTP server.
Comment 25 Pali Rohár 2016-03-18 23:13:45 UTC
(In reply to Ben Cooksley from comment #23)
> Probably better would be to tweak the name to either lack the "via KDE
> Bugzilla" (simpler) or say "Anonymous via KDE Bugzilla" / "Unknown via KDE
> Bugzilla" when we don't know the name.

But we known his email address. Please do not add another absurd and useless thing into From header.
Comment 26 Nicolás Alvarez 2016-03-18 23:19:26 UTC
(In reply to Pali Rohár from comment #24)
> Ben, but there is no difference between email sent by Bugzilla and email
> sent by kde.org list (both with d=kde.org DKIM signature). In both cases
> that email is sent by outgoing kde.org SMTP server.

Yes there is a difference, there is a huge difference. If a GMail user posts to a mailing list, that message will be DKIM-signed by Google. We will *also* add a kde.org DKIM signature, but that doesn't matter. From: something@gmail.com and properly signed by Google.

A message automatically sent by Bugzilla can't possibly have a DKIM signature from your provider.
Comment 27 Pali Rohár 2016-03-18 23:32:26 UTC
Alright, I missed that second DKIM-Signarure header (from google). So yes, there are two DKIM signartures, one from google and one from kde.

But it still does not explain, why emails from other 3rd web services signed *only* by  that 3rd SMTP server are normally received into INBOX (and not to spam like you writing in case for kde bugzilla).
Comment 28 Ben Cooksley 2016-03-19 06:52:27 UTC
The rejection of 3rd party originated email for a domain will depend on the policies of that domain. Yahoo and AOL are the only ones at this stage who have set aggressive rejection DMARC policies.

Other domains, including GMail related ones do not have these aggressive rejection policies, so are not affected.
Comment 29 Pali Rohár 2016-03-20 10:54:07 UTC
Ok, so in this case it is not problem with Gmail service...

Anyway... I still do not understand your decision. You broke all emails which are sent from Bugzilla, made them non-filter-able and useless for all email clients which are RFC822/2822/5322 compliant just because Yahoo and AOL decided to start dropping bugzilla emails.

If somebody set-up email address for receiving email, then it is expected that is using email service, not Yahoo or AOL (which just do not support *all* emails).

I really do not like your decision. I thought that bugzilla (and its email feature support) is there to help KDE developers to better track and fix bugs, not to complicate it.

So please again, reconsider support for proper emails and not dropping it because of yahoo-email or aol-email support...
Comment 30 Ben Cooksley 2016-03-21 05:37:03 UTC
Sorry, but users are going to use Yahoo (they're one of the top 5 domains for receiving email from KDE.org's main email server) and we need them to be able to receive our mails. Additionally, GMail bulk folders email which fails the authentication tests specified by Yahoo / AOL (which our mail will never pass). 

This may lead to developers who use GMail/Yahoo *missing* bug emails entirely simply because the reporter used Yahoo. The same could occur for users who use GMail/Yahoo and developers that use Yahoo to respond.

You can cite the RFC's as much as you like - we aren't following them - they're irrelevant. What we follow is what is necessary to ensure our email is successfully delivered, into the Inbox, for all providers. Deliverability is what is important. 

And technically the emails we send are RFC compliant, in that our automated system's address is marked as the originator and sender of the message - which is correct. 

Further, I don't see how this complicates things considering Bugzilla attaches a number of headers containing information such as Product/Component/Status to assist in filtering.
Comment 31 Valorie Zimmerman 2016-03-21 06:02:49 UTC
Pali, the points you make have also been made by others, so you are not alone in your dismay with the changes we've had to make in email both here and on lists as well. I'm old enough to remember a time before spammers, believe it or not. Now that they produce between 75 and 95% of all the mail sent, the old RFCs have had to be left behind.

Please believe that our sysadmin team is doing all they can to support our developers in every way they can. They have sweat blood over these changes, and do not welcome repeated pleas to "go back to the old way" -- a way that did not work for a large percentage of our developers.

We are all going to have to accept reality, and use the tools we have to assist us in our work.
Comment 32 Pali Rohár 2016-03-24 10:25:14 UTC
On Monday 21 March 2016 06:02:49 Valorie Zimmerman via KDE Bugzilla wrote:
> Pali, the points you make have also been made by others, so you are not alone
> in your dismay with the changes we've had to make in email both here and on
> lists as well. I'm old enough to remember a time before spammers, believe it or
> not. Now that they produce between 75 and 95% of all the mail sent, the old
> RFCs have had to be left behind.

Hi Valorie! Two important things:

1) Email clients implement those email RFCs (RFC2822 resp. RFC5322 and
others for MIME). RFC5322 is not old for sure and you cannot ignore it
or drop it. They are "email standards".

2) If such anti-spam protection drops more than 30% of non-spam emails
then it is not only useless but must be avoided because it drop regular
non-spam messages. Same as sending all emails to /dev/null. It is
absolute anti-spam protection but totally useless.

> Please believe that our sysadmin team is doing all they can to support our
> developers in every way they can. They have sweat blood over these changes, and
> do not welcome repeated pleas to "go back to the old way" -- a way that did not
> work for a large percentage of our developers.

I believe that. But I as person who working with emails, working on
application which parse, archive, view... incoming emails cannot accept
something which mangle emails or fill useless and incorrect info to
email headers.

Also as I'm (and probably other people) filtering emails by From header
it basically means that I cannot longer do it. Plus I need to totally
drop information from From header as it contains useless, incorrect and
invalid information also for *regular* (non-spam) emails.

By filtering I mean also active filter or search rule, not just applying
them on incoming emails.

If such anti-spam protection mangle emails, damage emails or is
inconsistent to "email standards", then that it cause problems to all
email clients which are compliant to "email standards".

Doing such thing really complicates everything for me for developing. I
cannot use anymore filter/search functionality on emails from KDE
bugzilla and so those emails have very low value. And bugzilla emails
for Kopete project what what I used, which means that I do not have any
relevant tool which works for Kopete bugs.

> We are all going to have to accept reality, and use the tools we have to assist
> us in our work.

So instead of making KDE bugzilla to be compliant for "email standards"
which is implemented by email clients and what they expect (and e.g.
what I use for my work and tracking of Kopete bugs), you chose to
support two/three provides of email-like services gmail, yahoo, aol
(which are incompatible for normal emails).

This is that reality? I cannot accept such thing! It means that
indirectly you are forcing all KDE people (including developer!) to use
one of that email-like service and stop using email applications which
expect "standard emails".

I'm really not happy with this situation it just means that tools for
KDE developers are going to be only "gmail" compatible.
Comment 33 Nicolás Alvarez 2016-03-26 18:13:24 UTC
(In reply to Pali Rohár from comment #32)
> 2) If such anti-spam protection drops more than 30% of non-spam emails
> then it is not only useless but must be avoided because it drop regular
> non-spam messages. Same as sending all emails to /dev/null. It is
> absolute anti-spam protection but totally useless.

Yup, I agree.

But it's not our fault. Go complain to Yahoo and to the inventors of DMARC. We can't fix it, we can only apply workarounds to ensure our mail reaches our users.
Comment 34 Pali Rohár 2016-03-26 18:35:51 UTC
Created attachment 98102 [details]
signature.asc

On Saturday 26 March 2016 19:13:24 Nicolás Alvarez via KDE Bugzilla 
wrote:
> https://bugs.kde.org/show_bug.cgi?id=359603
> 
> --- Comment #33 from Nicolás Alvarez <nicolas.alvarez@gmail.com> ---
> (In reply to Pali Rohár from comment #32)
> 
> > 2) If such anti-spam protection drops more than 30% of non-spam
> > emails then it is not only useless but must be avoided because it
> > drop regular non-spam messages. Same as sending all emails to
> > /dev/null. It is absolute anti-spam protection but totally
> > useless.
> 
> Yup, I agree.
> 
> But it's not our fault. Go complain to Yahoo and to the inventors of
> DMARC. We can't fix it, we can only apply workarounds to ensure our
> mail reaches our users.

Ok, here is my proposed solution for this problem:

Detect if provider of email receiver enforce DMARC and if not, do not 
mangle from/sender headers. Ideally add option to bugzilla account 
configuration to enable/disable mangling from/sender headers.

This could fix problem when yahoo (and others) drop emails from kde 
bugzilla and also allow other non-yahoo (and non-etc...) users to 
receive normal emails as before.

Nicolás, what do you think about it?
Comment 35 Pali Rohár 2016-04-11 12:49:07 UTC
Any response/objections/comments about my idea?
Comment 36 Pali Rohár 2016-04-21 10:49:44 UTC
PING?
Comment 37 Ben Cooksley 2016-04-23 01:55:21 UTC
Detection of DMARC enforcement would involve making DNS queries everytime an action takes place on Bugzilla (for every single recipient none the less), which would make the system unnecessarily unreliable and prone to failure.

It would also require making invasive changes to the Bugzilla codebase - patches we would have to support going forward.

It also wouldn't help where Bugzilla emails go to mailing lists, which then have subscribers which enforce DMARC.

Finally, it would make behaviour of Bugzilla inconsistent from person to person, which is bound to result in other bugs being raised.
Comment 38 Pali Rohár 2016-05-12 12:27:27 UTC
On Saturday 23 April 2016 01:55:21 Ben Cooksley via KDE Bugzilla wrote:
> Detection of DMARC enforcement would involve making DNS queries everytime an
> action takes place on Bugzilla (for every single recipient none the less),
> which would make the system unnecessarily unreliable and prone to failure.

DNS query is already sent for each recipient when sending email. Asking
for MX record... So I do not see problem here.

But if this is problem, then we can just add option ENABLE/DISABLE
mangling of address in profile settings. Where user could specify action
for emails sent to his address.

And we can choose default settings for this option when user register
account.

> It would also require making invasive changes to the Bugzilla codebase -
> patches we would have to support going forward.

Also that simple option? It could be just one query to database (or
where are stored user options) and based on this generate email.

But here I do not know. I have not seen bugzilla source code (I guess it
is some PHP application?), so I just try to find out solution to this
problem... But with PHP I could not help.

> It also wouldn't help where Bugzilla emails go to mailing lists, which then
> have subscribers which enforce DMARC.

Mailing lists adding subject prefix or change some email headers.
Therefore they are by DMARC definitions incompatible with DMARC.

> Finally, it would make behaviour of Bugzilla inconsistent from person to
> person, which is bound to result in other bugs being raised.

Same argument can be used from another point of view: Users who enforce
DMARC on their own address are already inconsistent with whole email
infrastructure.
Comment 39 Ben Cooksley 2016-05-13 08:48:10 UTC
In regards to DNS queries, Bugzilla doesn't execute those. Bugzilla passes the email off to the local mail server, which then handles things at it's discretion. This maximises the performance while minimizing the risk of email being lost or a query failing to complete successfully.

Note that adding an option is not acceptable, as anyone with such an option enabled would potentially represent a risk to the proper operation of Bugzilla (as parts of the database are locked, then a DNS query is made to a domain which isn't functioning properly, meaning it has to wait for network timeout - during which time other users who want to make changes to Bugzilla are stuck while their pages wait for the database tables to become unlocked again).

Bugzilla is a Perl application. 

Note that anyone's opinion on DMARC's validity is completely irrelevant. The reality is that clients of Bugzilla (users and developers) use providers who either publish DMARC policies or will enforce to a certain extent DMARC policies set by domain owners. We must ensure our email is not blocked by these policies.
Comment 40 Pali Rohár 2016-05-13 08:53:44 UTC
Ok, and what about option which do not check DNS records when user
change it? That will not lock bugzilla...
Comment 41 Ben Cooksley 2016-05-14 00:58:01 UTC
Adding custom code will still add a maintenance burden, especially as our database schema will differ from upstream. It will also create unexpected behaviour and create a support burden.

We won't be changing the behaviour of Bugzilla in this regard.
Comment 42 Pali Rohár 2016-05-16 14:08:04 UTC
On Saturday 14 May 2016 00:58:01 Ben Cooksley via KDE Bugzilla wrote:
> We won't be changing the behaviour of Bugzilla in this regard.

So how then fix this bug?
Comment 43 Ben Cooksley 2016-05-17 09:29:17 UTC
This isn't a bug - it's intended behaviour and conforms with what email providers these days expect. It therefore won't be changing.
Comment 44 Pali Rohár 2016-05-17 11:06:18 UTC
On Tuesday 17 May 2016 09:29:17 Ben Cooksley via KDE Bugzilla wrote:
> This isn't a bug - it's intended behaviour and conforms with what email
> providers these days expect. It therefore won't be changing.

I cannot agree with you. It is a bug, email sent by kde bugzilla does
not conform to email standards. And more over it is regression, because
it worked correctly before.

I want to have working bugzilla again and so, I proposed solutions to
this problems.

If somebody use email provider who does not respect email standards (and
still mark that service as "email") it is his problem. But I'm really
against mangling emails, make them less usable and similar... for people
who support correct email behaviour. Still, I proposed solutions which
does not affect those "other providers". And I could think about other
options.

But I really want to see this problem fixed, not ignored from KDE admins
and KDE community. If you are going to do that, then I'm forced to move
my projects outside of (non working) KDE infrastructure... which I
really do not want...
Comment 45 Valorie Zimmerman 2016-05-18 07:25:15 UTC
(In reply to Pali Rohár from comment #44)
> On Tuesday 17 May 2016 09:29:17 Ben Cooksley via KDE Bugzilla wrote:
> > This isn't a bug - it's intended behaviour and conforms with what email
> > providers these days expect. It therefore won't be changing.
> 
> I cannot agree with you. It is a bug, email sent by kde bugzilla does
> not conform to email standards. And more over it is regression, because
> it worked correctly before.

If it is a bug, as you have been told, it is in bugzilla itself, which KDE does not develop. Please take your bug report to people who can fix it.

> I want to have working bugzilla again and so, I proposed solutions to
> this problems.

The changes you propose can't be done here.

> If somebody use email provider who does not respect email standards (and
> still mark that service as "email") it is his problem. But I'm really
> against mangling emails, make them less usable and similar... for people
> who support correct email behaviour. Still, I proposed solutions which
> does not affect those "other providers". And I could think about other
> options.

The world has changed, and so have the standards.

> But I really want to see this problem fixed, not ignored from KDE admins
> and KDE community. If you are going to do that, then I'm forced to move
> my projects outside of (non working) KDE infrastructure... which I
> really do not want...

This is comment 45! This issue has taken a great deal of sysadmin time, and is creating a great deal of sysadmin upset. We would prefer that you find a way to stay in the KDE community, but if that is impossible, we wish you the best elsewhere.
Comment 46 Ben Cooksley 2016-05-18 07:27:25 UTC
This will be my very final correspondence on this issue.

In regards to email standards, they're not relevant here. What commonly used email providers dictate as being required is relevant, as what people care about is getting their email in a readable form. The RFCs and other material are just a guideline towards that. Oh, and DMARC is an RFC, so it's part of the standards. GMail, followed by Yahoo are our top destinations for email delivery. So they set the rules.

In regards to the solutions you proposed, they either:
a) Impacted upon the reliability of Bugzilla service; or
b) Significantly increased the maintenance burden Sysadmin would face to maintain Bugzilla. And we have enough maintenance workload as it stands. Plus we'd have to author the change in the first place, and Bugzilla is written in Perl.

And in case you are wondering - every other bug tracking system out there, including vanilla Bugzilla, follows the behaviour we follow now. Our previous behaviour was a hack, added in many, many years ago, and which is not able to be continued with, due to the issues noted previously.

This issue is far from being ignored. It has been looked into and discussed here for a very long time. The behaviour the system now follows is known, and is the desired and correct behaviour.

I have no idea why your workflow requires / demands filtering based on commenter/reporter email address (in which case, may I suggest headers in the email which Bugzilla includes), but it is no longer something we can accommodate using the From field in an email.
Comment 47 Pali Rohár 2016-06-08 20:34:09 UTC
On Wednesday 18 May 2016 09:27:25 Ben Cooksley via KDE Bugzilla wrote:
> https://bugs.kde.org/show_bug.cgi?id=359603
> 
> --- Comment #46 from Ben Cooksley <bcooksley@kde.org> ---
> This will be my very final correspondence on this issue.
> 
> In regards to email standards, they're not relevant here. What
> commonly used email providers dictate as being required is relevant,
> as what people care about is getting their email in a readable form.
> The RFCs and other material are just a guideline towards that. Oh,
> and DMARC is an RFC, so it's part of the standards. GMail, followed
> by Yahoo are our top destinations for email delivery. So they set
> the rules.

Your words basically means: "I do not care about anybody except Gmail & 
Yahoo providers which set and dictate rules how emails could work". 
Sorry, but in open community and for free & open source projects, I 
cannot accept this statement.

If you are doing such thing, please explicitly write to KDE webpages:
We are not supporting email service and appreciate standards anymore. We 
only supports Gmail & Yahoo services and due to our limits we ignore all 
requests for supporting standard email service. Our users are restricted 
to use Gmail or Yahoo otherwise our service will not work correctly.

If you are *not* doing such thing, you cannot mark this report as 
INVALID. As INVALID means "we do not care about non Gmail users".

So I really would like to know what is current state. I really cannot 
accept community which declare themselves as free & open source 
software, but follow big corporations which "set the rules".

> In regards to the solutions you proposed, they either:
> a) Impacted upon the reliability of Bugzilla service; or
> b) Significantly increased the maintenance burden Sysadmin would face
> to maintain Bugzilla. And we have enough maintenance workload as it
> stands. Plus we'd have to author the change in the first place, and
> Bugzilla is written in Perl.

If you need I can help with Perl, this is something which I know.

I proposed solutions, because nobody else proposed anything. And I 
wanted to see this bug fixed. So I did everything which I could.

> And in case you are wondering - every other bug tracking system out
> there, including vanilla Bugzilla, follows the behaviour we follow
> now. Our previous behaviour was a hack, added in many, many years
> ago, and which is not able to be continued with, due to the issues
> noted previously.
> 
> This issue is far from being ignored. It has been looked into and
> discussed here for a very long time. The behaviour the system now
> follows is known, and is the desired and correct behaviour.
> 
> I have no idea why your workflow requires / demands filtering based
> on commenter/reporter email address (in which case, may I suggest
> headers in the email which Bugzilla includes), but it is no longer
> something we can accommodate using the From field in an email.

Reason is quite obvious. Every and I'm sure every one email client show 
3 important headers to user: From, To and Subject. Lot of email clients 
(and those web based too) have some index view, where is list of emails, 
e.g. clickable one email per line and on that line show again only 
From/To and Subject (sometimes also Date). And based on this information 
user open/select email.

Email headers From and To are there for 20+ years and they are not going 
to be changed. All existing email software is based on them and once you 
start rewriting them or mangling, you just broke email support...

And I'm using this information in From, To and Subject headers to 
quickly filter emails and mark what is important and what not. Also for 
searching for comments from specific people, etc...

====

And problem via total nonsense in From header

"From: via KDE Bugzilla <bugzilla_noreply@kde.org>"

is still there and did not hear anybody how and when to fix it. In 
comment 10 you wrote that this is when somebody does entered name at 
all. Should I interpret your last comment, that you are ignoring also 
this problem?
Comment 48 Nicolás Alvarez 2016-06-08 21:32:25 UTC
Since we're not going to write and maintain a custom patch to deal with this, please take this problem to upstream Bugzilla. I'm sure you would be interested in not having this problem with any other bugtracker either, rather than just KDE...