Bug 286180 - SSL connections on Windows can take >30seconds and block with max cpu
Summary: SSL connections on Windows can take >30seconds and block with max cpu
Status: RESOLVED FIXED
Alias: None
Product: kdelibs
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: 4.7
Platform: Microsoft Windows Microsoft Windows
: NOR normal
Target Milestone: ---
Assignee: KDE-Windows
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-11-09 11:44 UTC by Andre Heinecke
Modified: 2012-01-11 09:59 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andre Heinecke 2011-11-09 11:44:49 UTC
Version:           4.7 (using KDE 4.7.2) 
OS:                MS Windows

The kio_smtp shows a bug on Windows (which might well lie in the layers below)
that it is extremly slow. 
When sending a mail with the akonadi_maildispatcher agent it takes ~33seconds until it is sent.

The "slow" step according to debug output lies between the following lines debug output:

[5428] Debug:kio_smtp(5428)/kssl KIO::TCPSlaveBase::connectToHost: , Socket state: 3 Socket error: 0 , Connection succeeded: true
<--- 33 seconds pause at 100% cpu -->
5428] Debug:kio_smtp(5428)/kssl KTcpSocket::showSslErrors: "The issuer certificate of a locally looked up certificate could not be found"

When Switching to TLS:
[5548] Debug:kio_smtp(5548) SMTPProtocol::getResponse: S: >> 220 2.0.0 Ready to start TLS <<
<--- 32 second pause at 100% cpu --->
[5548] Debug:kio_smtp(5548)/kssl KTcpSocket::showSslErrors: "The issuer certificate of a locally looked up certificate could not be found"

Test was done on a server with a self signed certifcate with the root certifcate not installed but certificate errors ignored.

Reproducible: Always

Steps to Reproduce:
1. Install Kmail on Windows and configure an smtp server.
2. Send a mail.
3. Look at cpu usage

Actual Results:  
kio_smtp uses one core at 100% for 33 seconds and then sends the mail

Expected Results:  
The mail should be sent more quickly
Comment 1 Andre Heinecke 2012-01-06 08:26:04 UTC
Bad news here, with my recent master build of kdepim i ran into a probably related problem:

On start of Kontact the following happens:
<normal startup>
Output:

[3924] Debug:kontact(3924) KManageSieve::Session::dataReceived: S:  "OK "Begin TLS negotiation now""
[3924] Debug:kontact(3924) KManageSieve::Session::dataReceived: 2 "OK "Begin TLS negotiation now"" "" "" 0
[3924] Debug:kontact(3924) KManageSieve::Session::startSsl:

      <----- 42 second hang ------>

[3924] Debug:kontact(3924)/kssl KTcpSocket::showSslErrors: "The issuer certificate of a locally looked up certificate could not be found"
[3924] Debug:kontact(3924)/kssl KTcpSocket::showSslErrors: "The root CA certificate is not trusted for this purpose"
[3924] Debug:kontact(3924) KManageSieve::Session::startSsl: Initial SSL handshake failed. cipher.isNull() is false , cipher.usedBits() is 256 , the socket says: "Unknown error" and the list of SSL errors contains 2 items.

       
The problem is that now the UI freezes for that time making this bug very critical as on every start of Kontact the Window becomes unresponsive / greys out and you get a dialog "Kontact is not responding do you want to terminate it

At least this qualifies the bug more of a problem not in kio smtp but in kssl but it still leaves the possibility to have it even further down in the stack.

My next step will be to get some unit tests of kssl running on Windows and reduce the testcase.

Has anyone noticed that with other Windows applications that use ssl? Konqueror for example?
Comment 2 Andre Heinecke 2012-01-08 23:08:19 UTC
Git commit ffca1713506ecdf3a6b6ca0f987b4bde12f69442 by Andre Heinecke.
Committed on 09/01/2012 at 00:08.
Pushed by aheinecke into branch 'enterprise'.

Add workaround for openssl bug 2100

    There is now a time limit on time spent while
    walking the heap to gather entropy. This has
    already been added in 0.9.8f but was only added
    for msvc.

M  +11   -0    portage/win32libs-sources/openssl-src/openssl-1.0.0d.diff

http://commits.kde.org/emerge/ffca1713506ecdf3a6b6ca0f987b4bde12f69442
Comment 3 Andre Heinecke 2012-01-08 23:10:10 UTC
Git commit 921c0632dd811129c160a6bd99d3c2a840f714df by Andre Heinecke.
Committed on 09/01/2012 at 00:08.
Pushed by aheinecke into branch 'master'.

Add workaround for openssl bug 2100

    There is now a time limit on time spent while
    walking the heap to gather entropy. This has
    already been added in 0.9.8f but was only added
    for msvc.
(cherry picked from commit ffca1713506ecdf3a6b6ca0f987b4bde12f69442)

M  +11   -0    portage/win32libs-sources/openssl-src/openssl-1.0.0d.diff

http://commits.kde.org/emerge/921c0632dd811129c160a6bd99d3c2a840f714df
Comment 4 Andre Heinecke 2012-01-10 18:20:45 UTC
This was actually barking at the wrong tree. The issue from commit 921c0632dd811129c160a6bd99d3c2a840f714df is still valid on Windows 7 but was only contributing in a minor way to the problem. Repeated testing showed still a freeze for ~30 seconds. 

The reason for this is a bug in qt which caused the complete working directory (in our case /bin ) to be loaded in the memory as it was reading every file in there to check if it was a certificate.

I will try to get this fixed in qt for now i add the fix to emerge so that i can make a new kdepim release.
Comment 5 Andre Heinecke 2012-01-10 18:27:01 UTC
Git commit a4d7948e65cce89e5c6c493e75d4c095f4c3fd23 by Andre Heinecke.
Committed on 10/01/2012 at 19:21.
Pushed by aheinecke into branch 'enterprise'.

Fix regular expression lookup in qsslcertificate

    This correctly escapes the backslash in the first group
    otherwise those regular expressions would match on
    the first character and cause the pathPrefix to be
    set to .
    As kssl looks for user certificates in
    ~/.kde/share/apps/kssl/CaCertificates/*
    it was actually looked in ./* when the working directory
    was /bin this caused all the binaries to be read each
    time the systemcertificates were loaded.

A  +31   -0    portage/libs/qt/patches/4.7.4/Fix-regular-expression-lookup-in-qsscertificate-from.patch
M  +3    -2    portage/libs/qt/qt-4.7.1.20101224.py

http://commits.kde.org/emerge/a4d7948e65cce89e5c6c493e75d4c095f4c3fd23
Comment 6 Andre Heinecke 2012-01-11 09:54:34 UTC
Just for completness:

https://bugreports.qt.nokia.com/browse/QTBUG-23573
http://rt.openssl.org/Ticket/Display.html?id=2100

Both are fixed with emerge patches in master now, i hope that upstream will also pick it up so expect the patches to fail in future versions.

For what it's worth, i had a look at the qt code again and as there is another check for the regexp matching later the "not matching" files were not loaded into memory. But traversing all files (recursively) from the working directory downward to check the filenames against the regular experssion was apperantly expensive enough to cause that hang.
Comment 7 Till Adam 2012-01-11 09:59:00 UTC
Hey André,

nice work!

Till

On Jan 11, 2012, at 10:54 AM, Andre Heinecke wrote:

> --- Comment #6 from Andre Heinecke <aheinecke intevation de>  2012-01-11 09:54:34 ---
> Just for completness:
> 
> https://bugreports.qt.nokia.com/browse/QTBUG-23573
> http://rt.openssl.org/Ticket/Display.html?id=2100
> 
> Both are fixed with emerge patches in master now, i hope that upstream will
> also pick it up so expect the patches to fail in future versions.
> 
> For what it's worth, i had a look at the qt code again and as there is another
> check for the regexp matching later the "not matching" files were not loaded
> into memory. But traversing all files (recursively) from the working directory
> downward to check the filenames against the regular experssion was apperantly
> expensive enough to cause that hang.