Bug 459128 - CI job using docker image 'kdeorg/ci-suse-qt515:latest' fails with timeout at running alkonlinequotestest
Summary: CI job using docker image 'kdeorg/ci-suse-qt515:latest' fails with timeout at...
Status: RESOLVED FIXED
Alias: None
Product: alkimia
Classification: Frameworks and Libraries
Component: CI (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: KMyMoney Devel Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-09-14 21:57 UTC by Ralf Habacker
Modified: 2024-05-01 10:26 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
Job which showing a non fatal error (23.89 KB, image/png)
2022-10-02 09:21 UTC, Ralf Habacker
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ralf Habacker 2022-09-14 21:57:56 UTC
STEPS TO REPRODUCE
1. inspect https://invent.kde.org/office/alkimia/-/jobs/474005/raw

OBSERVED RESULT
The log shows: 6/7 Test #6: alkonlinequotestest ..............***Timeout  60.41 sec 
More details are visible from the mentioned log file

EXPECTED RESULT
All tests should pass

SOFTWARE/OS VERSIONS
Linux: openSUSE /Tumbleweed
KDE Frameworks Version: 5.98.0
Qt Version: 5.15
Comment 1 Ralf Habacker 2022-09-14 22:12:51 UTC
Git commit 3e355e8e4df4a78ce03c296be806d23dfa62d9b2 by Ralf Habacker.
Committed on 14/09/2022 at 22:11.
Pushed by habacker into branch 'master'.

.gitlab-ci.yml: disable broken build jobs for now

M  +3    -2    .gitlab-ci.yml

https://invent.kde.org/office/alkimia/commit/3e355e8e4df4a78ce03c296be806d23dfa62d9b2
Comment 2 Dawid Wróbel 2022-09-28 20:55:14 UTC
test::AlkQuoteReceiver::slotStatus(  "Fetching URL https://fx-rate.net/EUR/USD..."  )

AFIK CI jobs have no Internet access, which is why this tests times out. We'll have to disable it.
Comment 3 Dawid Wróbel 2022-09-28 21:00:02 UTC
I suggest labelling the tests that require Internet access accordingly, and passing the 'exclude label' CTest param to ctest-arguments: '' in .kde-ci.yml
Comment 4 Dawid Wróbel 2022-09-28 21:16:27 UTC
Actually, my bad, I was corrected by sysadmins and CI jobs indeed *do* have Internet access, to scratch all that.
Comment 5 Ralf Habacker 2022-09-29 06:39:45 UTC
(In reply to Dawid Wróbel from comment #4)
> Actually, my bad, I was corrected by sysadmins and CI jobs indeed *do* have
> Internet access, to scratch all that.
Yes, otherwise the other online quote test would also fail.
Comment 6 Ralf Habacker 2022-09-29 06:43:22 UTC
(In reply to Ralf Habacker from comment #0)
> The log shows: 6/7 Test #6: alkonlinequotestest ..............***Timeout 
> 60.41 sec 

This is also an issue with  image 'kdeorg/ci-suse-q62:latest', see https://invent.kde.org/office/alkimia/-/jobs/499789#L1831

 SOFTWARE/OS VERSIONS
 Linux: openSUSE /Tumbleweed
 KDE Frameworks Version: 5.99
 Qt Version: 6.2
Comment 7 Ben Cooksley 2022-09-29 07:39:17 UTC
Reading the log for the test shows:
QSYSTEM: AlkOnlineQuoteTest::testLaunchWithBrowser() js: Uncaught ReferenceError: $ is not defined
QSYSTEM: AlkOnlineQuoteTest::testLaunchWithBrowser() js: Uncaught ReferenceError: jQuery is not defined
QSYSTEM: AlkOnlineQuoteTest::testLaunchWithBrowser() js: Uncaught ReferenceError: $ is not defined
QSYSTEM: AlkOnlineQuoteTest::testLaunchWithBrowser() js: Uncaught ReferenceError: jQuery is not defined

My guess would be that you are popping up a message box which is what is blocking the unit test from proceeding.
Comment 8 Ben Cooksley 2022-09-29 07:39:53 UTC
Also, by removing the CI for Alkimia you have also removed the ability for CI to be used by anything depending on Alkimia so please do not use that as a fix.
Comment 9 Ralf Habacker 2022-10-02 09:21:39 UTC
Created attachment 152545 [details]
Job which showing a non fatal error

(In reply to Ben Cooksley from comment #8)
> https://invent.kde.org/office/alkimia/-/commit/92ba9a477845b0a049c1ddce53b0920f73f6302b

The problem with this commit is that it now hides the issue completly.

The job "kde5 leap webengine" show how this can be done better - It shows that there is an error, but it is non fatal (see screenshot), which is provided by the gitlab "allow-failure" attribute (https://docs.gitlab.com/ee/ci/yaml/#allow_failure). I suggest to add similar support to .kde-ci.yml.
Comment 10 Ben Cooksley 2022-10-02 09:27:48 UTC
It doesn't hide the issue at all.

https://invent.kde.org/office/alkimia/-/pipelines/239906/test_report

We cannot have a separate job for tests as depending on the project tests have to be run either before or after installation (not to mention that we need all the build artifacts around).
Comment 11 Ralf Habacker 2024-03-08 10:38:10 UTC
This is still an issue, see https://invent.kde.org/office/alkimia/-/jobs/1633353

DEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia"
QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia"
QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() test::AlkQuoteReceiver::slotStatus(  "(Debug) symbol=EUR USD id=EUR USD..."  )
QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia"
QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia"
QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia"
QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia"
QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia"
QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia"
QINFO  : AlkOnlineQuoteTest::testLaunchWithBrowser() Sandboxing disabled by user.
QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() test::AlkQuoteReceiver::slotStatus(  "Fetching URL https://fx-rate.net/EUR/USD..."  )

      Start  8: alkdateformattest
 8/11 Test  #8: alkdateformattest ................   Passed    0.17 sec
      Start  9: alkonlinequoteprivatetest
 9/11 Test  #9: alkonlinequoteprivatetest ........   Passed    0.22 sec
      Start 10: alknewstuffenginetest
10/11 Test #10: alknewstuffenginetest ............   Passed    0.39 sec
      Start 11: appstreamtest
11/11 Test #11: appstreamtest ....................   Passed    0.92 sec

82% tests passed, 2 tests failed out of 11

Total Test time (real) =  68.40 sec

The following tests FAILED:
	  1 - appstreamtest (Failed)
	  7 - alkonlinequotestest (Timeout)
Comment 12 Ben Cooksley 2024-03-08 10:56:10 UTC
Running curl against that URL works fine in a CI container, your test must be doing something else to trigger the failure (either IPv4/IPv6 related, or otherwise cannot handle the high bandwidth connection the server has)
Comment 13 Ralf Habacker 2024-03-11 15:38:57 UTC
I added a dedicated unit test for downloading a file using the webengine browser.  On Windows https://invent.kde.org/habacker/alkimia/-/jobs/1643894 it returns: 

QDEBUG : AlkDownloadEngineTest::testDownloadError(type) 1
QWARN  : AlkDownloadEngineTest::testDownloadError(type) Qt WebEngine resources not found at C:/Craft/windows-msvc2019_64-cl/resources. Trying parent directory...
QWARN  : AlkDownloadEngineTest::testDownloadError(type) Qt WebEngine resources not found at C:/Craft/windows-msvc2019_64-cl. Trying application directory...
QWARN  : AlkDownloadEngineTest::testDownloadError(type) Qt WebEngine resources not found at C:/builds/habacker/alkimia/_build/bin. Trying fallback directory... The application MAY NOT work.
[0311/153359.917:ERROR:icu_util.cc(251)] Couldn't mmap icu data file
A crash occurred in C:\builds\habacker\alkimia\_build\bin\alkdownloadenginetest.exe.
Comment 14 Ralf Habacker 2024-03-11 15:45:09 UTC
The same test on linux using the webengine browser (see https://invent.kde.org/habacker/alkimia/-/jobs/1643891#L977) fails with a timeout.

 10 - alkdownloadenginetest (Timeout)


The same test in which the KIO framework was used to download a file (see https://invent.kde.org/habacker/alkimia/-/jobs/1643891#L976) was successful.
Comment 15 Ben Cooksley 2024-03-11 18:33:48 UTC
Not sure why we're trying to launch WebEngine here?
Comment 16 Ralf Habacker 2024-03-12 07:55:19 UTC
(In reply to Ben Cooksley from comment #15)
> Not sure why we're trying to launch WebEngine here?

One component of the alkimia library is the retrieval of online courses from a remote website by downloading the HTML content. Some websites generate the required data with javascript, which is why a browser with javascript support is required to be able to retrieve the data. The failing test attempts to do just that.
Comment 17 Ben Cooksley 2024-03-12 09:59:54 UTC
Given that both KIO and Curl are working fine, i'd suggest that something isn't quite right with how your test is utilising WebEngine - especially given it fails on both FreeBSD and Linux.

For Windows, that installation issue is a matter for the Craft developers in #kde-craft:kde.org i'm afraid. It is likely WebEngine is highly untested as very little software makes use of it.
Comment 18 Ralf Habacker 2024-03-22 00:32:16 UTC
I have added a comparable unit test with Webkit that runs without problems in a Docker container, see https://invent.kde.org/office/alkimia/-/jobs/1672271

The same job with the web engine (after adding google-droid-font package) still fails, see https://invent.kde.org/office/alkimia/-/jobs/1672270#L3113. The only difference between the two jobs is the way the HTML data is retrieved from the engine. Both engines provide a load method to load a URL, both engines provide a loadFinished() signal when the page has been loaded, and for both engines the mentioned signal is emitted. The only difference is the way in which the page source is retrieved.  Webkit provides a synchronous method toHtml() for this (https://github.com/qt/qtwebkit/blob/ab1bd15209abaf7effc51dbc2f272c5681af7223/Source/WebKit/qt/WidgetApi/qwebframe.cpp#L253) and the web engine offers an asynchronous approach, see https://doc.qt.io/qt-6/qwebenginepage.html#toHtml. In the event of an error, the defined callback (see https://invent.kde.org/office/alkimia/-/merge_requests/28/diffs?commit_id=20104bacf5cfd88a6041c2bef648d337b324e9d7#2ecb60da360d26ee29c244d2739b13d6409db4e3_125_125) is never called, which is shown in callstack frame #5 and below. This generates the timeout error .

#0  0x00007f38bfdb0a09 in poll () from /lib64/libc.so.6
#1  0x00007f38be1f2b49 in ?? () from /usr/lib64/libglib-2.0.so.0
#2  0x00007f38be1f2c5c in g_main_context_iteration () from /usr/lib64/libglib-2.0.so.0
#3  0x00007f38c043e95c in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/libQt5Core.so.5
#4  0x00007f38c03da87a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/libQt5Core.so.5
#5  0x00007f38c150ba0c in AlkWebPage::toHtml (this=0x1119610) at /mnt/src/alkwebpage.cpp:135
#6  0x00007f38c14d700f in AlkDownloadEngine::Private::slotFinishedJavaScriptEngine (this=0x111a910, ok=true) at /mnt/src/alkdownloadengine.cpp:268
#7  0x00007f38c14d75a2 in AlkDownloadEngine::Private::qt_static_metacall (_o=0x111a910, _c=QMetaObject::InvokeMetaMethod, _id=1, _a=0x7ffc07b5bbc0) at /mnt/ci-build-kf5-webengine-native/src/alkimia_autogen/include/alkdownloadengine.moc:92
#8  0x00007f38c0419785 in ?? () from /usr/lib64/libQt5Core.so.5
#9  0x00007f38bfc57192 in QWebEnginePage::loadFinished(bool) () from /usr/lib64/libQt5WebEngineWidgets.so.5
#10 0x00007f38c04104ab in QObject::event(QEvent*) () from /usr/lib64/libQt5Core.so.5
#11 0x00007f38c0f9353c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /usr/lib64/libQt5Widgets.so.5
#12 0x00007f38c0f9a2ff in QApplication::notify(QObject*, QEvent*) () from /usr/lib64/libQt5Widgets.so.5
#13 0x00007f38c03dc013 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /usr/lib64/libQt5Core.so.5
#14 0x00007f38c03dea31 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /usr/lib64/libQt5Core.so.5
#15 0x00007f38c043f2b3 in ?? () from /usr/lib64/libQt5Core.so.5
#16 0x00007f38be1f282b in g_main_context_dispatch () from /usr/lib64/libglib-2.0.so.0
#17 0x00007f38be1f2bd0 in ?? () from /usr/lib64/libglib-2.0.so.0
#18 0x00007f38be1f2c5c in g_main_context_iteration () from /usr/lib64/libglib-2.0.so.0
#19 0x00007f38c043e95c in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/libQt5Core.so.5
#20 0x00007f38c03da87a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/libQt5Core.so.5
#21 0x00007f38c14d7278 in AlkDownloadEngine::Private::downloadUrlWithJavaScriptEngine (this=0x111a910, url=...) at /mnt/src/alkdownloadengine.cpp:288
#22 0x00007f38c14d74d2 in AlkDownloadEngine::downloadUrl (this=0x7f382400e430, url=..., type=AlkDownloadEngine::JavaScriptEngine, timeout=-1) at /mnt/src/alkdownloadengine.cpp:325
#23 0x000000000040655f in AlkDownloadEngineTest::testDownloadFinished (this=0x7ffc07b5ce00) at /mnt/autotests/alkdownloadenginetest.cpp:122
#24 0x0000000000403600 in AlkDownloadEngineTest::qt_static_metacall (_o=0x7ffc07b5ce00, _c=QMetaObject::InvokeMetaMethod, _id=3, _a=0x7ffc07b5c410) at /mnt/ci-build-kf5-webengine-native/autotests/alkdownloadenginetest_autogen/EWIEGA46WW/moc_alkdownloadenginetest.cpp:93
#25 0x00007f38c03ee70d in QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericReturnArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const () from /usr/lib64/libQt5Core.so.5
#26 0x00007f38c155094a in ?? () from /usr/lib64/libQt5Test.so.5
#27 0x00007f38c1551629 in ?? () from /usr/lib64/libQt5Test.so.5
#28 0x00007f38c1551bd1 in ?? () from /usr/lib64/libQt5Test.so.5
#29 0x00007f38c15520db in QTest::qRun() () from /usr/lib64/libQt5Test.so.5
#30 0x00007f38c15525fb in QTest::qExec(QObject*, int, char**) () from /usr/lib64/libQt5Test.so.5
#31 0x000000000040680e in main (argc=1, argv=0x7ffc07b5cf28) at /mnt/autotests/alkdownloadenginetest.cpp:140
Comment 19 Ralf Habacker 2024-03-22 00:35:20 UTC
(In reply to Ralf Habacker from comment #18)
> In the event of an error, the defined callback (see
> https://invent.kde.org/office/alkimia/-/merge_requests/28/
> diffs?commit_id=20104bacf5cfd88a6041c2bef648d337b324e9d7#2ecb60da360d26ee29c2
> 44d2739b13d6409db4e3_125_125) is never called, which is shown in callstack
> frame #5 and below. This generates the timeout error .

gitlab does not show the associated content - here is it, setup as mentioned several times on the web: 

QString AlkWebPage::toHtml()
{
    QString html;
    QEventLoop loop;
    QWebEnginePage::toHtml([&html, &loop](const QString &result)
        {
            html = result;
            loop.quit();
        }
    );
    loop.exec();
    return html;
}
Comment 20 Ralf Habacker 2024-03-23 10:46:01 UTC
I have added a small test example `alkwebpagetest-webengine` for Webengine which retrieves a URL from kmymoney.org which is within the infrastructure of KDE control (see https://invent.kde.org/office/alkimia/-/jobs/1675401) and it works in a docker container.
Comment 21 Ralf Habacker 2024-03-23 11:51:12 UTC
https://doc.qt.io/qt-5/qwebenginepage.html#toHtml mentions: 

> Warning: We guarantee that the callback (resultCallback) is always called,
>  but it might be done during page destruction. 
> When QWebEnginePage is deleted, the callback is triggered
> with an invalid value and it is not safe to use the 
> corresponding QWebEnginePage or QWebEngineView instance inside it.

It looks that there are cases where this statement is not true.
Comment 22 Ralf Habacker 2024-03-25 12:05:49 UTC
(In reply to Ben Cooksley from comment #17)

> For Windows, that installation issue is a matter for the Craft developers in
> #kde-craft:kde.org i'm afraid. It is likely WebEngine is highly untested as
> very little software makes use of it.

There seem to be general problems with Webengine on Windows, see https://bugs.kde.org/show_bug.cgi?id=484446.
Comment 23 Ralf Habacker 2024-03-27 17:46:15 UTC
Now that the supported download backends only address servers from a more controlled environment (KDE infrastructure), the returns and triggered signals are checked in unit tests and the unit tests are separated per backend, the picture has become somewhat clearer. 

openSUSE Tumbleweed (https://invent.kde.org/office/alkimia/-/jobs/1687774#L1082)

All tests in relation to wegengine fail 
           7 - alkonlinequotes-webengine-test (Failed)
	 10 - alkdownloadengine-webengine-test (Failed)
	 12 - alkwebpage-webengine-test (Failed)

FreeBsD (https://invent.kde.org/office/alkimia/-/jobs/1687776#L1018)
	   7 - alkonlinequotes-webengine-test (Subprocess aborted)
	 10 - alkdownloadengine-webengine-test (Subprocess aborted)
	 12 - alkwebpage-webengine-test (Failed)

Windows (https://invent.kde.org/office/alkimia/-/jobs/1687777#L957) 
          6 - alkonlinequotes-webengine-test (Failed)
	  9 - alkdownloadengine-webengine-test (Failed)
	 11 - alkwebpage-webengine-test (Failed)

while the tests with Webkit work

openSUSE-Leap (https://invent.kde.org/office/alkimia/-/jobs/1687786#L1850

 7/18 Test #7: alkonlinequotes-webkit-test .........   Passed 1.68 sec
10/18 Test #10: alkdownloadengine-qt-test ...........   Passed 6.58 sec
11/18 Test #11: alkdownloadengine-kio-test ..........   Passed 9.95 sec
12/18 Test #12: alkdownloadengine-webkit-test .......   Passed 7.04 se
14/18 Test #14: alkwebpage-webkit-test ..............   Passed 1.14 sec
Comment 24 Ralf Habacker 2024-03-27 22:35:24 UTC
Git commit f5a1a92f00d77085d29deff50d07e3ddfaf74cdb by Ralf Habacker.
Committed on 27/03/2024 at 18:00.
Pushed by habacker into branch 'master'.

Add support to prevent possible endless downloads

In the event of timeouts, the download engines for which a local event
loop exists now output the timeout() signal.

M  +2    -0    autotests/alkdownloadenginetest.cpp
M  +14   -4    autotests/alkonlinequotetest.cpp
M  +33   -9    src/alkdownloadengine.cpp
M  +14   -1    src/alkdownloadengine.h
M  +3    -2    src/alkonlinequote_p.cpp
M  +14   -0    src/alkwebpage.cpp
M  +7    -0    src/alkwebpage.h

https://invent.kde.org/office/alkimia/-/commit/f5a1a92f00d77085d29deff50d07e3ddfaf74cdb
Comment 25 Ralf Habacker 2024-04-15 08:41:18 UTC
It turned out, that a private header from the qwebengine source code  is required to fix this issue https://invent.kde.org/office/alkimia/-/merge_requests/33/diffs?commit_id=67b2e385cc4eab67f09715c80b77248f5163c6cb
Comment 26 Ralf Habacker 2024-04-16 19:19:59 UTC
(In reply to Ralf Habacker from comment #22)
> (In reply to Ben Cooksley from comment #17)
> 
> > For Windows, that installation issue is a matter for the Craft developers in #kde-craft:kde.org i'm afraid. 

According to https://invent.kde.org/office/alkimia/-/jobs/1740161#L25 craft does not appear to be involved here. Instead, a script from the sysadmin/ci-utilities repo is used here (see https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/run-ci-build.py?ref_type=heads).
Comment 27 Ralf Habacker 2024-04-16 19:21:47 UTC
(In reply to Ralf Habacker from comment #23)

> Windows (https://invent.kde.org/office/alkimia/-/jobs/1687777#L957) 
>           6 - alkonlinequotes-webengine-test (Failed)
> 	  9 - alkdownloadengine-webengine-test (Failed)
> 	 11 - alkwebpage-webengine-test (Failed)

This issue is tracked with bug 485642.
Comment 28 Ben Cooksley 2024-04-17 08:22:49 UTC
(In reply to Ralf Habacker from comment #26)
> (In reply to Ralf Habacker from comment #22)
> > (In reply to Ben Cooksley from comment #17)
> > 
> > > For Windows, that installation issue is a matter for the Craft developers in #kde-craft:kde.org i'm afraid. 
> 
> According to https://invent.kde.org/office/alkimia/-/jobs/1740161#L25 craft
> does not appear to be involved here. Instead, a script from the
> sysadmin/ci-utilities repo is used here (see
> https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/run-ci-build.
> py?ref_type=heads).

Craft is used to provide everything not provided by Visual Studio / the Windows SDK on Windows systems. This includes Qt.
See https://invent.kde.org/sysadmin/ci-images/-/blob/master/windows-msvc2022/Dockerfile?ref_type=heads followed by https://invent.kde.org/sysadmin/ci-images/-/blob/master/windows-msvc2022-qt66/Dockerfile?ref_type=heads and
Comment 29 Bug Janitor Service 2024-04-21 13:49:09 UTC
A possibly relevant merge request was started @ https://invent.kde.org/office/alkimia/-/merge_requests/37
Comment 30 Ralf Habacker 2024-04-21 14:41:13 UTC
Git commit 72ef78f3307aea87f08ad1f6440f33607f93dc6f by Ralf Habacker.
Committed on 21/04/2024 at 13:48.
Pushed by habacker into branch 'master'.

CI: install and use local webserver on unix os builds

This could help to avoid problems when downloading external URLs, as
firewalls or intrusion detecting systems could block requests.
Related: bug 485662, bug 485896

M  +11   -6    .gitlab-ci.yml
M  +63   -12   tools/ci-build.sh
M  +12   -0    tools/ci-install.sh

https://invent.kde.org/office/alkimia/-/commit/72ef78f3307aea87f08ad1f6440f33607f93dc6f
Comment 31 Ralf Habacker 2024-05-01 10:26:18 UTC
With https://invent.kde.org/office/alkimia/-/merge_requests/33 some internal source code from qtwebengine has been added to deal with this issue.