STEPS TO REPRODUCE 1. inspect https://invent.kde.org/office/alkimia/-/jobs/474005/raw OBSERVED RESULT The log shows: 6/7 Test #6: alkonlinequotestest ..............***Timeout 60.41 sec More details are visible from the mentioned log file EXPECTED RESULT All tests should pass SOFTWARE/OS VERSIONS Linux: openSUSE /Tumbleweed KDE Frameworks Version: 5.98.0 Qt Version: 5.15
Git commit 3e355e8e4df4a78ce03c296be806d23dfa62d9b2 by Ralf Habacker. Committed on 14/09/2022 at 22:11. Pushed by habacker into branch 'master'. .gitlab-ci.yml: disable broken build jobs for now M +3 -2 .gitlab-ci.yml https://invent.kde.org/office/alkimia/commit/3e355e8e4df4a78ce03c296be806d23dfa62d9b2
test::AlkQuoteReceiver::slotStatus( "Fetching URL https://fx-rate.net/EUR/USD..." ) AFIK CI jobs have no Internet access, which is why this tests times out. We'll have to disable it.
I suggest labelling the tests that require Internet access accordingly, and passing the 'exclude label' CTest param to ctest-arguments: '' in .kde-ci.yml
Actually, my bad, I was corrected by sysadmins and CI jobs indeed *do* have Internet access, to scratch all that.
(In reply to Dawid Wróbel from comment #4) > Actually, my bad, I was corrected by sysadmins and CI jobs indeed *do* have > Internet access, to scratch all that. Yes, otherwise the other online quote test would also fail.
(In reply to Ralf Habacker from comment #0) > The log shows: 6/7 Test #6: alkonlinequotestest ..............***Timeout > 60.41 sec This is also an issue with image 'kdeorg/ci-suse-q62:latest', see https://invent.kde.org/office/alkimia/-/jobs/499789#L1831 SOFTWARE/OS VERSIONS Linux: openSUSE /Tumbleweed KDE Frameworks Version: 5.99 Qt Version: 6.2
Reading the log for the test shows: QSYSTEM: AlkOnlineQuoteTest::testLaunchWithBrowser() js: Uncaught ReferenceError: $ is not defined QSYSTEM: AlkOnlineQuoteTest::testLaunchWithBrowser() js: Uncaught ReferenceError: jQuery is not defined QSYSTEM: AlkOnlineQuoteTest::testLaunchWithBrowser() js: Uncaught ReferenceError: $ is not defined QSYSTEM: AlkOnlineQuoteTest::testLaunchWithBrowser() js: Uncaught ReferenceError: jQuery is not defined My guess would be that you are popping up a message box which is what is blocking the unit test from proceeding.
Also, by removing the CI for Alkimia you have also removed the ability for CI to be used by anything depending on Alkimia so please do not use that as a fix.
Created attachment 152545 [details] Job which showing a non fatal error (In reply to Ben Cooksley from comment #8) > https://invent.kde.org/office/alkimia/-/commit/92ba9a477845b0a049c1ddce53b0920f73f6302b The problem with this commit is that it now hides the issue completly. The job "kde5 leap webengine" show how this can be done better - It shows that there is an error, but it is non fatal (see screenshot), which is provided by the gitlab "allow-failure" attribute (https://docs.gitlab.com/ee/ci/yaml/#allow_failure). I suggest to add similar support to .kde-ci.yml.
It doesn't hide the issue at all. https://invent.kde.org/office/alkimia/-/pipelines/239906/test_report We cannot have a separate job for tests as depending on the project tests have to be run either before or after installation (not to mention that we need all the build artifacts around).
This is still an issue, see https://invent.kde.org/office/alkimia/-/jobs/1633353 DEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia" QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia" QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() test::AlkQuoteReceiver::slotStatus( "(Debug) symbol=EUR USD id=EUR USD..." ) QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia" QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia" QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia" QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia" QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia" QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() using profile "alkimia" QINFO : AlkOnlineQuoteTest::testLaunchWithBrowser() Sandboxing disabled by user. QDEBUG : AlkOnlineQuoteTest::testLaunchWithBrowser() test::AlkQuoteReceiver::slotStatus( "Fetching URL https://fx-rate.net/EUR/USD..." ) Start 8: alkdateformattest 8/11 Test #8: alkdateformattest ................ Passed 0.17 sec Start 9: alkonlinequoteprivatetest 9/11 Test #9: alkonlinequoteprivatetest ........ Passed 0.22 sec Start 10: alknewstuffenginetest 10/11 Test #10: alknewstuffenginetest ............ Passed 0.39 sec Start 11: appstreamtest 11/11 Test #11: appstreamtest .................... Passed 0.92 sec 82% tests passed, 2 tests failed out of 11 Total Test time (real) = 68.40 sec The following tests FAILED: 1 - appstreamtest (Failed) 7 - alkonlinequotestest (Timeout)
Running curl against that URL works fine in a CI container, your test must be doing something else to trigger the failure (either IPv4/IPv6 related, or otherwise cannot handle the high bandwidth connection the server has)
I added a dedicated unit test for downloading a file using the webengine browser. On Windows https://invent.kde.org/habacker/alkimia/-/jobs/1643894 it returns: QDEBUG : AlkDownloadEngineTest::testDownloadError(type) 1 QWARN : AlkDownloadEngineTest::testDownloadError(type) Qt WebEngine resources not found at C:/Craft/windows-msvc2019_64-cl/resources. Trying parent directory... QWARN : AlkDownloadEngineTest::testDownloadError(type) Qt WebEngine resources not found at C:/Craft/windows-msvc2019_64-cl. Trying application directory... QWARN : AlkDownloadEngineTest::testDownloadError(type) Qt WebEngine resources not found at C:/builds/habacker/alkimia/_build/bin. Trying fallback directory... The application MAY NOT work. [0311/153359.917:ERROR:icu_util.cc(251)] Couldn't mmap icu data file A crash occurred in C:\builds\habacker\alkimia\_build\bin\alkdownloadenginetest.exe.
The same test on linux using the webengine browser (see https://invent.kde.org/habacker/alkimia/-/jobs/1643891#L977) fails with a timeout. 10 - alkdownloadenginetest (Timeout) The same test in which the KIO framework was used to download a file (see https://invent.kde.org/habacker/alkimia/-/jobs/1643891#L976) was successful.
Not sure why we're trying to launch WebEngine here?
(In reply to Ben Cooksley from comment #15) > Not sure why we're trying to launch WebEngine here? One component of the alkimia library is the retrieval of online courses from a remote website by downloading the HTML content. Some websites generate the required data with javascript, which is why a browser with javascript support is required to be able to retrieve the data. The failing test attempts to do just that.
Given that both KIO and Curl are working fine, i'd suggest that something isn't quite right with how your test is utilising WebEngine - especially given it fails on both FreeBSD and Linux. For Windows, that installation issue is a matter for the Craft developers in #kde-craft:kde.org i'm afraid. It is likely WebEngine is highly untested as very little software makes use of it.
I have added a comparable unit test with Webkit that runs without problems in a Docker container, see https://invent.kde.org/office/alkimia/-/jobs/1672271 The same job with the web engine (after adding google-droid-font package) still fails, see https://invent.kde.org/office/alkimia/-/jobs/1672270#L3113. The only difference between the two jobs is the way the HTML data is retrieved from the engine. Both engines provide a load method to load a URL, both engines provide a loadFinished() signal when the page has been loaded, and for both engines the mentioned signal is emitted. The only difference is the way in which the page source is retrieved. Webkit provides a synchronous method toHtml() for this (https://github.com/qt/qtwebkit/blob/ab1bd15209abaf7effc51dbc2f272c5681af7223/Source/WebKit/qt/WidgetApi/qwebframe.cpp#L253) and the web engine offers an asynchronous approach, see https://doc.qt.io/qt-6/qwebenginepage.html#toHtml. In the event of an error, the defined callback (see https://invent.kde.org/office/alkimia/-/merge_requests/28/diffs?commit_id=20104bacf5cfd88a6041c2bef648d337b324e9d7#2ecb60da360d26ee29c244d2739b13d6409db4e3_125_125) is never called, which is shown in callstack frame #5 and below. This generates the timeout error . #0 0x00007f38bfdb0a09 in poll () from /lib64/libc.so.6 #1 0x00007f38be1f2b49 in ?? () from /usr/lib64/libglib-2.0.so.0 #2 0x00007f38be1f2c5c in g_main_context_iteration () from /usr/lib64/libglib-2.0.so.0 #3 0x00007f38c043e95c in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/libQt5Core.so.5 #4 0x00007f38c03da87a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/libQt5Core.so.5 #5 0x00007f38c150ba0c in AlkWebPage::toHtml (this=0x1119610) at /mnt/src/alkwebpage.cpp:135 #6 0x00007f38c14d700f in AlkDownloadEngine::Private::slotFinishedJavaScriptEngine (this=0x111a910, ok=true) at /mnt/src/alkdownloadengine.cpp:268 #7 0x00007f38c14d75a2 in AlkDownloadEngine::Private::qt_static_metacall (_o=0x111a910, _c=QMetaObject::InvokeMetaMethod, _id=1, _a=0x7ffc07b5bbc0) at /mnt/ci-build-kf5-webengine-native/src/alkimia_autogen/include/alkdownloadengine.moc:92 #8 0x00007f38c0419785 in ?? () from /usr/lib64/libQt5Core.so.5 #9 0x00007f38bfc57192 in QWebEnginePage::loadFinished(bool) () from /usr/lib64/libQt5WebEngineWidgets.so.5 #10 0x00007f38c04104ab in QObject::event(QEvent*) () from /usr/lib64/libQt5Core.so.5 #11 0x00007f38c0f9353c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /usr/lib64/libQt5Widgets.so.5 #12 0x00007f38c0f9a2ff in QApplication::notify(QObject*, QEvent*) () from /usr/lib64/libQt5Widgets.so.5 #13 0x00007f38c03dc013 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /usr/lib64/libQt5Core.so.5 #14 0x00007f38c03dea31 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /usr/lib64/libQt5Core.so.5 #15 0x00007f38c043f2b3 in ?? () from /usr/lib64/libQt5Core.so.5 #16 0x00007f38be1f282b in g_main_context_dispatch () from /usr/lib64/libglib-2.0.so.0 #17 0x00007f38be1f2bd0 in ?? () from /usr/lib64/libglib-2.0.so.0 #18 0x00007f38be1f2c5c in g_main_context_iteration () from /usr/lib64/libglib-2.0.so.0 #19 0x00007f38c043e95c in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/libQt5Core.so.5 #20 0x00007f38c03da87a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/libQt5Core.so.5 #21 0x00007f38c14d7278 in AlkDownloadEngine::Private::downloadUrlWithJavaScriptEngine (this=0x111a910, url=...) at /mnt/src/alkdownloadengine.cpp:288 #22 0x00007f38c14d74d2 in AlkDownloadEngine::downloadUrl (this=0x7f382400e430, url=..., type=AlkDownloadEngine::JavaScriptEngine, timeout=-1) at /mnt/src/alkdownloadengine.cpp:325 #23 0x000000000040655f in AlkDownloadEngineTest::testDownloadFinished (this=0x7ffc07b5ce00) at /mnt/autotests/alkdownloadenginetest.cpp:122 #24 0x0000000000403600 in AlkDownloadEngineTest::qt_static_metacall (_o=0x7ffc07b5ce00, _c=QMetaObject::InvokeMetaMethod, _id=3, _a=0x7ffc07b5c410) at /mnt/ci-build-kf5-webengine-native/autotests/alkdownloadenginetest_autogen/EWIEGA46WW/moc_alkdownloadenginetest.cpp:93 #25 0x00007f38c03ee70d in QMetaMethod::invoke(QObject*, Qt::ConnectionType, QGenericReturnArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument, QGenericArgument) const () from /usr/lib64/libQt5Core.so.5 #26 0x00007f38c155094a in ?? () from /usr/lib64/libQt5Test.so.5 #27 0x00007f38c1551629 in ?? () from /usr/lib64/libQt5Test.so.5 #28 0x00007f38c1551bd1 in ?? () from /usr/lib64/libQt5Test.so.5 #29 0x00007f38c15520db in QTest::qRun() () from /usr/lib64/libQt5Test.so.5 #30 0x00007f38c15525fb in QTest::qExec(QObject*, int, char**) () from /usr/lib64/libQt5Test.so.5 #31 0x000000000040680e in main (argc=1, argv=0x7ffc07b5cf28) at /mnt/autotests/alkdownloadenginetest.cpp:140
(In reply to Ralf Habacker from comment #18) > In the event of an error, the defined callback (see > https://invent.kde.org/office/alkimia/-/merge_requests/28/ > diffs?commit_id=20104bacf5cfd88a6041c2bef648d337b324e9d7#2ecb60da360d26ee29c2 > 44d2739b13d6409db4e3_125_125) is never called, which is shown in callstack > frame #5 and below. This generates the timeout error . gitlab does not show the associated content - here is it, setup as mentioned several times on the web: QString AlkWebPage::toHtml() { QString html; QEventLoop loop; QWebEnginePage::toHtml([&html, &loop](const QString &result) { html = result; loop.quit(); } ); loop.exec(); return html; }
I have added a small test example `alkwebpagetest-webengine` for Webengine which retrieves a URL from kmymoney.org which is within the infrastructure of KDE control (see https://invent.kde.org/office/alkimia/-/jobs/1675401) and it works in a docker container.
https://doc.qt.io/qt-5/qwebenginepage.html#toHtml mentions: > Warning: We guarantee that the callback (resultCallback) is always called, > but it might be done during page destruction. > When QWebEnginePage is deleted, the callback is triggered > with an invalid value and it is not safe to use the > corresponding QWebEnginePage or QWebEngineView instance inside it. It looks that there are cases where this statement is not true.
(In reply to Ben Cooksley from comment #17) > For Windows, that installation issue is a matter for the Craft developers in > #kde-craft:kde.org i'm afraid. It is likely WebEngine is highly untested as > very little software makes use of it. There seem to be general problems with Webengine on Windows, see https://bugs.kde.org/show_bug.cgi?id=484446.
Now that the supported download backends only address servers from a more controlled environment (KDE infrastructure), the returns and triggered signals are checked in unit tests and the unit tests are separated per backend, the picture has become somewhat clearer. openSUSE Tumbleweed (https://invent.kde.org/office/alkimia/-/jobs/1687774#L1082) All tests in relation to wegengine fail 7 - alkonlinequotes-webengine-test (Failed) 10 - alkdownloadengine-webengine-test (Failed) 12 - alkwebpage-webengine-test (Failed) FreeBsD (https://invent.kde.org/office/alkimia/-/jobs/1687776#L1018) 7 - alkonlinequotes-webengine-test (Subprocess aborted) 10 - alkdownloadengine-webengine-test (Subprocess aborted) 12 - alkwebpage-webengine-test (Failed) Windows (https://invent.kde.org/office/alkimia/-/jobs/1687777#L957) 6 - alkonlinequotes-webengine-test (Failed) 9 - alkdownloadengine-webengine-test (Failed) 11 - alkwebpage-webengine-test (Failed) while the tests with Webkit work openSUSE-Leap (https://invent.kde.org/office/alkimia/-/jobs/1687786#L1850 7/18 Test #7: alkonlinequotes-webkit-test ......... Passed 1.68 sec 10/18 Test #10: alkdownloadengine-qt-test ........... Passed 6.58 sec 11/18 Test #11: alkdownloadengine-kio-test .......... Passed 9.95 sec 12/18 Test #12: alkdownloadengine-webkit-test ....... Passed 7.04 se 14/18 Test #14: alkwebpage-webkit-test .............. Passed 1.14 sec
Git commit f5a1a92f00d77085d29deff50d07e3ddfaf74cdb by Ralf Habacker. Committed on 27/03/2024 at 18:00. Pushed by habacker into branch 'master'. Add support to prevent possible endless downloads In the event of timeouts, the download engines for which a local event loop exists now output the timeout() signal. M +2 -0 autotests/alkdownloadenginetest.cpp M +14 -4 autotests/alkonlinequotetest.cpp M +33 -9 src/alkdownloadengine.cpp M +14 -1 src/alkdownloadengine.h M +3 -2 src/alkonlinequote_p.cpp M +14 -0 src/alkwebpage.cpp M +7 -0 src/alkwebpage.h https://invent.kde.org/office/alkimia/-/commit/f5a1a92f00d77085d29deff50d07e3ddfaf74cdb
It turned out, that a private header from the qwebengine source code is required to fix this issue https://invent.kde.org/office/alkimia/-/merge_requests/33/diffs?commit_id=67b2e385cc4eab67f09715c80b77248f5163c6cb
(In reply to Ralf Habacker from comment #22) > (In reply to Ben Cooksley from comment #17) > > > For Windows, that installation issue is a matter for the Craft developers in #kde-craft:kde.org i'm afraid. According to https://invent.kde.org/office/alkimia/-/jobs/1740161#L25 craft does not appear to be involved here. Instead, a script from the sysadmin/ci-utilities repo is used here (see https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/run-ci-build.py?ref_type=heads).
(In reply to Ralf Habacker from comment #23) > Windows (https://invent.kde.org/office/alkimia/-/jobs/1687777#L957) > 6 - alkonlinequotes-webengine-test (Failed) > 9 - alkdownloadengine-webengine-test (Failed) > 11 - alkwebpage-webengine-test (Failed) This issue is tracked with bug 485642.
(In reply to Ralf Habacker from comment #26) > (In reply to Ralf Habacker from comment #22) > > (In reply to Ben Cooksley from comment #17) > > > > > For Windows, that installation issue is a matter for the Craft developers in #kde-craft:kde.org i'm afraid. > > According to https://invent.kde.org/office/alkimia/-/jobs/1740161#L25 craft > does not appear to be involved here. Instead, a script from the > sysadmin/ci-utilities repo is used here (see > https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/run-ci-build. > py?ref_type=heads). Craft is used to provide everything not provided by Visual Studio / the Windows SDK on Windows systems. This includes Qt. See https://invent.kde.org/sysadmin/ci-images/-/blob/master/windows-msvc2022/Dockerfile?ref_type=heads followed by https://invent.kde.org/sysadmin/ci-images/-/blob/master/windows-msvc2022-qt66/Dockerfile?ref_type=heads and
A possibly relevant merge request was started @ https://invent.kde.org/office/alkimia/-/merge_requests/37
Git commit 72ef78f3307aea87f08ad1f6440f33607f93dc6f by Ralf Habacker. Committed on 21/04/2024 at 13:48. Pushed by habacker into branch 'master'. CI: install and use local webserver on unix os builds This could help to avoid problems when downloading external URLs, as firewalls or intrusion detecting systems could block requests. Related: bug 485662, bug 485896 M +11 -6 .gitlab-ci.yml M +63 -12 tools/ci-build.sh M +12 -0 tools/ci-install.sh https://invent.kde.org/office/alkimia/-/commit/72ef78f3307aea87f08ad1f6440f33607f93dc6f
With https://invent.kde.org/office/alkimia/-/merge_requests/33 some internal source code from qtwebengine has been added to deal with this issue.