Bug 501113

Summary: KWin asserts in qCeil
Product: [Plasma] kwin Reporter: Nicolas Fella <nicolas.fella>
Component: coreAssignee: KWin default assignee <kwin-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: kde, nate
Priority: HI    
Version First Reported In: master   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed/Implemented In:
Sentry Crash Report:
Attachments: Wayland log

Description Nicolas Fella 2025-03-05 19:28:25 UTC
When starting the session

#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007f43e5a9b453 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:89
#2  0x00007f43e5a41cb6 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007f43e5a2938b in __GI_abort () at abort.c:73
#4  0x00007f43e63aef50 in qAbort () at /home/nico/workspace/qt6-dev/qtbase/src/corelib/global/qassert.cpp:46
#5  0x00007f43e640a3bf in qt_maybe_message_fatal<QString&> (msgType=msgType@entry=QtFatalMsg, context=..., message=...)
    at /home/nico/workspace/qt6-dev/qtbase/src/corelib/global/qlogging.cpp:2059
#6  0x00007f43e640a530 in qt_message(QtMsgType, const QMessageLogContext &, const char *, typedef __va_list_tag __va_list_tag *)
    (msgType=msgType@entry=QtFatalMsg, context=..., msg=msg@entry=0x7f43e6f18060 "ASSERT: \"%s\" in file %s, line %d", ap=ap@entry=0x7f43dd6cb020)
    at /home/nico/workspace/qt6-dev/qtbase/src/corelib/global/qlogging.cpp:337
#7  0x00007f43e6418ba7 in QMessageLogger::fatal (this=this@entry=0x7f43dd4634a0, msg=msg@entry=0x7f43e6f18060 "ASSERT: \"%s\" in file %s, line %d")
    at /home/nico/workspace/qt6-dev/qtbase/src/corelib/global/qlogging.cpp:826
#8  0x00007f43e63af0b6 in qt_assert (assertion=0x7f43e7020400 "truncatedValue <= FP((std::numeric_limits<Result>::max)())", 
    assertion@entry=0x7f43e7045460 "truncatedValue <= FP((std::numeric_limits<Result>::max)())", file=0x7f43e701fe80 "/home/nico/workspace/qt6-dev/qtbase/src/corelib/global/qnumeric.h", 
    file@entry=0x7f43e7045380 "/home/nico/workspace/qt6-dev/qtbase/src/corelib/global/qnumeric.h", line=line@entry=508)
    at /home/nico/workspace/qt6-dev/qtbase/src/corelib/global/qassert.cpp:106
#9  0x00007f43e693bf3a in QtPrivate::qCheckedFPConversionToInteger<int, double, true, true> (value=<optimized out>)
    at /home/nico/workspace/qt6-dev/qtbase/src/corelib/global/qnumeric.h:508
#10 0x00007f43e69aac2e in qCeil<double> (v=<optimized out>) at /home/nico/workspace/qt6-dev/qtbase/src/corelib/kernel/qmath.h:30
#11 0x00007f43e69a9a22 in QRectF::toAlignedRect (this=0x7f43dd7d8a00) at /home/nico/workspace/qt6-dev/qtbase/src/corelib/tools/qrect.cpp:2337
#12 0x00007f43f259d5d8 in KWin::SurfaceInterfacePrivate::mapToBuffer (this=0x51a00001b080, region=...) at /home/nico/kde-qtdev/src/kwin/src/wayland/surface.cpp:799
#13 0x00007f43f259c172 in KWin::SurfaceInterfacePrivate::applyState (this=0x51a00001b080, next=0x519000136580) at /home/nico/kde-qtdev/src/kwin/src/wayland/surface.cpp:719
#14 0x00007f43f2662e57 in KWin::Transaction::apply (this=0x5030004df860) at /home/nico/kde-qtdev/src/kwin/src/wayland/transaction.cpp:229
#15 0x00007f43f26632f0 in KWin::Transaction::tryApply (this=0x5030004df860) at /home/nico/kde-qtdev/src/kwin/src/wayland/transaction.cpp:262
#16 0x00007f43f26620a6 in KWin::Transaction::unlock (this=0x5030004df860) at /home/nico/kde-qtdev/src/kwin/src/wayland/transaction.cpp:113
#17 0x00007f43f2661345 in operator() (__closure=0x5030004df7e0) at /home/nico/kde-qtdev/src/kwin/src/wayland/transaction.cpp:51
#18 0x00007f43f26668d8 in operator() (__closure=0x7f43dd45c840) at /home/nico/kde-qtdev/usr/include/QtCore/qobjectdefs_impl.h:116
#19 0x00007f43f266734d in QtPrivate::FunctorCallBase::call_internal<void, QtPrivate::FunctorCall<std::integer_sequence<long unsigned int>, QtPrivate::List<>, void, KWin::TransactionDmaBufLocker::TransactionDmaBufLocker(const KWin::DmaBufAttributes*)::<lambda()> >::call(KWin::TransactionDmaBufLocker::TransactionDmaBufLocker(const KWin::DmaBufAttributes*)::<lambda()>&, void**)::<lambda()> >(void **, struct {...} &&) (args=0x7f43dd5b2b60, fn=...) at /home/nico/kde-qtdev/usr/include/QtCore/qobjectdefs_impl.h:65
#20 0x00007f43f26669de in QtPrivate::FunctorCall<std::integer_sequence<long unsigned int>, QtPrivate::List<>, void, KWin::TransactionDmaBufLocker::TransactionDmaBufLocker(const KWin::DmaBufAttributes*)::<lambda()> >::call(struct {...} &, void **) (f=..., arg=Python Exception <class 'gdb.MemoryError'>: Cannot access memory at address 0xffffffffffffffc0

   #21 0x00007f43f2665ad7 in QtPrivate::FunctorCallable<KWin::TransactionDmaBufLocker::TransactionDmaBufLocker(const KWin::DmaBufAttributes*)::<lambda()> >::call<QtPrivate::List<>, void>(struct {...} &, void *, void **) (f=..., arg=0x7f43dd5b2b60) at /home/nico/kde-qtdev/usr/include/QtCore/qobjectdefs_impl.h:337
#22 0x00007f43f2664e1d in QtPrivate::QCallableObject<KWin::TransactionDmaBufLocker::TransactionDmaBufLocker(const KWin::DmaBufAttributes*)::<lambda()>, QtPrivate::List<>, void>::impl(int, QtPrivate::QSlotObjectBase *, QObject *, void **, bool *) (which=1, this_=0x5030004df7d0, r=0x508000c5f1a0, a=0x7f43dd5b2b60, ret=0x0)
    at /home/nico/kde-qtdev/usr/include/QtCore/qobjectdefs_impl.h:547
#23 0x00007f43e66c9614 in QtPrivate::QSlotObjectBase::call (this=<optimized out>, r=0x508000c5f1a0, a=0x7f43dd5b2b60)
    at /home/nico/workspace/qt6-dev/qtbase/src/corelib/kernel/qobjectdefs_impl.h:461
#24 doActivate<false> (sender=sender@entry=0x50200014b830, signal_index=<optimized out>, argv=<optimized out>, argv@entry=0x7f43dd5b2b60)
    at /home/nico/workspace/qt6-dev/qtbase/src/corelib/kernel/qobject.cpp:4231
#25 0x00007f43e66a798c in QMetaObject::activate
    (sender=sender@entry=0x50200014b830, m=m@entry=0x7f43e7210b40 <QSocketNotifier::staticMetaObject>, local_signal_index=local_signal_index@entry=0, argv=argv@entry=0x7f43dd5b2b60)
    at /home/nico/workspace/qt6-dev/qtbase/src/corelib/kernel/qobject.cpp:4291
#26 0x00007f43e66ff00e in QMetaObject::activate<void, QSocketDescriptor, QSocketNotifier::Type, QSocketNotifier::QPrivateSignal>
    (sender=0x50200014b830, mo=0x7f43e7210b40 <QSocketNotifier::staticMetaObject>, local_signal_index=0, ret=0x0)
    at /home/nico/workspace/qt6-dev/qtbase/src/corelib/kernel/qobjectdefs.h:319
#27 QSocketNotifier::activated (this=this@entry=0x50200014b830, _t1=..., _t2=QSocketNotifier::Read, _t3=...)
    at /home/nico/workspace/qt6-dev/qtbase/src/corelib/Core_autogen/include/moc_qsocketnotifier.cpp:161
#28 0x00007f43e6700e84 in QSocketNotifier::event (this=0x50200014b830, e=<optimized out>) at /home/nico/workspace/qt6-dev/qtbase/src/corelib/kernel/qsocketnotifier.cpp:327
#29 0x00007f43ea481bc5 in QApplicationPrivate::notify_helper (this=this@entry=0x515000001200, receiver=receiver@entry=0x50200014b830, e=e@entry=0x7f43dd3f90a0)
--Type <RET> for more, q to quit, c to continue without paging--c
    at /home/nico/workspace/qt6-dev/qtbase/src/widgets/kernel/qapplication.cpp:3309
#30 0x00007f43ea49d4d9 in QApplication::notify (this=0x7f43ddb07240, receiver=<optimized out>, e=<optimized out>)
    at /home/nico/workspace/qt6-dev/qtbase/src/widgets/kernel/qapplication.cpp:3259
#31 0x00007f43e65ae908 in QCoreApplication::notifyInternal2 (receiver=0x50200014b830, event=event@entry=0x7f43dd3f90a0)
    at /home/nico/workspace/qt6-dev/qtbase/src/corelib/kernel/qcoreapplication.cpp:1111
#32 0x00007f43e65aeaf9 in QCoreApplication::sendEvent (receiver=<optimized out>, event=event@entry=0x7f43dd3f90a0)
    at /home/nico/workspace/qt6-dev/qtbase/src/corelib/kernel/qcoreapplication.cpp:1551
#33 0x00007f43e6aab855 in QEventDispatcherUNIXPrivate::activateSocketNotifiers (this=this@entry=0x51200002ca40)
    at /home/nico/workspace/qt6-dev/qtbase/src/corelib/kernel/qeventdispatcher_unix.cpp:254
#34 0x00007f43e6aac708 in QEventDispatcherUNIX::processEvents (this=<optimized out>, flags=...) at /home/nico/workspace/qt6-dev/qtbase/src/corelib/kernel/qeventdispatcher_unix.cpp:470
#35 0x00007f43e8bd2f78 in QUnixEventDispatcherQPA::processEvents (this=<optimized out>, flags=...) at /home/nico/workspace/qt6-dev/qtbase/src/gui/platform/unix/qunixeventdispatcher.cpp:27
#36 0x00007f43e65ce208 in QEventLoop::processEvents (this=this@entry=0x7f43dd5fc840, flags=...) at /home/nico/workspace/qt6-dev/qtbase/src/corelib/kernel/qeventloop.cpp:104
#37 0x00007f43e65cf7c7 in QEventLoop::exec (this=this@entry=0x7f43dd5fc840, flags=...) at /home/nico/workspace/qt6-dev/qtbase/src/corelib/kernel/qeventloop.cpp:186
#38 0x00007f43e65b7bcc in QCoreApplication::exec () at /home/nico/workspace/qt6-dev/qtbase/src/corelib/kernel/qcoreapplication.cpp:1454
#39 0x00007f43e7c47508 in QGuiApplication::exec () at /home/nico/workspace/qt6-dev/qtbase/src/gui/kernel/qguiapplication.cpp:1993
#40 0x00007f43ea47eae1 in QApplication::exec () at /home/nico/workspace/qt6-dev/qtbase/src/widgets/kernel/qapplication.cpp:2576
#41 0x0000000000567386 in main (argc=14, argv=0x7ffdcd0d06c8) at /home/nico/kde-qtdev/src/kwin/src/main_wayland.cpp:622

This is triggered by https://codereview.qt-project.org/c/qt/qtbase/+/622602, so only affects Qt dev, but it points to a bug in our code since we are invoking UB
Comment 1 Nicolas Fella 2025-03-26 20:38:16 UTC
In /home/nico/kde-qtdev/src/kwin/src/wayland/surface.cpp:799 rect looks bogous: QRectF(0,0 2.14748e+09x2.14748e+09)
Comment 2 Nicolas Fella 2025-03-26 20:47:14 UTC
The values come wl_surface.damage

> [3865604.482] {mesa egl surface queue}  -> wl_surface#45.damage(0, 0, 2147483647, 2147483647)
Comment 3 Nicolas Fella 2025-03-26 20:53:13 UTC
Created attachment 179763 [details]
Wayland log
Comment 4 Bug Janitor Service 2025-03-27 07:20:34 UTC
A possibly relevant merge request was started @ https://invent.kde.org/plasma/kwin/-/merge_requests/7408
Comment 5 Vlad Zahorodnii 2025-03-27 13:31:45 UTC
Git commit bfff48e6aaeaf1f2f619e5251cf5b2134e9ac006 by Vlad Zahorodnii.
Committed on 27/03/2025 at 12:56.
Pushed by vladz into branch 'master'.

wayland: Clip surface damage

The surface damage can contain rectangles with INT32_MAX size, which will
trigger asserts in qCeil().

M  +2    -1    src/wayland/surface.cpp

https://invent.kde.org/plasma/kwin/-/commit/bfff48e6aaeaf1f2f619e5251cf5b2134e9ac006
Comment 6 Vlad Zahorodnii 2025-03-27 14:20:18 UTC
Git commit 72f59addda9915dd758aca2d43cc0f8d4eea5c5c by Vlad Zahorodnii.
Committed on 27/03/2025 at 14:03.
Pushed by vladz into branch 'Plasma/6.3'.

wayland: Clip surface damage

The surface damage can contain rectangles with INT32_MAX size, which will
trigger asserts in qCeil().


(cherry picked from commit bfff48e6aaeaf1f2f619e5251cf5b2134e9ac006)

Co-authored-by: Vlad Zahorodnii <vlad.zahorodnii@kde.org>

M  +2    -1    src/wayland/surface.cpp

https://invent.kde.org/plasma/kwin/-/commit/72f59addda9915dd758aca2d43cc0f8d4eea5c5c