Bug 259347

Summary: shared data cache crash with full disk
Product: [Frameworks and Libraries] kdelibs Reporter: Jon Nelson <jnelson-kde>
Component: kshareddatacacheAssignee: kdelibs bugs <kdelibs-bugs>
Status: RESOLVED DUPLICATE    
Severity: crash CC: bugs_kde, kwin-bugs-null, lamarque, mpyne
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: openSUSE   
OS: Linux   
Latest Commit: Version Fixed In:

Description Jon Nelson 2010-12-09 17:11:10 UTC
Application: kwin (4.5.4 (KDE 4.5.4) "release 9")
KDE Platform Version: 4.5.4 (KDE 4.5.4) "release 9"
Qt Version: 4.6.3
Operating System: Linux 2.6.34.7-0.5-desktop x86_64
Distribution: "openSUSE 11.3 (x86_64)"

-- Information about the crash:
- What I was doing when the application crashed:

$HOME got full, kwin and plasma crashed before I could do anything about it.

The crash can be reproduced every time.

-- Backtrace:
Application: KWin (kwin), signal: Bus error
[KCrash Handler]
#6  memcpy () at ../sysdeps/x86_64/memcpy.S:196
#7  0x00007fb48d26ab39 in KSharedDataCache::insert (this=<value optimized out>, key=..., data=...) at /usr/include/bits/string3.h:52
#8  0x00007fb48fdabe16 in KIconLoaderPrivate::insertCachedPixmapWithPath (this=0x67c3b0, key=..., data=..., path=...) at /usr/src/debug/kdelibs-4.5.4/kdeui/icons/kiconloader.cpp:840
#9  0x00007fb48fdac8f6 in KIconLoader::loadIcon (this=0x6a6670, _name=<value optimized out>, group=KIconLoader::Small, size=32, state=0, overlays=..., path_store=0x0, canReturnNull=true)
    at /usr/src/debug/kdelibs-4.5.4/kdeui/icons/kiconloader.cpp:1318
#10 0x00007fb48fdb9189 in KWindowSystem::icon (win=<value optimized out>, width=32, height=32, scale=true, flags=12) at /usr/src/debug/kdelibs-4.5.4/kdeui/windowmanagement/kwindowsystem_x11.cpp:655
#11 0x00007fb4905001fc in KWin::Client::getIcons (this=0xf90a30) at /usr/src/debug/kdebase-workspace-4.5.4/kwin/client.cpp:1937
#12 0x00007fb49053fd4c in KWin::Client::manage (this=0xf90a30, w=<value optimized out>, isMapped=false) at /usr/src/debug/kdebase-workspace-4.5.4/kwin/manage.cpp:122
#13 0x00007fb490541de2 in KWin::Workspace::createClient (this=<value optimized out>, w=52428804, is_mapped=false) at /usr/src/debug/kdebase-workspace-4.5.4/kwin/workspace.cpp:558
#14 0x00007fb490548306 in KWin::Workspace::workspaceEvent (this=0x6e5370, e=0x7fff3441a430) at /usr/src/debug/kdebase-workspace-4.5.4/kwin/events.cpp:385
#15 0x00007fb4905483a5 in KWin::Application::x11EventFilter (this=0x7fff3441a680, e=0x7fff3441a430) at /usr/src/debug/kdebase-workspace-4.5.4/kwin/main.cpp:363
#16 0x00007fb48bd43b61 in qt_x11EventFilter (ev=0x7fff3441a430) at kernel/qapplication_x11.cpp:409
#17 0x00007fb48bd55650 in QApplication::x11ProcessEvent (this=0x7fff3441a680, event=0x7fff3441a430) at kernel/qapplication_x11.cpp:3243
#18 0x00007fb48bd7cbf4 in QEventDispatcherX11::processEvents (this=0x612010, flags=...) at kernel/qeventdispatcher_x11.cpp:132
#19 0x00007fb48cb06292 in QEventLoop::processEvents (this=<value optimized out>, flags=...) at kernel/qeventloop.cpp:149
#20 0x00007fb48cb06495 in QEventLoop::exec (this=0x7fff3441a5c0, flags=...) at kernel/qeventloop.cpp:201
#21 0x00007fb48cb0a88b in QCoreApplication::exec () at kernel/qcoreapplication.cpp:1003
#22 0x00007fb490545474 in kdemain (argc=<value optimized out>, argv=0x7fff3441a680) at /usr/src/debug/kdebase-workspace-4.5.4/kwin/main.cpp:531
#23 0x00007fb49013cb7d in __libc_start_main (main=0x400770 <main(int, char**)>, argc=1, ubp_av=0x7fff3441ac98, init=<value optimized out>, fini=<value optimized out>, 
    rtld_fini=<value optimized out>, stack_end=0x7fff3441ac88) at libc-start.c:226
#24 0x0000000000400699 in _start () at ../sysdeps/x86_64/elf/start.S:113

Reported using DrKonqi
Comment 1 Thomas Lübking 2010-12-09 17:29:26 UTC
Nothing in KDE is secured against "out of resources" issues, this crash occured on opening an icon - what happens in every GUI application, all over the time.
I also doubt that kconfig is somehow checking resources.

The situation gets worse:
if not only your $HOME path is affected (mount/quota) but the disk is really full, you'll get crashes all over the place and there's a good chance that at some point the kernel will halt.

So ultimately this would be a general kdelibs issue, but i assmue it "won't fix", sorry.
Comment 2 Pino Toscano 2010-12-09 18:25:53 UTC
*** Bug 259346 has been marked as a duplicate of this bug. ***
Comment 3 Christoph Feck 2010-12-09 20:31:30 UTC
Thomas, there is no reason a program should crash when the disk is full. Especially when the disk is just used as a cache.
Comment 4 Thomas Lübking 2010-12-09 21:33:57 UTC
No, it _should_ not, and in a perfect world it _would_ not.
Stupid thing is that reality is getting in your way:

"gg:linux shm bus error mmap" :-(

i didn't mean to imply "KDE code should require infinite resources" and actually kshareddatacache appears to properly prevent allocation of invalid shm accesses, but it's just that many applications/libraries will stumble on this.

Long story short: rather don't run out of disk space ;-)
Comment 5 Michael Pyne 2010-12-10 03:42:18 UTC
Indeed, I am pretty sure I have done all that is feasible:

* The file that backs the cache is resized only once, the first time it is used, and that resize must succeed to proceed further.
* If a new cache is being created, and the resize doesn't succeed, KSDC immediately falls back to an anonymous memory mapping, which shouldn't involve disk access at all.
* Once the file-to-memory mapping succeeds, it is very difficult to reliably tell in all situations that an upcoming valid shared memory access will actually cause a crash, at least to my knowledge. I have "The Linux Programming Interface" on my wishlist so maybe I will find a suitable solution. I am, as always, willing to be clued into better ways to do it. Should there be a good way to do this I'm very willing to add it so that KSDC can simply return appropriate null values (it is, after all, just a cache, apps should expect data to disappear at random already).

Short story is that I'm not sure why resizing the file should succeed if the disk is full, or close to full, or why existing space allocated to a .kcache file should somehow be reassigned elsewhere. Is this possibly a "sparse file" problem? Is there a way to disable that, if that's the case?
Comment 6 Oswald Buddenhagen 2010-12-10 09:22:29 UTC
a copy-on-write file system like btrfs needs space for overwriting existing files.

catching this situation in correlation with mmaps is indeed somewhat nasty.
i wonder whether it would be feasible to use the mmaps only for read access, while writing would be conventional file handle writes.
Comment 7 Christophe Marin 2010-12-12 21:05:00 UTC
*** Bug 259635 has been marked as a duplicate of this bug. ***
Comment 8 Michael Pyne 2010-12-13 01:57:11 UTC
That would likely handle the case of crash-on-update just fine. One issue is that strictly speaking a simple fetch updates the use count and "last accessed" portions of the applicable index entry. But, it's probably possible to batch those updates together so that file access can be used for those as well.

One other way to go is add a way for non-file-backed caching (and make it default). I think this is supported on both POSIX shm and Linux (with "anonymous files")
Comment 9 Thomas Lübking 2010-12-13 23:46:29 UTC
grepping kshareddatacache & qt/corelib/io/* - and regarding the little discussion there was about ext3... i wonder if this could just lack an fsync? (after Qfile::open())
Comment 10 Lamarque V. Souza 2011-02-09 21:58:15 UTC

*** This bug has been marked as a duplicate of bug 245173 ***