Bug 314385

Summary: Digikam crashes at startup: "Creating pipes for GWakeup: Too many open files"
Product: [Applications] digikam Reporter: Luke <lcampagn>
Component: Portability-RuntimeAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED DOWNSTREAM    
Severity: crash CC: caulier.gilles, edmon, lueck
Priority: NOR    
Version: 2.8.0   
Target Milestone: ---   
Platform: Ubuntu   
OS: Linux   
Latest Commit: Version Fixed In: 7.6.0
Sentry Crash Report:

Description Luke 2013-02-03 22:08:20 UTC
When I start digikam, it immediately crashes with the following errors:

    QFileSystemWatcher: failed to add paths: /home/luke/Pictures/2001
    [ .. snip: many more similar errors .. ]
    QFileSystemWatcher: failed to add paths: /home/luke/Pictures/2012/12
    digikam(19492) KIO::SlavePrivate::SlavePrivate: Connection server not listening, could not connect

    (digikam:19492): GLib-ERROR **: Creating pipes for GWakeup: Too many open files

    Trace/breakpoint trap (core dumped)

My collection is about 32k files and 950 directories.
I started seeing this error upon copying my Pictures folder (including digikam4.db and all photos) to a new machine on which I had not previously run digikam (and I have the same problem even after deleting .kde/share/apps/digikamrc).

I can work around this by running 'ulimit -n 2048' before running digikam, but probably digikam should check or modify its own limits instead of crashing.  I'll also mention that, after increasing the open file limit, digikam still crashes, but for a seemingly unrelated reason.

Reproducible: Always

Steps to Reproduce:
1. Start fresh: empty ~/Pictures, remove .kde/share/config/digikamrc
2. Create ~/Pictures with 1000 subdirs:
     for i in {1..1000}; do mkdir ~/Pictures/$i; done
3. Start digikam
Actual Results:  
Crashes with error similar to above.
Comment 1 caulier.gilles 2013-02-04 06:23:08 UTC
We need a full backtrace of crash to investiguate. Look here for details :

http://www.digikam.org/contrib

Also about too many open files error, We have already seen in the past :

https://bugs.kde.org/show_bug.cgi?id=186920

this file is probably a duplicate of #186920. Please confirm...

Gilles Caulier
Comment 2 Luke 2013-02-04 11:54:44 UTC
This bug may be a duplicate of #186920, but the error messages bear little similarity so it seemed best to keep them separate.

Interesting: If I try 'strace digikam', the error goes away for the example posted above (1000 empty directories), but persists with my actual photo collection. This tells me there might be some randomness to the bug (perhaps a race condition) and it might be tricky to reproduce elsewhere..

Here's the backtrace:

Program received signal SIGTRAP, Trace/breakpoint trap.
[Switching to Thread 0x7fffc5cb4700 (LWP 6369)]
0x00007fffea642c7f in g_logv () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
(gdb) backtr
#0  0x00007fffea642c7f in g_logv () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#1  0x00007fffea642e42 in g_log () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#2  0x00007fffea678113 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#3  0x00007fffea639d47 in g_main_context_new () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#4  0x00007ffff223b9b9 in QEventDispatcherGlibPrivate::QEventDispatcherGlibPrivate (this=0x7fffb0000960, context=<optimized out>)
    at kernel/qeventdispatcher_glib.cpp:312
#5  0x00007ffff223bab0 in QEventDispatcherGlib::QEventDispatcherGlib (this=0x7fffb00025f0, parent=0x0) at kernel/qeventdispatcher_glib.cpp:359
#6  0x00007ffff210f9a7 in QThreadPrivate::createEventDispatcher (data=data@entry=0x25d5bf0) at thread/qthread_unix.cpp:272
#7  0x00007ffff2110a94 in QThreadPrivate::start (arg=0x1c182f0) at thread/qthread_unix.cpp:322
#8  0x00007fffecc17e9a in start_thread (arg=0x7fffc5cb4700) at pthread_create.c:308
#9  0x00007ffff1339cbd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#10 0x0000000000000000 in ?? ()
Comment 3 caulier.gilles 2013-02-04 12:15:14 UTC
*** Bug 186920 has been marked as a duplicate of this bug. ***
Comment 4 caulier.gilles 2013-02-04 12:23:53 UTC
Look like crash appear in Qt library, not digiKam. I can be an internal failure of your system...

Gilles Caulier
Comment 5 Luke 2013-02-04 13:30:44 UTC
This is my interpretation: digikam is opening a large number of file handles in multiple threads, and whichever thread crosses the 1024 limit first is the one to crash. My understanding from some previous threads is that digikam creates one file handle per directory to monitor for changes, and falls back to some other mechanism when it runs out of file handles. The problem is that by acquiring all available handles, any other part of the process may become prone to crash. 

My understanding of this is pretty thin, though, so I'm happy to open an Ubuntu bug as well if you think that is appropriate.

I have tried a variety of different conditions and the only way I can get digikam to start reliably is to reduce the number of directories in my collection. Expanding the open-file limit is one obvious solution, but as I mentioned this just leads to other errors. 

Is there a way to disable directory monitoring, or to implement this in some way that does not use so many handles?
Comment 6 caulier.gilles 2013-11-25 13:02:35 UTC
Please report this problem in Ubuntu. It's a system core settings limitation....

Gilles Caulier
Comment 7 caulier.gilles 2017-07-19 04:21:26 UTC
*** Bug 314386 has been marked as a duplicate of this bug. ***