Bug 333571 - Nepomuk baloo migration can't seem to access the database
Summary: Nepomuk baloo migration can't seem to access the database
Status: RESOLVED INTENTIONAL
Alias: None
Product: Baloo
Classification: Unmaintained
Component: General (show other bugs)
Version: unspecified
Platform: Kubuntu Linux
: NOR crash
Target Milestone: ---
Assignee: Vishesh Handa
URL:
Keywords: drkonqi
: 334439 337150 338181 342812 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-04-18 07:41 UTC by J. "TAKUMI" Burton
Modified: 2015-01-14 18:08 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description J. "TAKUMI" Burton 2014-04-18 07:41:59 UTC
Application: nepomukbaloomigrator (1.0)
KDE Platform Version: 4.13.0
Qt Version: 4.8.6
Operating System: Linux 3.13.0-24-generic x86_64
Distribution: Ubuntu 14.04 LTS

-- Information about the crash:
- What I was doing when the application crashed:

"Nepomuk baloo migration" autostarted in the system tray immediately after I'd upgraded to trusty, but crashed. It seems like it simply can't start up at all, because when I tell it to restart it always just crashes again.

When I tried to run nepomukbaloomigrator in the terminal, with or without sudo, it told me 'nepomukbaloomigrator(5953): "database is locked Unable to fetch row"' a lot. I tried this several times but then I realised that even when I used Ctrl+C to kill the process it was still running in the system monitor, so that probably didn't help.

So, this is what the terminal said when I killed all the nepomukbaloomigrator processes, then ran "sudo nepomukbaloomigrator" again.

takumi@EIEN-WASHIGOKORO:~$ sudo nepomukbaloomigrator
QDBusConnection: session D-Bus connection created before QCoreApplication. Application may misbehave.
QDBusConnection: session D-Bus connection created before QCoreApplication. Application may misbehave.
takumi@EIEN-WASHIGOKORO:~$ nepomukbaloomigrator(5953): Bananana Error 
nepomukbaloomigrator(5953):  
nepomukbaloomigrator(5953): Bananana Error 
nepomukbaloomigrator(5953): Could not obtain lock for Xapian Database. This is bad 
nepomukbaloomigrator(5953):  
nepomukbaloomigrator(5953): "database is locked Unable to fetch row" 
nepomukbaloomigrator(5953): Could not obtain lock for Xapian Database. This is bad 
nepomukbaloomigrator(5953): Serious Error:   
nepomukbaloomigrator(5953): Error opening table `/home/takumi/.local/share/baloo/file/record.':
Couldn't open /home/takumi/.local/share/baloo/file/record.baseA: Too many open files
Couldn't open /home/takumi/.local/share/baloo/file/record.baseB: Too many open files
  DatabaseOpeningError: Error opening table `/home/takumi/.local/share/baloo/file/record.':
Couldn't open /home/takumi/.local/share/baloo/file/record.baseA: Too many open files
Couldn't open /home/takumi/.local/share/baloo/file/record.baseB: Too many open files
 
nepomukbaloomigrator(5953): Could not obtain lock for Xapian Database. This is bad 
nepomukbaloomigrator(5953):  
nepomukbaloomigrator(5953): "database is locked Unable to fetch row" 
nepomukbaloomigrator(5953): Could not obtain lock for Xapian Database. This is bad 
nepomukbaloomigrator(5953): Serious Error:   
nepomukbaloomigrator(5953): Error opening table `/home/takumi/.local/share/baloo/file/record.':
Couldn't open /home/takumi/.local/share/baloo/file/record.baseA: Too many open files
Couldn't open /home/takumi/.local/share/baloo/file/record.baseB: Too many open files
  DatabaseOpeningError: Error opening table `/home/takumi/.local/share/baloo/file/record.':
Couldn't open /home/takumi/.local/share/baloo/file/record.baseA: Too many open files
Couldn't open /home/takumi/.local/share/baloo/file/record.baseB: Too many open files
 
nepomukbaloomigrator(5953): Could not obtain lock for Xapian Database. This is bad 
nepomukbaloomigrator(5953):  
nepomukbaloomigrator(5953): Could not obtain lock for Xapian Database. This is bad 
nepomukbaloomigrator(5953): Serious Error:   
nepomukbaloomigrator(5953): Error opening table `/home/takumi/.local/share/baloo/file/record.':
Couldn't open /home/takumi/.local/share/baloo/file/record.baseA: Too many open files
Couldn't open /home/takumi/.local/share/baloo/file/record.baseB: Too many open files
  DatabaseOpeningError: Error opening table `/home/takumi/.local/share/baloo/file/record.':
Couldn't open /home/takumi/.local/share/baloo/file/record.baseA: Too many open files
Couldn't open /home/takumi/.local/share/baloo/file/record.baseB: Too many open files
 
KCrash: Application 'nepomukbaloomigrator' crashing...
KCrash: Attempting to start /usr/lib/kde4/libexec/drkonqi from kdeinit
sock_file=/home/takumi/.kde/socket-EIEN-WASHIGOKORO/kdeinit4__0
Unable to start Dr. Konqi
Not forwarding the crash to Apport.

I don't really know what this means, but if it helps, I had Firefox, Skype, and the bug report window open when running that command. I'm fairly sure Firefox writes to a couple of files when it runs but I'm not sure whether that's the kind of files this is talking about or not.

The crash can be reproduced every time.

-- Backtrace:
Application: NepomukBalooMigrator (nepomukbaloomigrator), signal: Segmentation fault
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[Current thread is 1 (Thread 0x7fd04e9e87c0 (LWP 2144))]

Thread 2 (Thread 0x7fd03a9c7700 (LWP 2360)):
#0  0x00007fd0485f262a in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#1  0x00007fd0485f29b9 in g_mutex_unlock () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#2  0x00007fd0485b0680 in g_main_context_prepare () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#3  0x00007fd0485b0f03 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#4  0x00007fd0485b10ec in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#5  0x00007fd04c2727be in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#6  0x00007fd04c2440af in QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#7  0x00007fd04c2443a5 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#8  0x00007fd04c140c5f in QThread::exec() () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#9  0x00007fd04c225823 in ?? () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#10 0x00007fd04c14332f in ?? () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#11 0x00007fd048a90182 in start_thread (arg=0x7fd03a9c7700) at pthread_create.c:312
#12 0x00007fd04baf330d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 1 (Thread 0x7fd04e9e87c0 (LWP 2144)):
[KCrash Handler]
#5  0x00007fd049360e78 in Xapian::Database::get_document(unsigned int) const () from /usr/lib/libxapian.so.22
#6  0x00007fd048ef44c4 in Baloo::XapianDatabase::document(unsigned int) () from /usr/lib/libbalooxapian.so.4
#7  0x00007fd04dcbd66d in Baloo::FileModifyJob::doStart() () from /usr/lib/libbaloofiles.so.4
#8  0x00007fd04c25dc1e in QObject::event(QEvent*) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#9  0x00007fd04cec1e2c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /usr/lib/x86_64-linux-gnu/libQtGui.so.4
#10 0x00007fd04cec84a0 in QApplication::notify(QObject*, QEvent*) () from /usr/lib/x86_64-linux-gnu/libQtGui.so.4
#11 0x00007fd04e3e1baa in KApplication::notify(QObject*, QEvent*) () from /usr/lib/libkdeui.so.5
#12 0x00007fd04c2454dd in QCoreApplication::notifyInternal(QObject*, QEvent*) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#13 0x00007fd04c248b3d in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#14 0x00007fd04c272f83 in ?? () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#15 0x00007fd0485b0e04 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#16 0x00007fd0485b1048 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#17 0x00007fd0485b10ec in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#18 0x00007fd04c2727a1 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#19 0x00007fd04cf63bb6 in ?? () from /usr/lib/x86_64-linux-gnu/libQtGui.so.4
#20 0x00007fd04c2440af in QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#21 0x00007fd04c2443a5 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#22 0x00007fd04c953803 in KJob::exec() () from /usr/lib/libkdecore.so.5
#23 0x0000000000404a18 in _start ()

Reported using DrKonqi
Comment 1 Vishesh Handa 2014-07-29 16:21:36 UTC
*** Bug 334439 has been marked as a duplicate of this bug. ***
Comment 2 Vishesh Handa 2014-07-29 16:25:21 UTC
*** Bug 337150 has been marked as a duplicate of this bug. ***
Comment 3 Vishesh Handa 2014-08-12 18:30:45 UTC
*** Bug 338181 has been marked as a duplicate of this bug. ***
Comment 4 Alex Savin 2014-08-18 20:35:52 UTC
I set:
ulimit -n 900000
and tried again - it is imported some metadata, but then import is seems to be stopped with the messages like that:

nepomukbaloomigrator(6610) Nepomuk2::SysTray::startMigration: "/home/pictures/screenshots/umplayer/shot0133.png" 0 "" ("HunterD")
nepomukbaloomigrator(6610): "database is locked Unable to fetch row" 
nepomukbaloomigrator(6610): "database is locked Unable to fetch row" 
nepomukbaloomigrator(6610): "database is locked Unable to fetch row" 
nepomukbaloomigrator(6610): "database is locked Unable to fetch row" 
nepomukbaloomigrator(6610) Baloo::XapianDatabase::commit: Adding: 1 docs
nepomukbaloomigrator(6610) Baloo::XapianDatabase::commit: Removing: 0 docs
nepomukbaloomigrator(6610) Baloo::XapianDatabase::commit: Xapian Committed
nepomukbaloomigrator(6610) Nepomuk2::SysTray::startMigration: "/home/pictures/screenshots/umplayer/shot0119.png" 0 "" ("HunterxHunter")
nepomukbaloomigrator(6610): "database is locked Unable to fetch row" 
nepomukbaloomigrator(6610): "database is locked Unable to fetch row" 
nepomukbaloomigrator(6610): "database is locked Unable to fetch row" 
nepomukbaloomigrator(6610): "database is locked Unable to fetch row" 
nepomukbaloomigrator(6610) Baloo::XapianDatabase::commit: Adding: 1 docs
nepomukbaloomigrator(6610) Baloo::XapianDatabase::commit: Removing: 0 docs
nepomukbaloomigrator(6610) Baloo::XapianDatabase::commit: Xapian Committed
Comment 5 Alex Savin 2014-08-19 17:47:11 UTC
After i set: ulimit -n 900000
it stop crashing and successfully migrated most(or at least a big part) of metadata, but where was a plenty of files which metadata still didn't migrated

if message:
nepomukbaloomigrator(6610): "database is locked Unable to fetch row" 
appeared 1 time - metadata migrated,

but if it appeared 4 times - migration failed
Comment 6 Vishesh Handa 2014-10-17 14:28:18 UTC
Hey guys.

I'm sorry, but I'm going to be marking this BUG as WONTFIX. It's not that the crash isn't important, it's just that reproducing it is quite hard. Additionally the underlying code has changed quite a bit between Baloo from Qt4 and Qt5.

Considering that the Nepomuk-Baloo migrator required Nepomuk, which has not been ported to Qt5, and will probably never be. The migrator no longer exists in Qt5.
Comment 7 Vishesh Handa 2015-01-14 14:48:39 UTC
*** Bug 342812 has been marked as a duplicate of this bug. ***
Comment 8 Richard Z. 2015-01-14 18:08:04 UTC
My best workaround so far:

# some really big number
vim /etc/security/limits.conf

balooctl stop; ulimit -S -n 100000; rm ~/.kde/share/config/nepomukserverrc; nepomukctl start; sleep 600; nepomukbaloomigrator

#- and try that a few times