Bug 304982

Summary: kde-base/nepomuk-4.9.0 file indexer doesn't work
Product: nepomuk Reporter: Michael Mair-Keimberger <mmk+bugs>
Component: fileindexerAssignee: Nepomuk Bugs Coordination <nepomuk-bugs>
Status: RESOLVED FIXED    
Severity: major CC: johu, kensington, me
Priority: NOR    
Version: 4.9   
Target Milestone: ---   
Platform: Gentoo Packages   
OS: Linux   
URL: https://bugs.gentoo.org/show_bug.cgi?id=430296
Latest Commit: Version Fixed In:
Attachments: nepomuk debug output
nepomuk gdb output
nepomuk gdb fileindexer output
pid's of nepomuk via htop
nepomuk gdb fileindexer output with debug enabled
nepomuk gdb output with debug enabled

Description Michael Mair-Keimberger 2012-08-11 14:25:27 UTC
I made already an bug-report on bugs.gentoo.org, so i just copy/paste from my original bug-report:

After turning on the nepomuk file indexer in system settings -> desktop search i get following text:
did not receive a reply. possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

When i try to force file indexing i'll also get this message:
~ $ qdbus -v org.kde.nepomuk.services.nepomukfileindexer /nepomukfileindexer
Error: org.freedesktop.DBus.Error.NoReply
Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

This happens also with completely new users. Deleting anything in ~ doesn't help at all. I could reproduce it on two system: 32bit/64bit - both unstable

Stable kde doesn't seems to be affected. Just tried it there and it runs (kde-base/nepomuk-4.8.3)

Reproducible: Always

Steps to Reproduce:
1. go to system settings -> desktop search
2. turn on nepomuk file indexer
Actual Results:  
did not receive a reply. possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
System-settings also hangs/freezes when trying to disable nepomuk again. It always takes a few minutes. 

Expected Results:  
nepomuk file indexer should start to index

$ emerge --info
Portage 2.1.11.9 (default/linux/amd64/10.0, gcc-4.6.3, glibc-2.15-r2, 3.5.0-gentoo x86_64)
=================================================================
System uname: Linux-3.5.0-gentoo-x86_64-AMD_FX-tm-8150_Eight-Core_Processor-with-gentoo-2.1
Timestamp of tree: Sat, 04 Aug 2012 04:15:01 +0000
app-shells/bash:          4.2_p37
dev-lang/python:          2.7.3-r2, 3.2.3-r1
dev-util/cmake:           2.8.8-r3
dev-util/pkgconfig:       0.27
sys-apps/baselayout:      2.1-r1
sys-apps/openrc:          0.10.5
sys-apps/sandbox:         2.6
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.11.5, 1.12.2
sys-devel/binutils:       2.22.90
sys-devel/gcc:            4.6.3
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r3
sys-kernel/linux-headers: 3.5 (virtual/os-headers)
sys-libs/glibc:           2.15-r2
Repositories: gentoo x-local sunrise virtualization x11
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"                                                                                                                                                                                                                          
CFLAGS="-O2 -pipe -fomit-frame-pointer -mcx16 -msahf -maes -mpclmul -mpopcnt -mabm -mlwp -mavx -march=native -march=bdver1 -mtune=bdver1"                                                                                                             
CHOST="x86_64-pc-linux-gnu"                                                                                                                                                                                                                           
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0"                                                                                                                                           
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"                                                                                      
CXXFLAGS="-O2 -pipe -fomit-frame-pointer -mcx16 -msahf -maes -mpclmul -mpopcnt -mabm -mlwp -mavx -march=native -march=bdver1 -mtune=bdver1"                                                                                                           
DISTDIR="/usr/portage/distfiles"                                                                                                                                                                                                                      
FCFLAGS="-O2 -pipe"                                                                                                                                                                                                                                   
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles news parallel-fetch parse-eapi-ebuild-head protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"       
FFLAGS="-O2 -pipe"                                                                                                                                                                                                                                    
GENTOO_MIRRORS="http://distfiles.gentoo.org"                                                                                                                                                                                                          
LANG="en_US.utf8"                                                                                                                                                                                                                                     
LDFLAGS="-Wl,-O1 -Wl,--as-needed"                                                                                                                                                                                                                     
LINGUAS="en"                                                                                                                                                                                                                                          
MAKEOPTS="-j14"                                                                                                                                                                                                                                       
PKGDIR="/usr/portage/packages"                                                                                                                                                                                                                        
PORTAGE_CONFIGROOT="/"                                                                                                                                                                                                                                
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"                                       
PORTAGE_TMPDIR="/var/tmp"                                                                                                                                                                                                                             
PORTDIR="/usr/portage"                                                                                                                                                                                                                                
PORTDIR_OVERLAY="/media/public/overlays/local /media/public/overlays/layman/sunrise /media/public/overlays/layman/virtualization /media/public/overlays/layman/x11"                                                                                   
SYNC="rsync://192.168.2.60/gentoo-portage"                                                                                                                                                                                                            
USE="acl alsa amd64 avx berkdb bzip2 cli consolekit cracklib crypt cups cxx dbus dri exif fftw flac fortran gallium gdbm gif gpm graphite iconv icu ipv6 jabber jpeg kde kipi lzma mad mmx mmxext modules mp3 mp4 mudflap multilib ncurses nls nptl opengl openmp pam pcre phonon png policykit pppd qt4 readline sdl semantic-desktop session sip sse sse2 sse3 sse4_1 ssl ssse3 tcpd threads tiff truetype unicode v4l2 vorbis vpx x264 xcb xinerama xv zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en" PHP_TARGETS="php5-3" PYTHON_TARGETS="python3_2 python2_7" RUBY_TARGETS="ruby18 ruby19" SANE_BACKENDS="net niash" USERLAND="GNU" VIDEO_CARDS="radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON

=================================================================
                        Package Settings
=================================================================

kde-base/nepomuk-4.9.0 was built with the following:
USE="handbook (multilib) (-aqua) -debug"


app-misc/strigi-0.7.7-r1 was built with the following:
USE="dbus exif (multilib) qt4 -clucene -debug -fam -ffmpeg -hyperestraier -inotify (-log) -test"
Comment 1 Vishesh Handa 2012-08-11 14:42:08 UTC
I'm not sure what 'kde-base/nepomuk-4.9.0' is. There is a nepomuk-core repository. I can only assume you meant that. In that case, I don't understand why gentoo would want to call it by such a different name.

Anyway, could you please gdb into the fileindexer and provide me with a backtrace?

> $ gdb
> attach <pid>
> thread apply all backtrace
> detach <pid>
This should do the trick. Also if possible could you switch of nepomuk (qdbus org.kde.NepomukServer /nepomukserver quit) and restart it by running 'nepomukserver'. I'd like to see the entire debug log, if possible.

Please make sure nepomuk is enabled in kdebugdialog. Thanks.
Comment 2 Michael Mair-Keimberger 2012-08-12 12:40:39 UTC
Created attachment 73115 [details]
nepomuk debug output
Comment 3 Michael Mair-Keimberger 2012-08-12 12:41:21 UTC
Created attachment 73116 [details]
nepomuk gdb output
Comment 4 Michael Mair-Keimberger 2012-08-12 12:41:44 UTC
Created attachment 73117 [details]
nepomuk gdb fileindexer output
Comment 5 Michael Mair-Keimberger 2012-08-12 12:42:44 UTC
Created attachment 73118 [details]
pid's of nepomuk via htop
Comment 6 Michael Mair-Keimberger 2012-08-12 12:42:55 UTC
For the name thing i simply don't know. there is a package called nepomuk-core which contains, regarding their description, the core libraries of nepomuk, while the nepomuk package just contains the kde4 client.
Anyway, i really don't know about that :)

I made what you said, i hope it's correct. I made a screenshot from htop, where you see the pid's of nepomuk. with gdb i made a backtrace once from nepomuk itself and once from the pid of "nepomukservicestub nepomukfileindexer". 
I checked kdebugdialog and enabled everything which contains nepomuk. I turned off nepomuk and started it again from konsole. You'll see the output in nepomuk-debug.
Comment 7 Vishesh Handa 2012-08-20 08:05:13 UTC
(In reply to comment #4)
> Created attachment 73117 [details]
> nepomuk gdb fileindexer output

Please could you install the debugging symbols for nepomuk and then try again? I need the symbols in order to be able to debug this.

Hint: You shouldn't be getting stuff like this -

#7  0xb53d1aaa in ?? () from /usr/lib/kde4/nepomukfileindexer.so
#8  0xb53d1c61 in ?? () from /usr/lib/kde4/nepomukfileindexer.so
#9  0xb53dd482 in ?? () from /usr/lib/kde4/nepomukfileindexer.so
#10 0xb53ddc95 in ?? () from /usr/lib/kde4/nepomukfileindexer.so
#11 0xb53ddf69 in ?? () from /usr/lib/kde4/nepomukfileindexer.so
#12 0xb53cea81 in ?? () from /usr/lib/kde4/nepomukfileindexer.so

The '??' should have proper names. Thanks
Comment 8 Michael Mair-Keimberger 2012-08-20 16:36:34 UTC
Created attachment 73329 [details]
nepomuk gdb fileindexer output with debug enabled
Comment 9 Michael Mair-Keimberger 2012-08-20 16:37:19 UTC
Created attachment 73330 [details]
nepomuk gdb output with debug enabled
Comment 10 Michael Mair-Keimberger 2012-08-20 16:40:49 UTC
I've created another two gdb output's. This time with debug enabled. There are still a few "??" from QT libs, if you need them i can rebuild qt with debug too. :)
Comment 11 Vishesh Handa 2012-08-27 10:11:32 UTC
Thanks for the backtraces.

I've posted a review request, the fix should be there in 4.9.1

https://git.reviewboard.kde.org/r/106230/
Comment 12 Vishesh Handa 2012-08-29 12:44:15 UTC
Git commit 68dde513c437a218143fc8cd319b4897a64b8c20 by Vishesh Handa.
Committed on 27/08/2012 at 11:44.
Pushed by vhanda into branch 'KDE/4.9'.

FileIndexer: Fix a deadlock by making the mutex recursive

The following deadlock occurs during the first run of the
nepomuk-fileindexer, if you're on batery

QMutex::lockInternal()
QMutexLocker::QMutexLocker
Nepomuk2::IndexScheduler::suspend
Nepomuk2::EventMonitor::pauseIndexing
QMetaObject::activate(QObject*, QMetaObject
Nepomuk2::IndexScheduler::indexingSuspended
Nepomuk2::IndexScheduler::suspend
Nepomuk2::EventMonitor::pauseIndexing
Nepomuk2::EventMonitor::slotPowerManagementStatusChanged
Nepomuk2::EventMonitor::EventMonitor
Nepomuk2::FileIndexer::FileIndexer

This only happens during first run cause that is when the EventMonitor
is connected to the IndexScheduler
REVIEW: 106230

M  +1    -0    services/fileindexer/indexscheduler.cpp

http://commits.kde.org/nepomuk-core/68dde513c437a218143fc8cd319b4897a64b8c20