324307 2013-08-31 05:47:58 +0000 Nepomukindexer segfaults then core dumps ad finitum using 100% CPU 2013-09-03 18:31:37 +0000 1 1 10 Unmaintained nepomuk fileindexer 4.11.0 Arch Linux Linux RESOLVED FIXED https://bbs.archlinux.org/viewtopic.php?pid=1319145 NOR crash --- 0 lemaire.philippe nepomuk-bugs bladud jsamyth nepomuk-bugs sledge.sulaweyo sudhir http://commits.kde.org/nepomuk-core/6e6e676bc519eae62d66a6b69c1a83788c7a570a 4.11.2 0 oldest_to_newest 1393958 0 lemaire.philippe 2013-08-31 05:47:58 +0000 While nepomuk is activated, systemd-journal uses 100% CPU to log nepomukindexer core dumping. I tried to start over with a removed ~/.kde4 folder, but after a few hours of indexing, the problem was back. Here is an extract of journalctl: août 31 07:08:04 archie kernel: nepomukindexer[5649]: segfault at 560 ip 00007fd16a4d14e6 sp 00007fff6c377e00 error août 31 07:08:04 archie systemd-coredump[5653]: Process 5649 (nepomukindexer) dumped core. août 31 07:08:04 archie systemd-coredump[5662]: Process 5658 (nepomukindexer) dumped core. août 31 07:08:04 archie systemd-coredump[5667]: Process 5663 (nepomukindexer) dumped core. août 31 07:08:04 archie systemd-coredump[5672]: Process 5668 (nepomukindexer) dumped core. août 31 07:08:04 archie systemd-coredump[5681]: Process 5677 (nepomukindexer) dumped core. août 31 07:08:05 archie systemd-coredump[5686]: Process 5682 (nepomukindexer) dumped core. août 31 07:08:05 archie systemd-coredump[5691]: Process 5687 (nepomukindexer) dumped core. août 31 07:08:06 archie systemd-coredump[5708]: Process 5704 (nepomukindexer) dumped core. août 31 07:08:06 archie systemd-coredump[5713]: Process 5709 (nepomukindexer) dumped core. août 31 07:08:06 archie systemd-coredump[5722]: Process 5718 (nepomukindexer) dumped core. août 31 07:08:06 archie systemd-coredump[5727]: Process 5723 (nepomukindexer) dumped core. août 31 07:08:06 archie systemd-coredump[5732]: Process 5728 (nepomukindexer) dumped core. août 31 07:08:09 archie systemd-coredump[5741]: Process 5737 (nepomukindexer) dumped core. août 31 07:08:14 archie systemd-coredump[5746]: Process 5742 (nepomukindexer) dumped core. août 31 07:08:19 archie systemd-coredump[5755]: Process 5751 (nepomukindexer) dumped core. août 31 07:08:25 archie systemd-coredump[5760]: Process 5756 (nepomukindexer) dumped core. août 31 07:08:30 archie systemd-coredump[5769]: Process 5765 (nepomukindexer) dumped core. Reproducible: Always Steps to Reproduce: 1. Let nepomuk index files Actual Results: 1. After a while, nepomukindexer segfaults, and dumps core repeatedly 2. Systemd-journal uses 100% CPU to log the core dumps, making the computer less responsive. Expected Results: Nepomuk should not segfault See link provided showing other users having the same issue since 4.11 release. Everything was fine in 4.10. 1393959 1 sudhir 2013-08-31 05:53:48 +0000 In my case, originally nepomuk was bombarding systemd-journal but now it isn't running at all. It is checked-in in desktop search settings. When I start nepomukcleaner nothing happens. Dolphin or KDE menu also don't show filters and recent items respectively. 1394074 2 bladud 2013-08-31 17:20:48 +0000 Yeah, just telling us it crashes for you is not very helpful. We need a bit more information here. On what file is it segfaulting? Can you get a backtrace? Maybe you could even send us one of these core dumps you have been gathering? Also what link? You didn't provide one. 1394080 3 sudhir 2013-08-31 17:32:01 +0000 Is there a wiki or anything I can use to provide you better data? 1394082 4 bladud 2013-08-31 17:45:50 +0000 This page is useful in general: http://techbase.kde.org/Development/Tutorials/Debugging/How_to_create_useful_crash_reports For bugs like this one the best thing you can do if to attach a copy of the file nepomukindexer crashes on to the bug report. You can find this by looking at the process list (press ctrl+esc). There will be a process with the command "nepomukindexer FILENAME". Attach that filename to this report. If you are using arch like the original reporter, there are no debug symbols, so to get a useful backtrace/core dump you have to recompile using abs. To do this, in /var/abs/extra/nepomuk-core/PKGBUILD, change: "-DCMAKE_BUILD_TYPE=Release \" to "-DCMAKE_BUILD_TYPE=Debug \" (note the backslash!) and add a line with options=(!strip) near the top. In your case, Sudhir, I think nepomuk has crashed somehow. Try disabling it in systemsettings, hitting apply, and re-enabling it. If that doesn't work, try removing your database (WARNING! this will delete your tags) with rm -R $HOME/.kde4/share/apps/nepomuk/repository/main/data/virtuosobackend and follow the above instructions. Also, helpful bug reports are really appreciated - they're the best way we have to improve the software. If you can send me the file it crashes on, I can probably fix this pretty quickly, so good luck! 1394084 5 bladud 2013-08-31 17:50:25 +0000 We also already fixed crashes in the epub extractor, the office 2007 extractor and the taglib (ie, music files) extractor since 4.11, so it is possible the problem will go away in 4.11.1 1394098 6 lemaire.philippe 2013-08-31 18:57:10 +0000 (In reply to comment #2) > Yeah, just telling us it crashes for you is not very helpful. We need a bit > more information here. On what file is it segfaulting? Can you get a > backtrace? Maybe you could even send us one of these core dumps you have > been gathering? > > Also what link? You didn't provide one. The link is right there, in the url section of the bug report: https://bbs.archlinux.org/viewtopic.php?pid=1319145 Besides, other users are having the exact same issue, so I didn't think it was due to a specific file of mine. Finally, I am unable to reproduce at the moment. Strange, because I had this for days. Maybe the file that was causing this was deleted. I'll add more info and hopefully a useful trace when I can reproduce this. 1394103 7 bladud 2013-08-31 19:38:32 +0000 > Besides, other users are having the exact same issue, so I didn't think it > was due to a specific file of mine. Probably because they have a similar type of file - nepomukindexer is the process that does the actual indexing. > Finally, I am unable to reproduce at the moment. Strange, because I had this > for days. > Maybe the file that was causing this was deleted. I'll add more info and > hopefully a useful trace when I can reproduce this. Probably. It's most likely an epub or an office document, since those are the new indexers, but it could also be webm or flac. 1394105 8 bladud 2013-08-31 19:41:58 +0000 > > Finally, I am unable to reproduce at the moment. Strange, because I had this > > for days. If you want to check for sure, move your db and get it to reindex everything, then see if it crashes. 1394640 9 jsamyth 2013-09-02 22:53:48 +0000 Here are some messages and addresses if it helps. 013-09-02T14:49:12.145135-07:00 poulsbo kernel: [ 170.549546] nepomukindexer[2376]: segfault at 8 ip 00007f622a6cb261 sp 00007fff8490c598 error 4 in libc-2.17.so[7f622a647000+1a3000] 2013-09-02T14:49:15.498135-07:00 poulsbo kernel: [ 173.902521] nepomukindexer[2381]: segfault at 560 ip 00007f2aa5a6096e sp 00007fffda4fda10 error 4 in libQtGui.so.4.8.5[7f2aa564a000+a6b000] 2013-09-02T14:49:18.774151-07:00 poulsbo kernel: [ 177.178976] nepomukindexer[2385]: segfault at 8 ip 00007f020d74b261 sp 00007fff5d80ff68 error 4 in libc-2.17.so[7f020d6c7000+1a3000] 2013-09-02T14:49:22.043141-07:00 poulsbo kernel: [ 180.447747] nepomukindexer[2389]: segfault at 8 ip 00007f913b3ba261 sp 00007fff1b325838 error 4 in libc-2.17.so[7f913b336000+1a3000] 2013-09-02T14:49:25.238146-07:00 poulsbo kernel: [ 183.642523] nepomukindexer[2393]: segfault at 560 ip 00007f0910fc696e sp 00007fffd787f0f0 error 4 in libQtGui.so.4.8.5[7f0910bb0000+a6b000] 2013-09-02T14:49:28.583244-07:00 poulsbo kernel: [ 186.987844] nepomukindexer[2397]: segfault at 560 ip 00007f23523da96e sp 00007fff773128f0 error 4 in libQtGui.so.4.8.5[7f2351fc4000+a6b000] 2013-09-02T14:49:31.902151-07:00 poulsbo kernel: [ 190.306155] nepomukindexer[2401]: segfault at 8 ip 00007f762834d261 sp 00007fffd315fb38 error 4 in libc-2.17.so[7f76282c9000+1a3000] 2013-09-02T14:49:38.630137-07:00 poulsbo kernel: [ 197.034574] nepomukindexer[2409]: segfault at 560 ip 00007f71d23b096e sp 00007fffe679c3a0 error 4 in libQtGui.so.4.8.5[7f71d1f9a000+a6b000] 2013-09-02T14:49:41.781137-07:00 poulsbo kernel: [ 200.185714] nepomukindexer[2413]: segfault at 8 ip 00007f68c09cf261 sp 00007fff59090958 error 4 in libc-2.17.so[7f68c094b000+1a3000] 2013-09-02T14:49:44.949141-07:00 poulsbo kernel: [ 203.353580] nepomukindexer[2418]: segfault at 8 ip 00007f6a26846261 sp 00007fff0ccd3d98 error 4 in libc-2.17.so[7f6a267c2000+1a3000] 1394650 10 sudhir 2013-09-03 00:18:18 +0000 I was able to get Dolphin filters back by removing virtuosobackend and it worked fine for a few days and today the bug spawned again. I haven't had time to go over the wiki posted by Simeon. As soon as I have time later this week I will post as much info as I can gather. I currently have 1.8GB of /var/log/journal. 1394651 11 bladud 2013-09-03 00:24:05 +0000 Don't worry about the wiki page - just try and find the file nepomukindexer is crashing on and attach it to the bug report. You can do this by looking at the output of "ps -ef | grep nepomuk" If you want to stop the journal going berserk, you can turn off core dump collection by following the instructions here: http://mailman.archlinux.org/pipermail/arch-general/2012-December/032375.html 1394652 12 jsamyth 2013-09-03 00:30:05 +0000 Sorry, but "ps -ef | grep nepomuk" gives not a single clue as to what file it is crashing on. What did you expect to see hare? nepomukindexer is respawning every 8 seconds , only to segfault again. 1394661 13 bladud 2013-09-03 01:24:50 +0000 n(In reply to comment #12) > Sorry, but "ps -ef | grep nepomuk" gives not a single clue as to what file > it is crashing on. > What did you expect to see hare? The indexing service (nepomukfileindexer) makes a list of not-yet-indexed files and then calls "nepomukindexer $file". I want a copy of the $file the nepomukindexer is crashing on, so I want the command line arguments of nepomukindexer. If ps is not working, could you perhaps try replacing /usr/bin/nepomukindexer with an (executable) script like: #!/usr/bin/bash echo $@ >> some-file-you-can-write-to then some-file-you-can-write-to should contain the file it crashes on. (but keep the old /usr/bin/nepomukindexer around somewhere) 1394662 14 bladud 2013-09-03 01:28:29 +0000 > If ps is not working, could you perhaps try replacing > /usr/bin/nepomukindexer with an (executable) script like: > > #!/usr/bin/bash > > echo $@ >> some-file-you-can-write-to > Better make that: #!/bin/bash echo $@ >> logfile 1394670 15 sudhir 2013-09-03 02:06:50 +0000 ps -ef | grep nepomuk donnie 759 671 0 19:54 ? 00:00:00 kdeinit4: nepomukserver [kdeinit] donnie 762 759 0 19:54 ? 00:00:00 /usr/bin/nepomukstorage donnie 1783 727 2 19:54 ? 00:00:05 /usr/bin/akonadi_nepomuk_feeder --identifier akonadi_nepomuk_feeder donnie 3812 759 0 19:55 ? 00:00:00 /usr/bin/nepomukfileindexer donnie 3813 759 0 19:55 ? 00:00:00 /usr/bin/nepomukfilewatch donnie 12348 3184 0 19:58 pts/1 00:00:00 grep nepomuk Some htop screenshots. http://i.imgur.com/6hVFWRE.png http://i.imgur.com/y90w9ie.png 1394672 16 sudhir 2013-09-03 02:29:31 +0000 So I have installed nepomuk-core with following settings in PKGBUILD options=(debug !strip) and -DCMAKE_BUILD_TYPE=Debug \ Now I am not sure how to get the actual backtraces. gdb nepomukindexer (gdb) run Starting program: /usr/bin/nepomukindexer warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000 warning: Could not load shared library symbols for linux-vdso.so.1. Do you need "set solib-search-path" or "set sysroot"? [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". Must input url of the file to be indexed[Inferior 1 (process 28307) exited with code 01] 1394677 17 jsamyth 2013-09-03 03:11:42 +0000 Ok, they are ebooks poulsbo:~ # cd /usr/bin/ poulsbo:/usr/bin # cp nepomukindexer keep-nepomukindexer poulsbo:/usr/bin # pico fake-nepomukindexer poulsbo:/usr/bin # chmod 755 fake-nepomukindexer poulsbo:/usr/bin # cp fake-nepomukindexer nepomukindexer enable nepomuk in KDE settings: jsa@poulsbo:~> tailf logfile /home/jsa/Documents/eBooks/Floyd Gibbons/''And they thought we wouldn't fight'' (300)/''And they thought we wouldn't fight'' - Floyd Gibbons.epub /home/jsa/Dropbox/eBooks-Backup-Library/William Andrew Johnston/The Apartment Next Door (297)/The Apartment Next Door - William Andrew Johnston.epub /home/jsa/Dropbox/eBooks-Backup-Library/Tim Dorsey/We Came to Observe You (530)/We Came to Observe You - Tim Dorsey.epub /home/jsa/Dropbox/eBooks-Backup-Library/Frank Richard Stockton/The late Mrs. Null (329)/The late Mrs. Null - Frank Richard Stockton.epub /home/jsa/Dropbox/eBooks-Backup-Library/Rene Gutteridge/Listen (521)/Listen - Rene Gutteridge.epub /home/jsa/Dropbox/eBooks-Backup-Library/Baron Spiegel von Und Zu Peckelsheim/The Adventures of the U-202 Submarine_ A (453)/The Adventures of the U-202 Submarine_ A - Baron Spiegel von Und Zu Peckelsheim.epub It goes on like that for some length of time... several hundred ebooks... I let it run its course. It started repeating on exactly 11 ebooks, none of the other several hundred ebooks and no other files. Email me if you need one of these books. I have several that are public domain in that list. 1394692 18 bladud 2013-09-03 06:51:40 +0000 It turns out there are in fact three separate nepomuk bugs at work here. First, there is a straightforward crash in the epub indexer Second, there was a bug where files without metadata weren't being pushed correctly. Third, I actually implemented for 4.11 a feature whereby files which crashed the indexer would not be indexed again, but it turns out this feature did not work at all - for three entirely separate reasons, no less. So, thank you for your patience everyone. I have fixed the bugs I found, and sent the non-trivial parts to Vishesh for review. Unfortunately we've missed the 4.11.1 release date, so these will have to wait until 4.11.2. 1394695 19 lemaire.philippe 2013-09-03 07:25:54 +0000 Thank you very much. Disabling indexing for now, and I'll wait for the dot releases. 1394702 20 jsamyth 2013-09-03 08:00:43 +0000 Epubs worked until yesterday. At least most of them did. I also had a mountain of source code indexed, but I can't seem to search for content in that either since the last time I applies opensuse updates late yesterday. 1394859 21 bladud 2013-09-03 18:09:04 +0000 (In reply to comment #20) > Epubs worked until yesterday. At least most of them did. > I also had a mountain of source code indexed, but I can't seem to search for > content in that either since the last time I applies opensuse updates late > yesterday. This would be 4.11 release? Everything gets re-indexed with it (more reliably) - source code is not now indexed by default, you have to turn on the "index source code" option in the indexing preferences. 1394860 22 bladud 2013-09-03 18:29:44 +0000 Git commit 37f5c9df266e93e4257dfd4c39bdcb3061a837a1 by Simeon Bird. Committed on 03/09/2013 at 04:08. Pushed by sbird into branch 'KDE/4.11'. epubextractor: Fix crash on some invalid files where epub_get_data returns an error. M +3 -2 services/fileindexer/indexer/epubextractor.cpp http://commits.kde.org/nepomuk-core/37f5c9df266e93e4257dfd4c39bdcb3061a837a1 1394861 23 bladud 2013-09-03 18:29:44 +0000 Git commit 9a4dcb1bffaef6c70a256fb3c2a759e6586e0da5 by Simeon Bird. Committed on 03/09/2013 at 04:43. Pushed by sbird into branch 'KDE/4.11'. Indexer and epubextractor: check more carefully that graphs are not empty. In particular, in the indexer we check that the graph is non-empty, but then we remove the plain text content, so we need to check again. REVIEW: 112476 M +2 -1 services/fileindexer/indexer/epubextractor.cpp M +19 -12 services/fileindexer/indexer/indexer.cpp http://commits.kde.org/nepomuk-core/9a4dcb1bffaef6c70a256fb3c2a759e6586e0da5 1394862 24 bladud 2013-09-03 18:29:47 +0000 Git commit d14700f5a02653260aa1129eb718f69d33e5c8a5 by Simeon Bird. Committed on 03/09/2013 at 06:05. Pushed by sbird into branch 'KDE/4.11'. FileIndexer: Really do not index file multiple times In commit c441198286562924ad3980527913b4888bdcb7a6 I attempted to implement not re-indexing files which failed to index. However, my code made no sense! It attempted to set kext::indexingLevel to -1 if the nepomukindexer job returned with an error. But, the KJob never set an error even if the indexer returned 1, so nothing happened. Even when this was fixed, it still didn't work, because I was updating the indexingLevel of the current filename, not the resource URI of the current filename. So, to summarise, I was silly. This one works, and an easy way to test it is to replace /usr/bin/nepomukindexer with: #!/bin/bash exit 1 M +11 -6 services/fileindexer/fileindexingjob.cpp M +11 -3 services/fileindexer/fileindexingqueue.cpp http://commits.kde.org/nepomuk-core/d14700f5a02653260aa1129eb718f69d33e5c8a5 1394863 25 bladud 2013-09-03 18:29:48 +0000 Git commit 6e6e676bc519eae62d66a6b69c1a83788c7a570a by Simeon Bird. Committed on 03/09/2013 at 06:29. Pushed by sbird into branch 'KDE/4.11'. FileIndexer: Check for nepomukindexer crashing. It is not enough to check that the indexer returns 0. We must also check that it did not crash, and, if it did, return an error. This patch also defines an error enum for FileIndexingJob. REVIEW: 112475 M +8 -4 services/fileindexer/fileindexingjob.cpp M +11 -2 services/fileindexer/fileindexingjob.h http://commits.kde.org/nepomuk-core/6e6e676bc519eae62d66a6b69c1a83788c7a570a