Bug 320237 - Indexing seems completely broken in RC1
Summary: Indexing seems completely broken in RC1
Status: RESOLVED FIXED
Alias: None
Product: Active
Classification: Plasma
Component: General (show other bugs)
Version: Testing
Platform: Other Linux
: NOR critical
Target Milestone: unscheduled
Assignee: active
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-05-24 20:56 UTC by Thomas Pfeiffer
Modified: 2013-08-09 11:25 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Pfeiffer 2013-05-24 20:56:13 UTC
Indexing does not work correctly at all in RC1:
- Newly copied/created files do not show up in Files or Add Items
- Artist/Album info for Music files is not available
- When I go to the Desktop Search settings and de- and reactivate indexing or tap "Check for New Files" in the Advanced Settings tab, the indexer claims to be indexing the files, but they do not show up
- If I tap "Clear Index" and then "Check for New Files", it claims to index files but afterwards, no files show up in Files anymore (not even the pre-installed ones)
- Rebooting does not help

Reproducible: Always
Comment 1 Marco Martin 2013-05-25 09:54:08 UTC
does this happen on devel as well?
Comment 2 Marco Martin 2013-05-25 09:57:29 UTC
Vishesh, do you have an idea of what it could be?
Comment 3 Vishesh Handa 2013-05-25 10:12:38 UTC
Please run nepomukindexer on any one of those files, and then run 'nepomukshow' on that file. If you get an appropriate output, then please repeat after restarting nepomuk

nepomukctl restart
Comment 4 Thomas Pfeiffer 2013-05-25 12:19:40 UTC
Here is the output of nepomukindexer (same on two files):

nepomukindexer(1027)/nepomuk (strigi service): SimpleIndexError:  "_:a does not exist in the graph. In statement (_:ub, http://www.semanticdesktop.org/ontologies/2007/01/19/nie#url, _:a)" 
nepomukindexer(1027)/nepomuk (strigi service): "_:a does not exist in the graph. In statement (_:ub, http://www.semanticdesktop.org/ontologies/2007/01/19/nie#url, _:a)" 

nepomukshow does not seem to be installed
Comment 5 Vishesh Handa 2013-05-25 14:56:14 UTC
Another developer had the same problem, which I could never reproduce. Reinstalling from scratch seemed to fix the issue for him.

Let me take another look at the code.
Comment 6 Thomas Pfeiffer 2013-05-25 23:01:58 UTC
Reinstalling the same image at least did not fix the issue for me.
Comment 7 Thomas Pfeiffer 2013-06-03 20:45:06 UTC
This bug does not happen with the latest Devel image, so it seems to be specific to the rc1 image.
Comment 8 Marco Martin 2013-06-03 21:54:58 UTC
i'm trying now to build newer soprano and nepomuk-core on stable.
i see that i get a lot of message of "property doesn't exist" messages on output if i run nepomukserver on konsole.
Vishes, what version of nepomuk, soprano, virtuoso and shared-desktop-ontologies should be used? (did i forgot anything? strigi pieces still used? i don't think so?)
Comment 9 Vishesh Handa 2013-06-04 11:01:36 UTC
Could you paste the entire log?

Strigi is not required.
Use Soprano 2.9.2 or master - It does not matter that much.

Cutting edge
-----------------
SDO - Master
NepomukCore - Master?

Stable
--------
NepomukCore - 4.10.3/.4 ?
SDO - 0.10 or master
Comment 10 Marco Martin 2013-06-04 12:12:29 UTC
what i get on standard output is a flood of stuff like that:
(smells like wrong/broken desktop ontologies?)

[/usr/bin/nepomukservicestub] nepomukfileindexer(1260)/nepomuk (strigi service): SimpleIndexError:  "_:gd does not exist in the graph. In statement (_:mg, http://www.semanticdesktop.org/ontologies/2007/01/19/nie#url, _:gd)"
[/usr/bin/nepomukservicestub] nepomukfileindexer(1260)/nepomuk (strigi service): SimpleIndexError:  "_:hd does not exist in the graph. In statement (_:og, http://www.semanticdesktop.org/ontologies/2007/01/19/nie#url, _:hd)"
[/usr/bin/nepomukservicestub] nepomukfileindexer(1260)/nepomuk (strigi service): SimpleIndexError:  "_:id does not exist in the graph. In statement (_:qg, http://www.semanticdesktop.org/ontologies/2007/01/19/nie#url, _:id)"
[/usr/bin/nepomukservicestub] nepomukfileindexer(1260)/nepomuk (strigi service): SimpleIndexError:  "_:jd does not exist in the graph. In statement (_:sg, http://www.semanticdesktop.org/ontologies/2007/01/19/nie#url, _:jd)"
[/usr/bin/nepomukservicestub] nepomukfileindexer(1260)/nepomuk (strigi service): SimpleIndexError:  "_:kd does not exist in the graph. In statement (_:ug, http://www.semanticdesktop.org/ontologies/2007/01/19/nie#url, _:kd)"
[/usr/bin/nepomukservicestub] nepomukfileindexer(1260)/nepomuk (strigi service): SimpleIndexError:  "_:ld does not exist in the graph. In statement (_:wg, http://www.semanticdesktop.org/ontologies/2007/01/19/nie#url, _:ld)"
[/usr/bin/nepomukservicestub] nepomukfileindexer(1260)/nepomuk (strigi service): SimpleIndexError:  "_:md does not exist in the graph. In statement (_:yg, http://www.semanticdesktop.org/ontologies/2007/01/19/nie#url, _:md)"
[/usr/bin/nepomukservicestub] nepomukfileindexer(1260)/nepomuk (strigi service): SimpleIndexError:  "_:nd does not exist in the graph. In statement (_:ah, http://www.semanticdesktop.org/ontologies/2007/01/19/nie#url, _:nd)"


an interesting thing is that if i create a new file on an indexed directory i immediately get one of those messages, so all the filesystem monitoring part is correct, is the storing in the db that doesn't work
Comment 11 Vishesh Handa 2013-06-04 12:15:47 UTC
Could you possibly add this and then paste the debug output of nepomukstorage-

diff --git a/services/storage/datamanagementmodel.cpp b/services/storage/datamanagementmodel.cpp
index f12ab55..f736118 100644
--- a/services/storage/datamanagementmodel.cpp
+++ b/services/storage/datamanagementmodel.cpp
@@ -1298,6 +1298,7 @@ QHash<QUrl, QUrl> DataManagementModel::storeResources(const SimpleResourceGraph&
                                                       StoreIdentificationMode identificationMode,
                                                       StoreResourcesFlags flags)
 {
+    kDebug() << resources;
     if(app.isEmpty()) {
         setError(QLatin1String("storeResources: Empty application specified. This is not supported."), Soprano::Error::ErrorInvalidArgument);
         return QHash<QUrl, QUrl>();


A simple way to do it would be to shutdown nepomuk via nepomukctl stop. And then only start the storage service via nepomukstorage. Then index any one file by running the nepomukindexer on that file.
Comment 12 Marco Martin 2013-06-04 20:45:02 UTC
i still have to take the output, but after updating to nepomuk master and shared-desktop-ontologies master it seems indexing works again.

if i start only nepomukstorage it doesn't seem to give any output.
but one thing that seems to always happen is that nepomukserver and virtuoso take the whole cpu for a couple of minutes before starting to work correctly
Comment 13 Marco Martin 2013-06-05 19:12:44 UTC
Thomas, can you try to update the packages on stable and see if it works?
with nepomuk-core from master seems to work better on an installation here
Comment 14 Thomas Pfeiffer 2013-06-05 20:08:33 UTC
Unfortunately I can't test right now because my tablet is currently being used for a usability test of Plasma Active by a group of students.
Plus, I've replaced RC1 with Devel again on it because the usability test couldn't be carried out as long as the bug was still present, so O'd have the same situation as anyone else who installed RC1 on a WeTab/ExoPC.

So, could someone else with a WeTab/ExoPC please install RC1, update and test?

I can ask the students to give the tablet back to me over the weekend if necessary, but even then I wouldn't be able to test until Friday evening.
Comment 15 Thomas Pfeiffer 2013-06-15 16:01:39 UTC
I just retested with the rc (basyskom-plasma-active-four-wetab-exopc-tablet-mer-release-rc.iso) from June 14 in live mode and the problem still persists.
Comment 16 Thomas Pfeiffer 2013-08-09 11:25:45 UTC
A new file saved in KWrite is now indexed, as well as files copied from a USB stick.
This bug seems fixed now.