Version: (using KDE 4.2.0) OS: Linux Installed from: SuSE RPMs Nepomuk's systray applet showed an index size of 1.2GB for ~60000 files. I stopped nepomuk and renamed the ~/.kde4/share/apps/nepomuk folder, then started nepomuk again. After it finished indexing and no other nepomuk service showing any CPU usage, the systray applet shows an index size of ~600MB for ~60000 files. This bug could be related to an external harddisk, which is part of the index an might not always be available. (bug 180460)
please issue the following command on both nepomuk dirs: sopranocmd --backend sesame2 --dir <folder>/repository/main/data/sesame2 list | wc -l
no reply in more than two months.
As it happened again I issued the command on the new and now also too large index. It will take some hours to create a fresh one. sopranocmd --backend sesame2 --dir nepomuk/repository/main/data/sesame2 list | wc -l (Soprano::PluginManager) loading all plugins (Soprano::PluginManager) searching plugin file from "/usr/share/soprano/plugins" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/nquadparser.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found parser plugin "nquadparser" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/nquadserializer.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found serializer plugin "nquadserializer" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/raptorparser.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found parser plugin "raptorparser" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/raptorserializer.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found serializer plugin "raptorserializer" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/redlandbackend.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found backend plugin "redland" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/sesame2backend.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found backend plugin "sesame2" (Soprano::PluginManager) searching plugin file from "/usr/share/soprano/plugins" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/nquadparser.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/nquadserializer.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/raptorparser.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/raptorserializer.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/redlandbackend.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/sesame2backend.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) searching plugin file from "/etc/opt/kde3/share/soprano/plugins" (Soprano::PluginManager) searching plugin file from "/opt/kde3/share/soprano/plugins" (Soprano::PluginManager) loaded plugin from "/usr/lib64/soprano/libsoprano_sesame2backend.so" 59 [main] INFO org.openrdf.query.parser.QueryParserRegistry - Registered service class org.openrdf.query.parser.serql.SeRQLParserFactory 61 [main] INFO org.openrdf.query.parser.QueryParserRegistry - Registered service class org.openrdf.query.parser.sparql.SPARQLParserFactory Query time: 00:00:00.33 1266760
it would be interesting to see the statement count on the smaller repository, too.
yes, but it will take some hours to create it.
This is the output after nepomuk rebuilt the index and stopped using any CPU, i.e. I hope that means it really finished. The systray dialogue shows ~540MB space used, it was 1.1 GB before the rebuilt. I do not tag anything, it is just indexing. sopranocmd --backend sesame2 --dir nepomuk/repository/main/data/sesame2 list | wc -l (Soprano::PluginManager) loading all plugins (Soprano::PluginManager) searching plugin file from "/usr/share/soprano/plugins" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/nquadparser.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found parser plugin "nquadparser" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/nquadserializer.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found serializer plugin "nquadserializer" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/raptorparser.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found parser plugin "raptorparser" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/raptorserializer.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found serializer plugin "raptorserializer" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/redlandbackend.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found backend plugin "redland" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/sesame2backend.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found backend plugin "sesame2" (Soprano::PluginManager) searching plugin file from "/usr/share/soprano/plugins" (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/nquadparser.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/nquadserializer.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/raptorparser.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/raptorserializer.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/redlandbackend.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) found plugin file "/usr/share/soprano/plugins/sesame2backend.desktop" (Soprano::PluginManager) plugin has proper version. (Soprano::PluginManager) searching plugin file from "/etc/opt/kde3/share/soprano/plugins" (Soprano::PluginManager) searching plugin file from "/opt/kde3/share/soprano/plugins" (Soprano::PluginManager) loaded plugin from "/usr/lib64/soprano/libsoprano_sesame2backend.so" 45 [main] INFO org.openrdf.query.parser.QueryParserRegistry - Registered service class org.openrdf.query.parser.serql.SeRQLParserFactory 47 [main] INFO org.openrdf.query.parser.QueryParserRegistry - Registered service class org.openrdf.query.parser.sparql.SPARQLParserFactory Query time: 00:00:00.32 1192812 Files in the new index: ~/.kde4/share/apps/nepomuk/repository/main/index> ls -la insgesamt 342512 drwxr-xr-x 2 rabauke users 4096 10. Mai 14:01 . drwxr-xr-x 4 rabauke users 4096 10. Mai 10:09 .. -rwxr-xr-x 1 rabauke users 347956508 10. Mai 13:22 _2xlw.cfs -rwxr-xr-x 1 rabauke users 606944 10. Mai 13:24 _2ylg.cfs -rwxr-xr-x 1 rabauke users 584603 10. Mai 13:26 _2zkt.cfs -rwxr-xr-x 1 rabauke users 520287 10. Mai 13:27 _30kk.cfs -rwxr-xr-x 1 rabauke users 141 10. Mai 13:27 _30kk.del -rwxr-xr-x 1 rabauke users 473330 10. Mai 13:29 _31k4.cfs -rwxr-xr-x 1 rabauke users 57281 10. Mai 13:29 _31nx.cfs -rwxr-xr-x 1 rabauke users 56249 10. Mai 13:29 _31rp.cfs -rwxr-xr-x 1 rabauke users 57199 10. Mai 13:29 _31vb.cfs -rwxr-xr-x 1 rabauke users 5256 10. Mai 13:29 _31vj.cfs -rwxr-xr-x 1 rabauke users 4 10. Mai 13:29 deletable -rw------- 1 rabauke users 49 10. Mai 14:01 .directory -rwxr-xr-x 1 rabauke users 110 10. Mai 13:29 segments Files in the old index: ~/.kde4/share/apps/nepomuk_neu_alt/repository/main/index> ls -la insgesamt 949744 drwxr-xr-x 2 rabauke users 4096 10. Mai 14:01 . drwxr-xr-x 4 rabauke users 4096 18. Feb 16:59 .. -rwxr-xr-x 1 rabauke users 357435751 18. Feb 19:18 _2x2z.cfs -rwxr-xr-x 1 rabauke users 13068 9. Mai 13:48 _2x2z.del -rwxr-xr-x 1 rabauke users 6212608 10. Mär 13:32 _5kg5.tmp -rwxr-xr-x 1 rabauke users 4784128 13. Mär 02:45 _5t7r.fdt -rwxr-xr-x 1 rabauke users 0 13. Mär 02:45 _5t7r.fdx -rwxr-xr-x 1 rabauke users 1921 13. Mär 02:45 _5t7r.fnm -rwxr-xr-x 1 rabauke users 7678976 17. Mär 23:22 _6e91.fdt -rwxr-xr-x 1 rabauke users 0 17. Mär 23:22 _6e91.fdx -rwxr-xr-x 1 rabauke users 1323 17. Mär 23:22 _6e91.fnm -rwxr-xr-x 1 rabauke users 8087552 18. Mär 14:24 _6mi2.fdt -rwxr-xr-x 1 rabauke users 0 18. Mär 14:24 _6mi2.fdx -rwxr-xr-x 1 rabauke users 1546 18. Mär 14:24 _6mi2.fnm -rwxr-xr-x 1 rabauke users 8692736 19. Mär 00:41 _6mte.fdt -rwxr-xr-x 1 rabauke users 0 19. Mär 00:41 _6mte.fdx -rwxr-xr-x 1 rabauke users 1869 19. Mär 00:41 _6mte.fnm -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f0 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f1 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f10 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f11 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f12 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f13 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f14 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f15 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f16 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f17 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f18 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f19 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f2 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f20 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f21 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f22 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f23 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f24 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f25 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f26 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f27 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f28 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f3 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f4 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f5 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f6 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f7 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f8 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f9 -rwxr-xr-x 1 rabauke users 182079903 27. Apr 22:23 _anrn.cfs -rwxr-xr-x 1 rabauke users 5371 9. Mai 15:46 _anrn.del -rwxr-xr-x 1 rabauke users 147809849 30. Apr 09:21 _b8gn.cfs -rwxr-xr-x 1 rabauke users 1301 6. Mai 13:19 _b8gn.del -rwxr-xr-x 1 rabauke users 129083655 3. Mai 13:41 _bilf.cfs -rwxr-xr-x 1 rabauke users 1321 6. Mai 13:25 _bilf.del -rwxr-xr-x 1 rabauke users 69023589 6. Mai 20:13 _c55j.cfs -rwxr-xr-x 1 rabauke users 1342 9. Mai 13:48 _c55j.del -rwxr-xr-x 1 rabauke users 32779532 8. Mai 23:07 _cn18.cfs -rwxr-xr-x 1 rabauke users 1272 9. Mai 13:47 _cn18.del -rwxr-xr-x 1 rabauke users 52888 8. Mai 23:07 _cn57.cfs -rwxr-xr-x 1 rabauke users 22 8. Mai 23:07 _cn57.del -rwxr-xr-x 1 rabauke users 121120 8. Mai 23:07 _cn92.cfs -rwxr-xr-x 1 rabauke users 22 8. Mai 23:07 _cn92.del -rwxr-xr-x 1 rabauke users 106492 9. Mai 00:58 _cncw.cfs -rwxr-xr-x 1 rabauke users 21 9. Mai 10:05 _cncw.del -rwxr-xr-x 1 rabauke users 8425045 9. Mai 12:27 _cnj0.cfs -rwxr-xr-x 1 rabauke users 21 9. Mai 13:47 _cnj0.del -rwxr-xr-x 1 rabauke users 50353 9. Mai 12:29 _cnmq.cfs -rwxr-xr-x 1 rabauke users 22 9. Mai 13:48 _cnmq.del -rwxr-xr-x 1 rabauke users 45725 9. Mai 12:32 _cnqe.cfs -rwxr-xr-x 1 rabauke users 22 9. Mai 13:48 _cnqe.del -rwxr-xr-x 1 rabauke users 8665983 9. Mai 13:48 _cnu7.cfs -rwxr-xr-x 1 rabauke users 22 9. Mai 16:38 _cnu7.del -rwxr-xr-x 1 rabauke users 6601 9. Mai 15:46 _cnuj.cfs -rwxr-xr-x 1 rabauke users 10225 9. Mai 15:46 _cnuu.cfs -rwxr-xr-x 1 rabauke users 10 9. Mai 16:38 _cnuu.del -rwxr-xr-x 1 rabauke users 5256 9. Mai 15:55 _cnvd.cfs -rwxr-xr-x 1 rabauke users 10 9. Mai 15:55 _cnvd.del -rwxr-xr-x 1 rabauke users 4751 9. Mai 15:55 _cnvo.cfs -rwxr-xr-x 1 rabauke users 22812 9. Mai 15:55 _cnvz.cfs -rwxr-xr-x 1 rabauke users 10 9. Mai 16:38 _cnvz.del -rwxr-xr-x 1 rabauke users 24545 9. Mai 16:38 _cnwb.cfs -rwxr-xr-x 1 rabauke users 830 9. Mai 16:38 _cnwd.cfs -rwxr-xr-x 1 rabauke users 4 9. Mai 16:38 deletable -rw------- 1 rabauke users 49 10. Mai 14:01 .directory -rwxr-xr-x 1 rabauke users 220 9. Mai 16:38 segments
The repos seem roughly the same size, statement-wise. Thus, nothing is stored twice, at least not in the RDF db. Please run the following command to see if it helps with the size: qdbus org.kde.NepomukStorage /nepomukstorage org.kde.nepomuk.Storage.optimize main
(In reply to comment #7) > qdbus org.kde.NepomukStorage /nepomukstorage org.kde.nepomuk.Storage.optimize > main Issued this on the 1.1 GB folder and it went down to ~750, yet one day later it was back to 830 and I had to run the above again which seems like a bug since it wastes space if the user does not interfere. Further, the actual size the index should have and stay at is below 600 MB since letting nepomuk recreate its index leads to a size of 540 MB. I tried that again today. If I had not run the optimize command nepomuk would have wasted 600 MB with that command it still wastes 110 MB and even more after a day.
This is the folder of the index after the omptimisation finished: insgesamt 549296 drwxr-xr-x 2 rabauke users 4096 23. Mai 13:39 . drwxr-xr-x 4 rabauke users 4096 18. Feb 16:59 .. -rwxr-xr-x 1 rabauke users 6212608 10. Mär 13:32 _5kg5.tmp -rwxr-xr-x 1 rabauke users 4784128 13. Mär 02:45 _5t7r.fdt -rwxr-xr-x 1 rabauke users 0 13. Mär 02:45 _5t7r.fdx -rwxr-xr-x 1 rabauke users 1921 13. Mär 02:45 _5t7r.fnm -rwxr-xr-x 1 rabauke users 7678976 17. Mär 23:22 _6e91.fdt -rwxr-xr-x 1 rabauke users 0 17. Mär 23:22 _6e91.fdx -rwxr-xr-x 1 rabauke users 1323 17. Mär 23:22 _6e91.fnm -rwxr-xr-x 1 rabauke users 8087552 18. Mär 14:24 _6mi2.fdt -rwxr-xr-x 1 rabauke users 0 18. Mär 14:24 _6mi2.fdx -rwxr-xr-x 1 rabauke users 1546 18. Mär 14:24 _6mi2.fnm -rwxr-xr-x 1 rabauke users 8692736 19. Mär 00:41 _6mte.fdt -rwxr-xr-x 1 rabauke users 0 19. Mär 00:41 _6mte.fdx -rwxr-xr-x 1 rabauke users 1869 19. Mär 00:41 _6mte.fnm -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f0 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f1 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f10 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f11 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f12 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f13 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f14 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f15 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f16 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f17 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f18 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f19 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f2 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f20 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f21 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f22 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f23 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f24 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f25 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f26 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f27 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f28 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f3 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f4 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f5 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f6 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f7 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f8 -rwxr-xr-x 1 rabauke users 61 23. Mär 14:38 _6ofy.f9 -rwxr-xr-x 1 rabauke users 526269586 23. Mai 13:29 _cyhp.cfs -rwxr-xr-x 1 rabauke users 2107 23. Mai 13:39 _cyht.cfs -rwxr-xr-x 1 rabauke users 4 23. Mai 13:39 deletable -rw------- 1 rabauke users 49 10. Mai 14:01 .directory -rwxr-xr-x 1 rabauke users 40 23. Mai 13:39 segments
Databases need space, especially when they are changed a lot which is the case here. With KDE 4.4 we use Virtuoso which also uses its fair share of hard disk space. No way to change that. Sorry. We do not use sesame2 or clucene anymore. As a result there is also no optimization anymore.