Bug 359038

Summary:	kde-frameworks/baloo-5.18.0 - index is truncated back to 0 Bytes when it reaches 4096 MB
Product:	[Frameworks and Libraries] frameworks-baloo	Reporter:	David Kredba <kredba>
Component:	general	Assignee:	Pinak Ahuja <pinak.ahuja>
Status:	RESOLVED DUPLICATE
Severity:	normal	CC:	aspotashev, christoph, dennis.lissov, kdebugs, pinak.ahuja, stupor_scurvy343, theivorytower
Priority:	NOR
Version:	5.18.0
Target Milestone:	---
Platform:	Gentoo Packages
OS:	Linux
Latest Commit:		Version Fixed In:
Sentry Crash Report:

Description David Kredba 2016-02-05 18:26:06 UTC

I am running into this for past few months. Indexing goes fine until Baloo index database grows close to 4 GiB and then it "truncates" it and all the search and file content indexing data are gone.

Before:
balooctl status        
Baloo File Indexer is running
Indexer state: Idle
Indexed 107651 / 556631 files
Current size of index is 3,95 GiB

After:
balooctl status
Baloo File Indexer is running
Indexer state: Idle
Indexed 107651 / 556631 files
Current size of index is 29,06 MiB

Baloo_file_extractor:
ls -l /proc/4215/fd
celkem 0
lr-x------ 1 rrr rrr 64  5. úno 19.02 0 -> 'pipe:[25292]'
l-wx------ 1 rrr rrr 64  5. úno 19.02 1 -> 'pipe:[25293]'
lrwx------ 1 rrr rrr 64  5. úno 19.02 10 -> /home/rrr/.local/share/baloo/index
lrwx------ 1 rrr rrr 64  5. úno 19.02 11 -> /home/rrr/.local/share/baloo/index
lr-x------ 1 rrr rrr 64  5. úno 19.02 12 -> anon_inode:inotify
lr-x------ 1 rrr rrr 64  5. úno 19.02 13 -> /proc/4215/mounts
lrwx------ 1 rrr rrr 64  5. úno 19.02 14 -> 'socket:[24037]'
lrwx------ 1 rrr rrr 64  5. úno 19.02 15 -> /home/rrr/.local/share/baloo/index-lock
lrwx------ 1 rrr rrr 64  5. úno 19.02 16 -> /home/rrr/.local/share/baloo/index
lrwx------ 1 rrr rrr 64  5. úno 19.02 17 -> /home/rrr/.local/share/baloo/index
lr-x------ 1 rrr rrr 64  5. úno 19.02 19 -> /usr/share/mime/mime.cache
l-wx------ 1 rrr rrr 64  5. úno 19.02 2 -> 'pipe:[25294]'
lrwx------ 1 rrr rrr 64  5. úno 19.02 3 -> 'socket:[27160]'
lrwx------ 1 rrr rrr 64  5. úno 19.02 4 -> 'anon_inode:[eventfd]'
lrwx------ 1 rrr rrr 64  5. úno 19.02 5 -> 'anon_inode:[eventfd]'
lrwx------ 1 rrr rrr 64  5. úno 19.02 6 -> 'socket:[25908]'
lr-x------ 1 rrr rrr 64  5. úno 19.02 7 -> /dev/urandom
lrwx------ 1 rrr rrr 64  5. úno 19.02 8 -> 'socket:[25911]'
lrwx------ 1 rrr rrr 64  5. úno 19.02 9 -> 'socket:[25912]'

Baloo_file:
ls -l /proc/3753/fd
celkem 0
lr-x------ 1 rrr rrr 64  5. úno 19.05 0 -> 'pipe:[6806]'
l-wx------ 1 rrr rrr 64  5. úno 19.05 1 -> /home/rrr/.xsession-errors
lrwx------ 1 rrr rrr 64  5. úno 19.05 10 -> /home/rrr/.local/share/baloo/index
lrwx------ 1 rrr rrr 64  5. úno 19.05 11 -> /home/rrr/.local/share/baloo/index
lr-x------ 1 rrr rrr 64  5. úno 19.05 13 -> anon_inode:inotify
lrwx------ 1 rrr rrr 64  5. úno 19.05 14 -> 'anon_inode:[eventfd]'
l-wx------ 1 rrr rrr 64  5. úno 19.05 15 -> 'pipe:[25292]'
lr-x------ 1 rrr rrr 64  5. úno 19.05 16 -> '/usr/share/mime/mime.cache (deleted)'
lr-x------ 1 rrr rrr 64  5. úno 19.05 17 -> 'pipe:[25293]'
lr-x------ 1 rrr rrr 64  5. úno 19.05 19 -> 'pipe:[25294]'
l-wx------ 1 rrr rrr 64  5. úno 19.05 2 -> /home/rrr/.xsession-errors
lr-x------ 1 rrr rrr 64  5. úno 19.05 23 -> 'pipe:[25296]'
l-wx------ 1 rrr rrr 64  5. úno 19.05 24 -> 'pipe:[25296]'
lrwx------ 1 rrr rrr 64  5. úno 19.05 3 -> 'anon_inode:[eventfd]'
lrwx------ 1 rrr rrr 64  5. úno 19.05 4 -> 'socket:[7672]'
lrwx------ 1 rrr rrr 64  5. úno 19.05 5 -> 'socket:[7673]'
lr-x------ 1 rrr rrr 64  5. úno 19.05 6 -> anon_inode:inotify
lr-x------ 1 rrr rrr 64  5. úno 19.05 7 -> /proc/3753/mounts
lrwx------ 1 rrr rrr 64  5. úno 19.05 8 -> 'socket:[7680]'
lrwx------ 1 rrr rrr 64  5. úno 19.05 9 -> /home/rrr/.local/share/baloo/index-lock

No Baloo related information in .xsession_errors file.

cat .config/baloofilerc 
[General]
dbVersion=2
exclude filters=CMakeCache.txt,lzo,*.omf,CMakeTmp,confstat,moc_*.cpp,*.m4,*.elc,*.pc,*.o,*.aux,core-dumps,*.moc,cmake_install.cmake,libtool,*.la,config.status,.hg,.git,.xsession-errors*,*.po,.svn,ui_*.h,Makefile.am,autom4te,*~,*.rej,*.gmo,*.swap,.bzr,CTestTestfile.cmake,*.nvram,.pch,*.csproj,*.lo,CMakeTmpQmake,*.vm*,*.rcore,qrc_*.cpp,confdefs.h,po,*.loT,CMakeFiles,.obj,*.orig,*.class,lost+found,__pycache__,_darcs,conftest,CVS,.uic,*.part,*.pyc,*.tmp,.moc,.histfile.*,litmain.sh
exclude filters version=2
first run=false

How can I debug this please?




Reproducible: Always

Comment 1 David Kredba 2016-02-05 18:31:44 UTC

free
              total        used        free      shared  buff/cache   available
Mem:        8169780      685688     5291612       10956     2192480     6712044
Swap:       8396796      955516     7441280

Linux srv5 4.4.1-gentoo #1 SMP PREEMPT Wed Feb 3 10:46:09 CET 2016 x86_64 Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz GenuineIntel GNU/Linux

There is the index file of size 4446269440 Bytes present in .local/share/baloo after I killed the baloo_file_extractor process and stopped baloo using balooctl stop.

Comment 2 Bob Johnson 2016-05-06 16:09:16 UTC

Same here, but I think it might be a 'balooctl status' display bug. It appears that the index is still there and searchable on my system:

~ $ balooctl status
Baloo File Indexer is running
Indexer state: Suspended
Indexed 458732 / 506012 files
Current size of index is 301.14 MiB

~/.local/share/baloo $ ls -al
total 4502692
drwxr-xr-x  2 bjohnson users       4096 May  6 08:12 .
drwx------ 46 bjohnson users       4096 May  5 14:57 ..
-rw-r--r--  1 bjohnson users 4610740224 May  6 08:16 index
-rw-r--r--  1 bjohnson users       8192 May  6 08:58 index-lock

kinfocenter:
Gentoo Linux
KDE Plasma Version: 5.5.5
Qt Version: 5.5.1
Kernel Version: 4.4.6-gentoo
OS Type: 64-bit

Like David reports, once the index hits 4G 'balooctl status' reports 0MiB. However, search is still working on my system.

Comment 3 Hao Zhang 2016-06-19 00:39:00 UTC

The reason for this bug is that the size is stored in a "quint32" variable, and the maximum value a quint32 variable can take is 2^32, which equals 4096 MB.

Comment 4 Christoph Cullmann 2016-09-11 13:53:51 UTC


*** This bug has been marked as a duplicate of bug 364475 ***