425017 2020-08-05 01:11:10 +0000 baloo_file_extractor crashes in a loop trying to index one particular file 2020-08-06 05:39:31 +0000 1 1 3 Frameworks and Libraries frameworks-baloo Baloo File Daemon unspecified Other Linux RESOLVED FIXED https://bugs.kde.org/show_bug.cgi?id=414318 https://bugs.kde.org/show_bug.cgi?id=419043 https://bugs.kde.org/show_bug.cgi?id=413694 NOR crash --- 0 nate stefan.bruens ilgaz kde 5.74.0 0 oldest_to_newest 1948900 0 130644 nate 2020-08-05 01:11:10 +0000 Created attachment 130644 The file that makes it crash All KDE software from git master. I'm seeing baloo_file_extractor crashing in a loop while trying to index a particular file of mine, which I am attaching. I saw this happen last week and I purged and rebuilt the baloo index. But it started happening again, for the exact same file! There seems to be something about the file that makes baloo very unhappy. (master) balooctl monitor Press ctrl+c to stop monitoring File indexer is running Indexing file content Indexing: /home/nate/SpiderOak Hive/Work & Money/Blue Systems/This week's work.txt: Ok Indexing: /home/nate/SpiderOak Hive/Work & Money/Blue Systems/This week's work.txt: Ok Indexing: /home/nate/SpiderOak Hive/Work & Money/Blue Systems/This week's work.txt: Ok Indexing: /home/nate/SpiderOak Hive/Work & Money/Blue Systems/This week's work.txt: Ok Indexing: /home/nate/SpiderOak Hive/Work & Money/Blue Systems/This week's work.txt: Ok Indexing: /home/nate/SpiderOak Hive/Work & Money/Blue Systems/This week's work.txt: Ok Indexing: /home/nate/SpiderOak Hive/Work & Money/Blue Systems/This week's work.txt: Ok balooctl index "/home/nate/SpiderOak Hive/Work & Money/Blue Systems/This week's work.txt" Skipping: /home/nate/SpiderOak Hive/Work & Money/Blue Systems/This week's work.txt Reason: Already scheduled for indexing File(s) indexed balooctl clear "/home/nate/SpiderOak Hive/Work & Money/Blue Systems/This week's work.txt" kf.baloo.engine: MTimeDB::del 0 109792322002158340 MDB_BAD_TXN: Transaction must abort, has a child, or is invalid kf.baloo.engine: Transaction::commit MDB_BAD_TXN: Transaction must abort, has a child, or is invalid The backtrace is unfortunately not very helpful: bt [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/home/nate/kde/usr/bin/baloo_file_extractor'. Program terminated with signal SIGABRT, Aborted. #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 50 return ret; (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00007fa1cfdff539 in __GI_abort () at abort.c:79 #2 0x00007fa1d037ac27 in qt_message_fatal (message=<synthetic pointer>..., context=...) at global/qlogging.cpp:1914 #3 QMessageLogger::fatal (this=this@entry=0x7fffbded0770, msg=msg@entry=0x7fa1d0e96f05 "%s") at global/qlogging.cpp:893 #4 0x00007fa1d09cd6d4 in init_platform (argv=<optimized out>, argc=@0x7fffbded097c: 1, platformThemeName=..., platformPluginPath=..., pluginNamesWithArguments=...) at ../../include/QtCore/../../src/corelib/tools/qarraydata.h:208 #5 QGuiApplicationPrivate::createPlatformIntegration (this=0x1e5b430) at kernel/qguiapplication.cpp:1481 #6 0x00007fa1d09cdb60 in QGuiApplicationPrivate::createEventDispatcher ( this=<optimized out>) at kernel/qguiapplication.cpp:1498 #7 0x00007fa1d059a696 in QCoreApplicationPrivate::init ( this=this@entry=0x1e5b430) at kernel/qcoreapplication.cpp:852 #8 0x00007fa1d09d0aaf in QGuiApplicationPrivate::init ( this=this@entry=0x1e5b430) at kernel/qguiapplication.cpp:1527 #9 0x00007fa1d10f1eb9 in QApplicationPrivate::init (this=0x1e5b430) at kernel/qapplication.cpp:513 #10 0x000000000040a291 in main (argc=<optimized out>, argv=0x7fffbded09c0) at /home/nate/kde/src/baloo/src/file/extractor/main.cpp:27 1948916 1 nate 2020-08-05 03:21:05 +0000 Eventually I can get it to stop crashing for a while by disabling the file indexer, killing all baloo* processes, and then enabling it again. 1948920 2 stefan.bruens 2020-08-05 03:47:25 +0000 I don't think its the file itself causing the error, but a somewhat deterministic chain of events (file index operations) which has somehow brought the DB into a bad state. 1948921 3 stefan.bruens 2020-08-05 03:55:04 +0000 Can you try https://invent.kde.org/frameworks/baloo/-/merge_requests/4/diffs 1948956 4 nate 2020-08-05 05:03:20 +0000 Thanks, that seems to fix it for me! 1949088 5 stefan.bruens 2020-08-05 17:53:10 +0000 Git commit fdc162f3f72eaafec2dd8609470ec0002d5c9517 by Stefan Brüns. Committed on 05/08/2020 at 17:47. Pushed by bruns into branch 'master'. [Engine] Propagate transaction errors In case a transaction can not be commited, exit the extractor process. If the commit only failed because it is too large the extractor is started again with half the batch. If the batch only contains a single file and marking it as failed is not possible we are stuck, so also exit from the main process. M +6 -3 src/engine/transaction.cpp M +1 -1 src/engine/transaction.h M +4 -1 src/file/extractor/app.cpp M +8 -1 src/file/extractorprocess.cpp M +6 -1 src/file/filecontentindexer.cpp M +2 -2 src/file/filecontentindexerprovider.cpp M +1 -1 src/file/filecontentindexerprovider.h https://invent.kde.org/frameworks/baloo/commit/fdc162f3f72eaafec2dd8609470ec0002d5c9517 1949089 6 stefan.bruens 2020-08-05 17:55:55 +0000 @Nate - can you check if the file is now in the failed list (balooctl failed), or indexed (balooshow <file>)? 1949093 7 nate 2020-08-05 17:58:27 +0000 The file is not in the failed list and appears to be indexed now: balooshow "/home/nate/SpiderOak Hive/Work & Money/Blue Systems/This week's work.txt" 109792322002158340 66308 25563017 /home/nate/SpiderOak Hive/Work & Money/Blue Systems/This week's work.txt Mtime: 1596589388 2020-08-04T19:03:08 Ctime: 1596589388 2020-08-04T19:03:08 Cached properties: Line Count: 104 1949139 8 stefan.bruens 2020-08-05 21:27:24 +0000 Anything else in the failed list? If not, then splitting up the batch/transaction apparently helped, or the batches were just different after restarting baloo_file. 1949144 9 nate 2020-08-05 21:58:03 +0000 The failed list is empty. 1949149 10 nate 2020-08-05 22:34:53 +0000 So should we call this fixed? 1949162 11 stefan.bruens 2020-08-06 02:21:29 +0000 The symptoms are cured, but the cause still exists. But as I have no idea how to find the cause, lets call it done. 1949164 12 stefan.bruens 2020-08-06 02:23:17 +0000 *** Bug 414318 has been marked as a duplicate of this bug. *** 1949166 13 stefan.bruens 2020-08-06 02:24:14 +0000 *** Bug 419043 has been marked as a duplicate of this bug. *** 1949181 14 cfeck 2020-08-06 05:39:31 +0000 5.73.0 is already tagged, see https://community.kde.org/Schedules/Frameworks 130644 2020-08-05 01:11:10 +0000 2020-08-05 01:11:10 +0000 The file that makes it crash This week's work.txt text/plain 3806 nate IyBYWFhYWFggIwotIFhYWFhYWC1yZWxhdGVkIHBhdGNoIHJldmlldyBhbmQgUUEgZm9yIEtTeXNH dWFyZFFNTCBhbmQgV2F5bGFuZAoKCiMgU2lnbmlmaWNhbnQgd29yayBsYW5kZWQgIwotIExvdHMg b2YgUGxhc21hIFFBIGFuZCBidWcgcHJpb3JpdGl6YXRpb24gYWN0aXZpdGllcy4gVHJpYWdlZCBh bGwgbmV3IGJ1ZyByZXBvcnRzIGV2ZXJ5IGRheS4KLSBPcHRpb24gdG8gY3ljbGUgdGhyb3VnaCBn cm91cGVkIHRhc2sgbWFuYWdlciB0YXNrcyBvbiBhY3RpdmF0aW9uOiBodHRwczovL2ludmVudC5r ZGUub3JnL3BsYXNtYS9wbGFzbWEtZGVza3RvcC8tL21lcmdlX3JlcXVlc3RzLzg0Ci0gUmVzcGVj dCBpY29uIGZvciBjaGVja2JveGVzOiBodHRwczovL2ludmVudC5rZGUub3JnL2ZyYW1ld29ya3Mv cGxhc21hLWZyYW1ld29yay8tL21lcmdlX3JlcXVlc3RzLzQ2Ci0gRml4IG1pc3BsYWNlZCBzbGlk ZXIgYmFja2dyb3VuZHMgaW4gRWxpc2E6IGh0dHBzOi8vaW52ZW50LmtkZS5vcmcvbXVsdGltZWRp YS9lbGlzYS8tL21lcmdlX3JlcXVlc3RzLzEzOAoKCiMgVGhpbmdzIEkgd291bGQgcmVhbGx5IGFw cHJlY2lhdGUgYmVpbmcgcmV2aWV3ZWQgIwotIERyb3AgaW1wbGljaXQgbWluaW11bSBzaXppbmcg ZnJvbSBQQzIgYW5kIFBDMyBCdXR0b25zOiBodHRwczovL2ludmVudC5rZGUub3JnL2ZyYW1ld29y a3MvcGxhc21hLWZyYW1ld29yay8tL21lcmdlX3JlcXVlc3RzLzU3Ci0gRGlzYW1iaWd1YXRlIHNl bGVjdGVkIHZzIGhvdmVyZWQgc3RhdGVzIGZvciBHcmlkRGVsZWdhdGU6IGh0dHBzOi8vaW52ZW50 LmtkZS5vcmcvZnJhbWV3b3Jrcy9rZGVjbGFyYXRpdmUvLS9tZXJnZV9yZXF1ZXN0cy83Ci0gW1Bs YXNtYUNvbXBvbmVudHMzXSBTdHJpcCBtbmVtb25pYyBhbXBlcnNhbmRzIGZyb20gdG9vbHRpcCB0 ZXh0OiBodHRwczovL2ludmVudC5rZGUub3JnL2ZyYW1ld29ya3MvcGxhc21hLWZyYW1ld29yay8t L21lcmdlX3JlcXVlc3RzLzUxCi0gU3RhcnQgcG9ydGluZyBrZGVwbGFzbWEtYWRkb25zIHRvIFBD MzogaHR0cHM6Ly9pbnZlbnQua2RlLm9yZy9wbGFzbWEva2RlcGxhc21hLWFkZG9ucy8tL21lcmdl X3JlcXVlc3RzLzEyCi0gUG9ydCBwbGFzbWEtcGstdXBkYXRlcyB0byBMYXlvdXRzLCBQbGFjZWhv bGRlck1lc3NhZ2UsIGFuZCBtb3N0bHkgdG8gUGxhc21hQ29tcG9uZW50czM6IGh0dHBzOi8vaW52 ZW50LmtkZS5vcmcvc3lzdGVtL3BsYXNtYS1way11cGRhdGVzLy0vbWVyZ2VfcmVxdWVzdHMvMQoK CiMgV2Vla2x5IGJsb2cgcG9zdCAjCi0gaHR0cHM6Ly9wb2ludGllc3RzdGljay5jb20vMjAyMC8w Ny8yNC90aGlzLXdlZWstaW4ta2RlLXNjcmVlbmNhc3RpbmctYW5kLXNoYXJlZC1jbGlwYm9hcmQt b24td2F5bGFuZC8KCgojIE90aGVyICMKTGVhcm5lZCBhYm91dCB0aGUgR2FyZGVuaW5nIChpLmUu IG1hbmFnZW1lbnQpIGluaXRpYXRpdmUgYW5kIGRlY2lkZWQgdG8gZm9ybWFsbHkgam9pbiBhbmQg dHJ5IHRvIHJlaGFiaWxpdGF0ZSBpdC4gSSByZXdyb3RlIGl0cyBvbGQgd2lraSBwYWdlIChodHRw czovL2NvbW11bml0eS5rZGUub3JnL0dhcmRlbmluZykgYW5kIGxpbmtlZCB0byBpdCBmcm9tIHZh cmlvdXMgcGxhY2VzLgoKCiMgUGxhZ3VlICMKLSBUb28gZGVwcmVzc2luZyB0byBzcGVuZCB0aW1l IHRoaW5raW5nIGFib3V0LgoKCiMgTmV4dCBXZWVrICMKTW9yZSBvZiB0aGUgYWJvdmUuCgoKCgoy MDIwIHByaW9yaXRpZXMKPT09PT09PT09PT09PT09CkRvbmUKLS0tLQotIEFkZCBGVVNFIG1vdW50 IHN1cHBvcnQgdG8gS0lPIGFuZCBEb2xwaGluIGZvciBub24tS0RFIGFwcGxpY2F0aW9ucwoKLSBB ZGQgSVNPIG1vdW50aW5nIHN1cHBvcnQgdG8gRG9scGhpbgoKLSBVc2UgYW4gSWNvbnMtT25seSBU YXNrIE1hbmFnZXIgYnkgZGVmYXVsdAoKLSBIdWdlbHkgaW1wcm92ZSBTeXN0ZW0gVHJheSBVWAoK LSBJbXByb3ZlIHRoZSBkZWZhdWx0IHNlbGVjdGlvbiBvZiB3YWxscGFwZXJzIGFuZCB1c2VyIGF2 YXRhcnMsIGFuZCBlbmNvdXJhZ2UgZGlzdHJvcyB0byBzaGlwIHRoZXNlIGJ5IGRlZmF1bHQKLS0g aHR0cHM6Ly9waGFicmljYXRvci5rZGUub3JnL1QxMDI2OQotLSBodHRwczovL3BoYWJyaWNhdG9y LmtkZS5vcmcvVDk5MTAKCgpJbiBwcm9ncmVzcwotLS0tLS0tLS0tLQoKLSBGaW5pc2ggdXAgYW5k IGxhbmQgUG9saWN5S2l0LWluLUtJTwoKLSBNYWtlIHRoZSBCcmVlemUgU3BsYXNoIHNjcmVlbiBz aG93IHRoZSBjdXJyZW50ICJOZXh0IiB3YWxscGFwZXIgZm9yIGltcHJvdmVkIGNvbnRpbnVpdHkK Ci0gTWFrZSBhbGwgY2F0ZWdvcnkgaWNvbnMgaW4gS0RFIGFwcHMgYW5kIFBsYXNtYSB1c2UgY29s b3JmdWwgaWNvbnM6IGh0dHBzOi8vcGhhYnJpY2F0b3Iua2RlLm9yZy9UMTAxNjUKCi0gUmVvcmdh bml6ZSBTeXN0ZW0gU2V0dGluZ3MgZm9yIGdyZWF0ZXIgZGlzY292ZXJhYmlsaXR5IGFuZCB1c2Vy LWNlbnRyaWNpdHkKCi0gTWFrZSBLREUgQXBwcyBhZG9wdCBhIGNvaGVzaXZlIGFuZCBtb2Rlcm5p emVkIGRlc2lnbiBsYW5ndWFnZSBhbmQgZXZvbHZlIHRoZSBCcmVlemUgdGhlbWU6IGh0dHBzOi8v cGhhYnJpY2F0b3Iua2RlLm9yZy9UMTA4OTEKLS0gRGVmaW5lIGEgIlRvb2xzIEFyZWEiIG9uIHRv cCBvZiB0aGUgd2luZG93IHRoYXQgYWxsIGhhcyB0aGUgc2FtZSBiYWNrZ3JvdW5kIGNvbG9yIGFu ZCBkZXNhdHVyYXRlcyB3aGVuIHRoZSB3aW5kb3cgaXMgaW5hY3RpdmU6IGh0dHBzOi8vcGhhYnJp Y2F0b3Iua2RlLm9yZy9UMTAyMDEKCi0gQWRvcHQgc21vb3RoIGFuZCBpbmVydGlhbCBzY3JvbGxp bmcgdGhyb3VnaG91dCBQbGFzbWEgYW5kIEtERSBhcHBzCgotIEltcHJvdmUgbXVsdGktc2NyZWVu IHN1cHBvcnQKCi0gTWFrZSBBcHBsaWNhdGlvbiBEYXNoYm9hcmQgc2hvdyB3aW5kb3dzIGFuZCB2 aXJ0dWFsIGRlc2t0b3BzIGxpa2UgbWFjT1MgTWlzc2lvbiBDb250cm9sIGFuZCBHTk9NRSdzIEFj dGl2aXRpZXMgT3ZlcnZpZXcKCi0gTW92ZSB0byBhbiAiSW5zdGFudCBBcHBseSIgcGFyYWRpZ20g aW4gU3lzdGVtIFNldHRpbmdzIGFuZCBLREUgYXBwcwoKLSBSZXBsYWNlIEtpY2tvZmYgd2l0aCBT aW1wbGVNZW51IGFuZCBwb2xpc2ggdGhlIGhlY2sgb3V0IG9mIGl0CgotIFBvcnQgT2t1bGFyJ3Mg c2V0dGluZ3MgZGlhbG9nIHRvIHVzZSBhIEZvcm1MYXlvdXQKCgpOb3Qgc3RhcnRlZAotLS0tLS0t LS0tLQotIENoYW5nZSB0aGUgZGVmYXVsdCB3aW5kb3cgcGxhY2VtZW50IG1vZGUgdG8gbWFrZSBu ZXcgd2luZG93cyBjZW50ZXJlZCwgYnV0IHdpdGggYWRkaXRpb25hbCB3aW5kb3dzIGNhc2NhZGlu ZywgYW5kIHdpdGggd2luZG93cyBhbHdheXMgcmVtZW1iZXJpbmcgdGhlaXIgcG9zaXRpb25zIHdo ZW4gY2xvc2VkIGFuZCByZS1vcGVuZWQuCgotIERlbGV0ZSB0aGUgc3BsYXNoIHNjcmVlbgoKLSBJ bXBsZW1lbnQgYXV0by1zd2l0Y2gtYXVkaW8tc2lua3Mtb24tbmV3LXNpbmstYXR0YWNobWVudAoK LSBTdG9wIHVzaW5nIHVnbHkgc3ltYm9saWMgc3ltYm9scyBmb3Igc21hbGwgbWltZXR5cGUsIGRl dmljZXMsIGFuZCBwbGFjZXMgaXRlbXM6IGh0dHBzOi8vcGhhYnJpY2F0b3Iua2RlLm9yZy9UMTA4 NzAKCi0gRW5jb3VyYWdlIGRpc3Ryb3MgdG8gc2hpcCBLdXAgYnkgZGVmYXVsdAoKLSBNYWtlIEtN YWlsIG1vcmUgcmVsaWFibGUsIGJlYXV0aWZ1bCwgYW5kIGxlc3MgcmVzb3VyY2UtaW50ZW5zaXZl CgotIFByb2R1Y2UgYSBzeXN0ZW0gc25hcHNob3QgZmVhdHVyZSBsaWtlIFRpbWVTaGlmdCAob3Ig ZW5jb3VyYWdlIEtERSBkaXN0cm9zIHRvIHNoaXAgdGhhdD8KCgoKCgoKCgo=