Bug 113730 - Crash when applying pipe through filters
Summary: Crash when applying pipe through filters
Status: RESOLVED FIXED
Alias: None
Product: kmail
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Compiled Sources Linux
: NOR crash with 20 votes (vote)
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
: 115206 116593 116887 117726 117802 119646 (view as bug list)
Depends on:
Blocks:
 
Reported: 2005-10-02 19:44 UTC by Xaos
Modified: 2007-09-14 12:17 UTC (History)
7 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
log created by valgrind (284.70 KB, text/plain)
2005-10-02 19:46 UTC, Xaos
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Xaos 2005-10-02 19:44:29 UTC
Version:            (using KDE Devel)
Installed from:    Compiled sources
Compiler:          gcc 4.0.2 
OS:                Linux

If bogofilter or spamassassin filters are present in the filter list kmail crashes upon receiving messages or applying filters manually ("Classify as spam" and "Classify as Non-spam" filters are OK and do not cause this error.) The bogofilter and spamassassin filters of which I speak are the ones generated automatically by the spam wizard in Kmail. When those are removed, filtering works fine with no crashes.

SpamAssassin version 3.0.4
  running on Perl version 5.8.7

bogofilter version 0.95.2

OS is Suse 10.0 RC1
Comment 1 Xaos 2005-10-02 19:46:28 UTC
Created attachment 12811 [details]
log created by valgrind

log created by "valgrind -v --num-callers=50 kmail --nofork 2>&1 | tee log "
Comment 2 Andreas Gungl 2005-10-04 20:12:04 UTC
Does "compiled sources" mean KDE 3.5 branch?
Can you provide a backtrace?
Any usefull console output?
Can you try to get a log of the filtering process (see Tools menu), which is unlikely when a crash occurs?
Comment 3 Xaos 2005-10-05 01:59:46 UTC
Below is the backtrace, though I read elsewhere that this may not be of much help. That's why I supplied the output from valgrind as seen in the attachment below my initial post. I can't view the filter log as the entire program crashes as soon as the filter is applied and it doesn't seem to be saved anywhere else.

Sorry about the compiled sources. I was sure I had selected RPMs. (of course I was also sure I had not misspelled "Crash" in the title as well.) Anyhow, the rpms are from the 3.5 unstable branch. kdepim3-3.4.91-1.1

(no debugging symbols found)
Using host libthread_db library "/lib/tls/libthread_db.so.1".
(no debugging symbols found)
`system-supplied DSO at 0xffffe000' has disappeared; keeping its symbols.
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
[New Thread 1104754624 (LWP 16175)]
[New Thread 1125018544 (LWP 22004)]
[Thread debugging using libthread_db enabled]
[New Thread 1104754624 (LWP 16175)]
[New Thread 1125018544 (LWP 22004)]
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
[New Thread 1104754624 (LWP 16175)]
[New Thread 1125018544 (LWP 22004)]
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
[KCrash handler]
#4  0x427eff21 in KMFilterActionExtFilter::processAsync ()
   from /opt/kde3/lib/libkmailprivate.so
#5  0x427ef999 in KMail::ActionScheduler::actionMessage ()
   from /opt/kde3/lib/libkmailprivate.so
#6  0x427f072f in KMail::ActionScheduler::filterMessage ()
   from /opt/kde3/lib/libkmailprivate.so
#7  0x427f1028 in KMail::ActionScheduler::qt_invoke ()
   from /opt/kde3/lib/libkmailprivate.so
#8  0x40ff2a09 in QObject::activate_signal ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#9  0x40ff2eb0 in QObject::activate_signal ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#10 0x413368f9 in QTimer::timeout () from /usr/lib/qt3/lib/libqt-mt.so.3
#11 0x41016d0f in QTimer::event () from /usr/lib/qt3/lib/libqt-mt.so.3
#12 0x40f92d01 in QApplication::internalNotify ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#13 0x40f93679 in QApplication::notify () from /usr/lib/qt3/lib/libqt-mt.so.3
#14 0x40c9e78e in KApplication::notify () from /opt/kde3/lib/libkdecore.so.4
#15 0x40f87a78 in QEventLoop::activateTimers ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#16 0x40f40f0f in QEventLoop::processEvents ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#17 0x40fa9bf9 in QEventLoop::enterLoop () from /usr/lib/qt3/lib/libqt-mt.so.3
#18 0x40fa9af6 in QEventLoop::exec () from /usr/lib/qt3/lib/libqt-mt.so.3
#19 0x40f9264f in QApplication::exec () from /usr/lib/qt3/lib/libqt-mt.so.3
#20 0x0805aabb in ?? ()
#21 0xbff67590 in ?? ()
#22 0x00000001 in ?? ()
#23 0x00000001 in ?? ()
#24 0x00000000 in ?? ()
#25 0x00000000 in ?? ()
#26 0x00000001 in ?? ()
#27 0x08068100 in typeinfo name for QToolTip ()
#28 0x00000000 in ?? ()
#29 0x080676c4 in typeinfo name for QToolTip ()
#30 0x080676b0 in typeinfo name for QToolTip ()
#31 0x00000021 in ?? ()
#32 0x41664890 in main_arena () from /lib/tls/libc.so.6
#33 0x00000004 in ?? ()
#34 0x00000020 in ?? ()
#35 0x415ac201 in _int_malloc () from /lib/tls/libc.so.6
#36 0x4155ce60 in __libc_start_main () from /lib/tls/libc.so.6
#37 0x08058181 in ?? ()
Comment 4 Andreas Gungl 2005-10-07 07:05:53 UTC
Can you try a simpler filter script (or even a simple "cat") to find out if the External Filter action works in general?
It's somehow strange, I'm using the KDE 3.5 beta packages for Suse 9.3 on my work system, and I can use spamassassin in a filter without any problem.
Comment 5 Xaos 2005-10-08 02:18:17 UTC
All filters work fine except any that use the "Pipe through" comand. If I set a filter to pipe to any command and that filter matches an incoming message, kmail crashes.
Comment 6 Xaos 2005-10-08 02:20:11 UTC
I might also mention that I get the same results on both 32 bit and 64 bit architectures when using their respective packages.
Comment 7 Andreas Gungl 2005-10-10 22:02:00 UTC
Well, I have three different installations: KDE 3.5 branch completely self-compiled, KDE 3.5 Beta 1 RPMs for Suse 9.3, and kdepim module from KDE 3.5 branch compiled against and running on Suse 9.3 with KDE 3.4.2 RPMs.

All three installations have different account types and storage formats. All behave well with SpamAssassin (Perl scrip and daemon mode using spamc/spamd).

So we have to find a way to make your crash reproducible on a developer's machine if you can't compile kdkepim with debug information and use a debugger.
Can you check, if filtering to external programs works different for:
- applying filters manually with Ctrl-J
- apply filters on mail check
both on local folders, folders of a disconnected as well as an online IMAP account (if available on your system)?

What system do you use? - hardware, gcc (you mentioned gcc 4.0), Linux distribution, provider of the KDE RPMs ...
If time permits and Beta 2 is out, can you check the newer version?

That's a lot to try and to answer, but I'm afraid there's no other way out.
Comment 8 Andreas Gungl 2005-10-27 18:21:51 UTC
*** Bug 115206 has been marked as a duplicate of this bug. ***
Comment 9 Don Sanders 2005-11-09 08:57:23 UTC
Xaos can you still reproduce this?
Comment 10 Xaos 2005-11-10 04:46:49 UTC
Yes. Even with kdepim3-3.4.92-1. I switched to evolution until 3.5 is final. I don't have any more useful output from the crash other than what I've already posted here.

BTW my distro is SUSE 10.0 Final, the RPMs are from the chuck.ucs.indiana.edu mirror.

My computer is a HP nx9600 laptop, 64-bit 3.4 ghz P4 CPU, 512 MB ram.
This crash also happens on a 32bit 2.4 ghz P4 with 1 GB of RAM and SUSE 9.3, although I have not tried on the latter of the two with Beta 2 of KDE 3.4, only beta 1.
Comment 11 Jan Huelsbergen 2005-11-14 21:56:54 UTC
Still happens with kmail 1.9 from KDE 3.5 RC1. Using archlinux, KDE binaries provided by distribution, compiled with gcc 4.0.3.
Comment 12 Thiago Macieira 2005-11-22 23:18:31 UTC
*** Bug 116887 has been marked as a duplicate of this bug. ***
Comment 13 Andreas Gungl 2005-11-23 11:53:09 UTC
I've tested filters which pipe the messages through external programs for all account types. Everything worked fine, I've never got a crash. The external filter tools were cat, spamassassin, spamc, and bogofilter.
(BTW I'm currently running Suse 10.0 with updated KDE RPMs from the supplementary tree - currently KDE 3.5 RC.)

Anyway, I see no chance to fix the crashes which happen on your systems unless you can provide backtraces with full debug information. I.e. you need to compile the KDE PIM package after having configured it as -enable-debug=full. If you can provide such backtraces or if you even can run KMail in gdb, that would be the only chance to do something for you.
If you cannot provide more data, I will close the report in a few days as most people can work with pipe-through filters in KMail without any problem. As soon as you have new information, you can still re-open the report.
Comment 14 Derek Broughton 2005-11-23 15:35:37 UTC
If 116887 really is a duplicate of this - I presume Thiago must have been able to tell that from the backtrace - then it _only_ started to happen for me with 1.9, using the KDE 3.5 RC packages from kubuntu.org
Comment 15 Craig Bradney 2005-12-01 23:00:33 UTC
(no debugging symbols found)
Using host libthread_db library "/lib/libthread_db.so.1".
(no debugging symbols found)
`system-supplied DSO at 0xffffe000' has disappeared; keeping its symbols.
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
[New Thread 16384 (LWP 23116)]
[KCrash handler]
#5  0xb55b851d in KMFilterActionExtFilter::processAsync (this=0x8be9580, 
    aMsg=0x8bfd2d0) at kmfilteraction.cpp:1700
#6  0xb55783bb in KMail::ActionScheduler::actionMessage (this=0x8c26918, 
    res=KMFilterAction::GoOn) at actionscheduler.cpp:611
#7  0xb56313be in KMail::ActionScheduler::filterMessage (this=0x8c26918)
    at actionscheduler.cpp:587
#8  0xb5631654 in KMail::ActionScheduler::qt_invoke (this=0x8c26918, _id=16, 
    _o=0xbfa4fc10) at actionscheduler.moc:207
#9  0xb6595e74 in QObject::activate_signal () from /usr/qt/3/lib/libqt-mt.so.3
#10 0xb6596592 in QObject::activate_signal () from /usr/qt/3/lib/libqt-mt.so.3
#11 0xb68ea5d9 in QTimer::timeout () from /usr/qt/3/lib/libqt-mt.so.3
#12 0xb65b7b2e in QTimer::event () from /usr/qt/3/lib/libqt-mt.so.3
#13 0xb653322f in QApplication::internalNotify ()
   from /usr/qt/3/lib/libqt-mt.so.3
#14 0xb65333cc in QApplication::notify () from /usr/qt/3/lib/libqt-mt.so.3
#15 0xb6cb636b in KApplication::notify (this=0xbfa502c0, receiver=0x8c14100, 
    event=0xbfa50050) at kapplication.cpp:550
#16 0xb65269dc in QEventLoop::activateTimers ()
   from /usr/qt/3/lib/libqt-mt.so.3
#17 0xb64dfb71 in QEventLoop::processEvents ()
   from /usr/qt/3/lib/libqt-mt.so.3
#18 0xb6549930 in QEventLoop::enterLoop () from /usr/qt/3/lib/libqt-mt.so.3
#19 0xb6549886 in QEventLoop::exec () from /usr/qt/3/lib/libqt-mt.so.3
#20 0xb65323df in QApplication::exec () from /usr/qt/3/lib/libqt-mt.so.3
#21 0x0806179f in ?? ()
#22 0xbfa502c0 in ?? ()
#23 0x0806b58e in typeinfo name for QPtrList<KParts::Part> ()
#24 0x00000000 in ?? ()
#25 0x00000000 in ?? ()
#26 0x00000000 in ?? ()
#27 0x00000001 in ?? ()
#28 0x0806be64 in typeinfo name for QPtrList<KParts::Part> ()
#29 0x00000000 in ?? ()
#30 0x0806ad88 in typeinfo name for QPtrList<KParts::Part> ()
#31 0x0806b3e2 in typeinfo name for QPtrList<KParts::Part> ()
#32 0xbfa502c8 in ?? ()
#33 0xb5e51cb1 in malloc () from /lib/libc.so.6
#34 0xb5e023b3 in __libc_start_main () from /lib/libc.so.6
#35 0x08059851 in ?? ()

Does this look the same? I get it with kmail 1.9 from kde 3.5 release on Gentoo 2005.1. It doesn't happen when I remove the spamassassin filters.
Comment 16 Craig Bradney 2005-12-01 23:01:54 UTC
Oh, I forgot, the folders are all IMAP folders, if that matters, on a local courier server.
Comment 17 Eyolf Østrem 2005-12-02 18:56:12 UTC
I have the same problem, the same symptoms, and the same install as has been mentioned in previous posts: KDE 3.5, archlinux, and crashes both with spamassassin and bogomail, when message filters are run on imap accounts (but  I think also on the local folders, with ctrl-J). I haven't specifically checked that it is the pipe through... filter that is the offender, but it sounds reasonable (the wizard-generated icon for manual marking of spam works fine). 
Comment 18 Andreas Gungl 2005-12-04 17:04:47 UTC
*** Bug 116593 has been marked as a duplicate of this bug. ***
Comment 19 Andreas Gungl 2005-12-04 17:09:15 UTC
I eventually can reproduce the bug (after having explicitely enabled the action scheduler in my kmailrc).
I already have a patch ready to fix that issue in the 3.5 branch.
Comment 20 Andreas Gungl 2005-12-04 19:08:50 UTC
SVN commit 485514 by gungl:

fix the crash when using pipe-through filters in combination
with the actionscheduler
BUG:113730


 M  +22 -22    kmcommands.cpp  
 M  +2 -2      kmkernel.cpp  


--- branches/KDE/3.5/kdepim/kmail/kmcommands.cpp #485513:485514
@@ -1496,30 +1496,30 @@
 
 void KMMetaFilterActionCommand::start()
 {
-#if 0 // use action scheduler
-  KMFilterMgr::FilterSet set = KMFilterMgr::All;
-  QPtrList<KMFilter> filters;
-  filters.append( mFilter );
-  ActionScheduler *scheduler = new ActionScheduler( set, filters, mHeaders );
-  scheduler->setAlwaysMatch( true );
-  scheduler->setAutoDestruct( true );
+  if (ActionScheduler::isEnabled() ) {
+    // use action scheduler
+    KMFilterMgr::FilterSet set = KMFilterMgr::All;
+    QValueList<KMFilter*> filters;
+    filters.append( mFilter );
+    ActionScheduler *scheduler = new ActionScheduler( set, filters, mHeaders );
+    scheduler->setAlwaysMatch( true );
+    scheduler->setAutoDestruct( true );
 
-  int contentX, contentY;
-  HeaderItem *nextItem = mHeaders->prepareMove( &contentX, &contentY );
-  QPtrList<KMMsgBase> msgList = *mHeaders->selectedMsgs(true);
-  mHeaders->finalizeMove( nextItem, contentX, contentY );
+    int contentX, contentY;
+    HeaderItem *nextItem = mHeaders->prepareMove( &contentX, &contentY );
+    QPtrList<KMMsgBase> msgList = *mHeaders->selectedMsgs(true);
+    mHeaders->finalizeMove( nextItem, contentX, contentY );
 
-
-  for (KMMsgBase *msg = msgList.first(); msg; msg = msgList.next())
-    scheduler->execFilters( msg );
-#else
-  KMCommand *filterCommand = new KMFilterActionCommand( mMainWidget,
-  *mHeaders->selectedMsgs(), mFilter);
-  filterCommand->start();
-  int contentX, contentY;
-  HeaderItem *item = mHeaders->prepareMove( &contentX, &contentY );
-  mHeaders->finalizeMove( item, contentX, contentY );
-#endif
+    for (KMMsgBase *msg = msgList.first(); msg; msg = msgList.next())
+      scheduler->execFilters( msg );
+  } else {
+    KMCommand *filterCommand = new KMFilterActionCommand( mMainWidget,
+    *mHeaders->selectedMsgs(), mFilter);
+    filterCommand->start();
+    int contentX, contentY;
+    HeaderItem *item = mHeaders->prepareMove( &contentX, &contentY );
+    mHeaders->finalizeMove( item, contentX, contentY );
+  }
 }
 
 FolderShortcutCommand::FolderShortcutCommand( KMMainWidget *mainwidget,
--- branches/KDE/3.5/kdepim/kmail/kmkernel.cpp #485513:485514
@@ -1432,11 +1432,11 @@
   the_msgIndex = 0;
 #endif
 
-#if 0
+//#if 0
   the_weaver =  new KPIM::ThreadWeaver::Weaver( this );
   the_weaverLogger = new KPIM::ThreadWeaver::WeaverThreadLogger(this);
   the_weaverLogger->attach (the_weaver);
-#endif
+//#endif
 
   connect( the_folderMgr, SIGNAL( folderRemoved(KMFolder*) ),
            this, SIGNAL( folderRemoved(KMFolder*) ) );
Comment 21 Craig Bradney 2005-12-05 23:10:38 UTC
I still have a crash with the patch applied in kmail-3.5.0-r1 on gentoo. I can control J on a non spam mail now, but when I run it on a definite hit, it still sig11s, but the trace is useless.

I have the following with debug symbols built in, and binary stripping off:
kde-base/kdepim-kioslaves
kde-base/kmail
kde-base/kontact
kde-base/libkdepim
kde-base/libkpimidentities
kde-base/kdelibs
Comment 22 Andreas Gungl 2005-12-06 08:27:11 UTC
*** Bug 117726 has been marked as a duplicate of this bug. ***
Comment 23 Andreas Gungl 2005-12-06 08:55:15 UTC
It crashes reproducible on moving a message (in ActionScheduler::moveMessage() / actionescheduler.cpp:658). I'll investigate further.
Comment 24 Andreas Gungl 2005-12-06 10:43:10 UTC
Okay, for those of you who suffer from the crashes, here is an advice about how to work around the problems:

1) Remove an "action-scheduler=true" entry from the [General] section of your kmailrc file if the entry exists.
2) If you're using SpamAssassin, make sure you have a line "report_safe 0" in your ~/.spamassassin/user_prefs file.
3) Never ever touch X-KMail-Filtered headers within your filter actions. Removing that header during the filtering process is a bad idea.
Comment 25 Derek Broughton 2005-12-06 14:36:36 UTC
Thanks for the suggestions, Andreas.  Unfortunately, this affects me even though I'm using bogofilter, not SpamAssassin, don't have an "action-scheduler" entry and never do anything that changes headers (unless bogofilter itself does that).

Is this suggestion on top of applying the 3.5 patch - I'm waiting on a batch of Ubuntu kde 3.5 changes before I try that.
Comment 26 Andreas Gungl 2005-12-06 21:25:58 UTC
I'm still investigating. I've found two critical places in the code. I have local hacks to circumvent the problems, but it's not yet ready for a commit into the 3.5 branch. So you still need to be a bit patient with me.
Comment 27 Andreas Gungl 2005-12-06 21:36:44 UTC
*** Bug 117802 has been marked as a duplicate of this bug. ***
Comment 28 Andreas Gungl 2005-12-06 22:27:54 UTC
SVN commit 486140 by gungl:

Fix some issues in the async filtering code
BUG:113730


 M  +5 -4      actionscheduler.cpp  
 M  +7 -0      kmfilteraction.cpp  


--- branches/KDE/3.5/kdepim/kmail/actionscheduler.cpp #486139:486140
@@ -570,7 +570,7 @@
        (!mAccount ||
 	(mAccount && (*mFilterIt).applyOnAccount(mAccountId)))) ||
       ((mSet & KMFilterMgr::Explicit) && (*mFilterIt).applyOnExplicit())) {
-      
+
       // filter is applicable
     if ( FilterLog::instance()->isLogging() ) {
       QString logText( i18n( "<b>Evaluating filter rules:</b> " ) );
@@ -646,8 +646,11 @@
   if (!orgMsg || !orgMsg->parent()) {
     // Original message is gone, no point filtering it anymore
     mSrcFolder->removeMsg( mSrcFolder->find( msg ) );
+    kdDebug(5006) << "The original serial number is missing. "
+                  << "Cannot complete the filtering." << endl;
     mExecutingLock = false;
     processMessageTimer->start( 0, true );
+    return;
   } else {
     if (!folder) // no filter folder specified leave in current place
       folder = orgMsg->parent();
@@ -659,7 +662,7 @@
   mSrcFolder->addMsg( msg );
   mIgnore = false;
 
-  if (msg && kmkernel->folderIsTrash( folder ))
+  if (msg && folder && kmkernel->folderIsTrash( folder ))
     KMFilterAction::sendMDN( msg, KMime::MDN::Deleted );
 
   timeOutTime = QTime::currentTime();
@@ -722,8 +725,6 @@
 
 void ActionScheduler::copyMessageFinished( KMCommand *command )
 {
-  // FIXME remove the debug output
-  kdDebug(5006) << "##### ActionScheduler::copyMessageFinished( KMCommand *command )" << endl;
   if ( command->result() != KMCommand::OK )
     actionMessage( KMFilterAction::ErrorButGoOn );
   else 
--- branches/KDE/3.5/kdepim/kmail/kmfilteraction.cpp #486139:486140
@@ -1630,7 +1630,14 @@
       pclose(p);
       if ( !ba.isEmpty() ) {
         KPIM::ThreadWeaver::debug (1, "PipeJob::run: %s", QString(ba).latin1() );
+        KMFolder *filterFolder =  mMsg->parent();
         mMsg->fromByteArray( ba );
+        if ( filterFolder ) {
+          filterFolder->take( filterFolder->find( mMsg ) );
+          filterFolder->addMsg( mMsg );
+        } else {
+          kdDebug(5006) << "Warning: Cannot refresh the message from the external filter." << endl;
+        }
       }
 
       KPIM::ThreadWeaver::debug (1, "PipeJob::run: done.\n" );
Comment 29 Craig Bradney 2005-12-08 08:43:42 UTC
Working fine here now! YAY! Thanks Andreas.
Comment 30 Andreas Gungl 2006-01-09 09:05:27 UTC
*** Bug 119646 has been marked as a duplicate of this bug. ***