Bug 113730

Created attachment 12811 [details]
log created by valgrind

log created by "valgrind -v --num-callers=50 kmail --nofork 2>&1 | tee log "

Does "compiled sources" mean KDE 3.5 branch?
Can you provide a backtrace?
Any usefull console output?
Can you try to get a log of the filtering process (see Tools menu), which is unlikely when a crash occurs?

Below is the backtrace, though I read elsewhere that this may not be of much help. That's why I supplied the output from valgrind as seen in the attachment below my initial post. I can't view the filter log as the entire program crashes as soon as the filter is applied and it doesn't seem to be saved anywhere else.

Sorry about the compiled sources. I was sure I had selected RPMs. (of course I was also sure I had not misspelled "Crash" in the title as well.) Anyhow, the rpms are from the 3.5 unstable branch. kdepim3-3.4.91-1.1

(no debugging symbols found)
Using host libthread_db library "/lib/tls/libthread_db.so.1".
(no debugging symbols found)
`system-supplied DSO at 0xffffe000' has disappeared; keeping its symbols.
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
[New Thread 1104754624 (LWP 16175)]
[New Thread 1125018544 (LWP 22004)]
[Thread debugging using libthread_db enabled]
[New Thread 1104754624 (LWP 16175)]
[New Thread 1125018544 (LWP 22004)]
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
[New Thread 1104754624 (LWP 16175)]
[New Thread 1125018544 (LWP 22004)]
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
[KCrash handler]
#4  0x427eff21 in KMFilterActionExtFilter::processAsync ()
   from /opt/kde3/lib/libkmailprivate.so
#5  0x427ef999 in KMail::ActionScheduler::actionMessage ()
   from /opt/kde3/lib/libkmailprivate.so
#6  0x427f072f in KMail::ActionScheduler::filterMessage ()
   from /opt/kde3/lib/libkmailprivate.so
#7  0x427f1028 in KMail::ActionScheduler::qt_invoke ()
   from /opt/kde3/lib/libkmailprivate.so
#8  0x40ff2a09 in QObject::activate_signal ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#9  0x40ff2eb0 in QObject::activate_signal ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#10 0x413368f9 in QTimer::timeout () from /usr/lib/qt3/lib/libqt-mt.so.3
#11 0x41016d0f in QTimer::event () from /usr/lib/qt3/lib/libqt-mt.so.3
#12 0x40f92d01 in QApplication::internalNotify ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#13 0x40f93679 in QApplication::notify () from /usr/lib/qt3/lib/libqt-mt.so.3
#14 0x40c9e78e in KApplication::notify () from /opt/kde3/lib/libkdecore.so.4
#15 0x40f87a78 in QEventLoop::activateTimers ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#16 0x40f40f0f in QEventLoop::processEvents ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#17 0x40fa9bf9 in QEventLoop::enterLoop () from /usr/lib/qt3/lib/libqt-mt.so.3
#18 0x40fa9af6 in QEventLoop::exec () from /usr/lib/qt3/lib/libqt-mt.so.3
#19 0x40f9264f in QApplication::exec () from /usr/lib/qt3/lib/libqt-mt.so.3
#20 0x0805aabb in ?? ()
#21 0xbff67590 in ?? ()
#22 0x00000001 in ?? ()
#23 0x00000001 in ?? ()
#24 0x00000000 in ?? ()
#25 0x00000000 in ?? ()
#26 0x00000001 in ?? ()
#27 0x08068100 in typeinfo name for QToolTip ()
#28 0x00000000 in ?? ()
#29 0x080676c4 in typeinfo name for QToolTip ()
#30 0x080676b0 in typeinfo name for QToolTip ()
#31 0x00000021 in ?? ()
#32 0x41664890 in main_arena () from /lib/tls/libc.so.6
#33 0x00000004 in ?? ()
#34 0x00000020 in ?? ()
#35 0x415ac201 in _int_malloc () from /lib/tls/libc.so.6
#36 0x4155ce60 in __libc_start_main () from /lib/tls/libc.so.6
#37 0x08058181 in ?? ()

Can you try a simpler filter script (or even a simple "cat") to find out if the External Filter action works in general?
It's somehow strange, I'm using the KDE 3.5 beta packages for Suse 9.3 on my work system, and I can use spamassassin in a filter without any problem.

All filters work fine except any that use the "Pipe through" comand. If I set a filter to pipe to any command and that filter matches an incoming message, kmail crashes.

I might also mention that I get the same results on both 32 bit and 64 bit architectures when using their respective packages.

Well, I have three different installations: KDE 3.5 branch completely self-compiled, KDE 3.5 Beta 1 RPMs for Suse 9.3, and kdepim module from KDE 3.5 branch compiled against and running on Suse 9.3 with KDE 3.4.2 RPMs.

All three installations have different account types and storage formats. All behave well with SpamAssassin (Perl scrip and daemon mode using spamc/spamd).

So we have to find a way to make your crash reproducible on a developer's machine if you can't compile kdkepim with debug information and use a debugger.
Can you check, if filtering to external programs works different for:
- applying filters manually with Ctrl-J
- apply filters on mail check
both on local folders, folders of a disconnected as well as an online IMAP account (if available on your system)?

What system do you use? - hardware, gcc (you mentioned gcc 4.0), Linux distribution, provider of the KDE RPMs ...
If time permits and Beta 2 is out, can you check the newer version?

That's a lot to try and to answer, but I'm afraid there's no other way out.

*** Bug 115206 has been marked as a duplicate of this bug. ***

Xaos can you still reproduce this?

Yes. Even with kdepim3-3.4.92-1. I switched to evolution until 3.5 is final. I don't have any more useful output from the crash other than what I've already posted here.

BTW my distro is SUSE 10.0 Final, the RPMs are from the chuck.ucs.indiana.edu mirror.

My computer is a HP nx9600 laptop, 64-bit 3.4 ghz P4 CPU, 512 MB ram.
This crash also happens on a 32bit 2.4 ghz P4 with 1 GB of RAM and SUSE 9.3, although I have not tried on the latter of the two with Beta 2 of KDE 3.4, only beta 1.

Still happens with kmail 1.9 from KDE 3.5 RC1. Using archlinux, KDE binaries provided by distribution, compiled with gcc 4.0.3.

*** Bug 116887 has been marked as a duplicate of this bug. ***

I've tested filters which pipe the messages through external programs for all account types. Everything worked fine, I've never got a crash. The external filter tools were cat, spamassassin, spamc, and bogofilter.
(BTW I'm currently running Suse 10.0 with updated KDE RPMs from the supplementary tree - currently KDE 3.5 RC.)

Anyway, I see no chance to fix the crashes which happen on your systems unless you can provide backtraces with full debug information. I.e. you need to compile the KDE PIM package after having configured it as -enable-debug=full. If you can provide such backtraces or if you even can run KMail in gdb, that would be the only chance to do something for you.
If you cannot provide more data, I will close the report in a few days as most people can work with pipe-through filters in KMail without any problem. As soon as you have new information, you can still re-open the report.

If 116887 really is a duplicate of this - I presume Thiago must have been able to tell that from the backtrace - then it _only_ started to happen for me with 1.9, using the KDE 3.5 RC packages from kubuntu.org

(no debugging symbols found)
Using host libthread_db library "/lib/libthread_db.so.1".
(no debugging symbols found)
`system-supplied DSO at 0xffffe000' has disappeared; keeping its symbols.
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
[New Thread 16384 (LWP 23116)]
[KCrash handler]
#5  0xb55b851d in KMFilterActionExtFilter::processAsync (this=0x8be9580, 
    aMsg=0x8bfd2d0) at kmfilteraction.cpp:1700
#6  0xb55783bb in KMail::ActionScheduler::actionMessage (this=0x8c26918, 
    res=KMFilterAction::GoOn) at actionscheduler.cpp:611
#7  0xb56313be in KMail::ActionScheduler::filterMessage (this=0x8c26918)
    at actionscheduler.cpp:587
#8  0xb5631654 in KMail::ActionScheduler::qt_invoke (this=0x8c26918, _id=16, 
    _o=0xbfa4fc10) at actionscheduler.moc:207
#9  0xb6595e74 in QObject::activate_signal () from /usr/qt/3/lib/libqt-mt.so.3
#10 0xb6596592 in QObject::activate_signal () from /usr/qt/3/lib/libqt-mt.so.3
#11 0xb68ea5d9 in QTimer::timeout () from /usr/qt/3/lib/libqt-mt.so.3
#12 0xb65b7b2e in QTimer::event () from /usr/qt/3/lib/libqt-mt.so.3
#13 0xb653322f in QApplication::internalNotify ()
   from /usr/qt/3/lib/libqt-mt.so.3
#14 0xb65333cc in QApplication::notify () from /usr/qt/3/lib/libqt-mt.so.3
#15 0xb6cb636b in KApplication::notify (this=0xbfa502c0, receiver=0x8c14100, 
    event=0xbfa50050) at kapplication.cpp:550
#16 0xb65269dc in QEventLoop::activateTimers ()
   from /usr/qt/3/lib/libqt-mt.so.3
#17 0xb64dfb71 in QEventLoop::processEvents ()
   from /usr/qt/3/lib/libqt-mt.so.3
#18 0xb6549930 in QEventLoop::enterLoop () from /usr/qt/3/lib/libqt-mt.so.3
#19 0xb6549886 in QEventLoop::exec () from /usr/qt/3/lib/libqt-mt.so.3
#20 0xb65323df in QApplication::exec () from /usr/qt/3/lib/libqt-mt.so.3
#21 0x0806179f in ?? ()
#22 0xbfa502c0 in ?? ()
#23 0x0806b58e in typeinfo name for QPtrList<KParts::Part> ()
#24 0x00000000 in ?? ()
#25 0x00000000 in ?? ()
#26 0x00000000 in ?? ()
#27 0x00000001 in ?? ()
#28 0x0806be64 in typeinfo name for QPtrList<KParts::Part> ()
#29 0x00000000 in ?? ()
#30 0x0806ad88 in typeinfo name for QPtrList<KParts::Part> ()
#31 0x0806b3e2 in typeinfo name for QPtrList<KParts::Part> ()
#32 0xbfa502c8 in ?? ()
#33 0xb5e51cb1 in malloc () from /lib/libc.so.6
#34 0xb5e023b3 in __libc_start_main () from /lib/libc.so.6
#35 0x08059851 in ?? ()

Does this look the same? I get it with kmail 1.9 from kde 3.5 release on Gentoo 2005.1. It doesn't happen when I remove the spamassassin filters.

Oh, I forgot, the folders are all IMAP folders, if that matters, on a local courier server.

I have the same problem, the same symptoms, and the same install as has been mentioned in previous posts: KDE 3.5, archlinux, and crashes both with spamassassin and bogomail, when message filters are run on imap accounts (but  I think also on the local folders, with ctrl-J). I haven't specifically checked that it is the pipe through... filter that is the offender, but it sounds reasonable (the wizard-generated icon for manual marking of spam works fine). 

*** Bug 116593 has been marked as a duplicate of this bug. ***

I eventually can reproduce the bug (after having explicitely enabled the action scheduler in my kmailrc).
I already have a patch ready to fix that issue in the 3.5 branch.

SVN commit 485514 by gungl:

fix the crash when using pipe-through filters in combination
with the actionscheduler
BUG:113730


 M  +22 -22    kmcommands.cpp  
 M  +2 -2      kmkernel.cpp  


--- branches/KDE/3.5/kdepim/kmail/kmcommands.cpp #485513:485514
@@ -1496,30 +1496,30 @@
 
 void KMMetaFilterActionCommand::start()
 {
-#if 0 // use action scheduler
-  KMFilterMgr::FilterSet set = KMFilterMgr::All;
-  QPtrList<KMFilter> filters;
-  filters.append( mFilter );
-  ActionScheduler *scheduler = new ActionScheduler( set, filters, mHeaders );
-  scheduler->setAlwaysMatch( true );
-  scheduler->setAutoDestruct( true );
+  if (ActionScheduler::isEnabled() ) {
+    // use action scheduler
+    KMFilterMgr::FilterSet set = KMFilterMgr::All;
+    QValueList<KMFilter*> filters;
+    filters.append( mFilter );
+    ActionScheduler *scheduler = new ActionScheduler( set, filters, mHeaders );
+    scheduler->setAlwaysMatch( true );
+    scheduler->setAutoDestruct( true );
 
-  int contentX, contentY;
-  HeaderItem *nextItem = mHeaders->prepareMove( &contentX, &contentY );
-  QPtrList<KMMsgBase> msgList = *mHeaders->selectedMsgs(true);
-  mHeaders->finalizeMove( nextItem, contentX, contentY );
+    int contentX, contentY;
+    HeaderItem *nextItem = mHeaders->prepareMove( &contentX, &contentY );
+    QPtrList<KMMsgBase> msgList = *mHeaders->selectedMsgs(true);
+    mHeaders->finalizeMove( nextItem, contentX, contentY );
 
-
-  for (KMMsgBase *msg = msgList.first(); msg; msg = msgList.next())
-    scheduler->execFilters( msg );
-#else
-  KMCommand *filterCommand = new KMFilterActionCommand( mMainWidget,
-  *mHeaders->selectedMsgs(), mFilter);
-  filterCommand->start();
-  int contentX, contentY;
-  HeaderItem *item = mHeaders->prepareMove( &contentX, &contentY );
-  mHeaders->finalizeMove( item, contentX, contentY );
-#endif
+    for (KMMsgBase *msg = msgList.first(); msg; msg = msgList.next())
+      scheduler->execFilters( msg );
+  } else {
+    KMCommand *filterCommand = new KMFilterActionCommand( mMainWidget,
+    *mHeaders->selectedMsgs(), mFilter);
+    filterCommand->start();
+    int contentX, contentY;
+    HeaderItem *item = mHeaders->prepareMove( &contentX, &contentY );
+    mHeaders->finalizeMove( item, contentX, contentY );
+  }
 }
 
 FolderShortcutCommand::FolderShortcutCommand( KMMainWidget *mainwidget,
--- branches/KDE/3.5/kdepim/kmail/kmkernel.cpp #485513:485514
@@ -1432,11 +1432,11 @@
   the_msgIndex = 0;
 #endif
 
-#if 0
+//#if 0
   the_weaver =  new KPIM::ThreadWeaver::Weaver( this );
   the_weaverLogger = new KPIM::ThreadWeaver::WeaverThreadLogger(this);
   the_weaverLogger->attach (the_weaver);
-#endif
+//#endif
 
   connect( the_folderMgr, SIGNAL( folderRemoved(KMFolder*) ),
            this, SIGNAL( folderRemoved(KMFolder*) ) );

I still have a crash with the patch applied in kmail-3.5.0-r1 on gentoo. I can control J on a non spam mail now, but when I run it on a definite hit, it still sig11s, but the trace is useless.

I have the following with debug symbols built in, and binary stripping off:
kde-base/kdepim-kioslaves
kde-base/kmail
kde-base/kontact
kde-base/libkdepim
kde-base/libkpimidentities
kde-base/kdelibs

*** Bug 117726 has been marked as a duplicate of this bug. ***

It crashes reproducible on moving a message (in ActionScheduler::moveMessage() / actionescheduler.cpp:658). I'll investigate further.

Okay, for those of you who suffer from the crashes, here is an advice about how to work around the problems:

1) Remove an "action-scheduler=true" entry from the [General] section of your kmailrc file if the entry exists.
2) If you're using SpamAssassin, make sure you have a line "report_safe 0" in your ~/.spamassassin/user_prefs file.
3) Never ever touch X-KMail-Filtered headers within your filter actions. Removing that header during the filtering process is a bad idea.

Thanks for the suggestions, Andreas.  Unfortunately, this affects me even though I'm using bogofilter, not SpamAssassin, don't have an "action-scheduler" entry and never do anything that changes headers (unless bogofilter itself does that).

Is this suggestion on top of applying the 3.5 patch - I'm waiting on a batch of Ubuntu kde 3.5 changes before I try that.

I'm still investigating. I've found two critical places in the code. I have local hacks to circumvent the problems, but it's not yet ready for a commit into the 3.5 branch. So you still need to be a bit patient with me.

*** Bug 117802 has been marked as a duplicate of this bug. ***

SVN commit 486140 by gungl:

Fix some issues in the async filtering code
BUG:113730


 M  +5 -4      actionscheduler.cpp  
 M  +7 -0      kmfilteraction.cpp  


--- branches/KDE/3.5/kdepim/kmail/actionscheduler.cpp #486139:486140
@@ -570,7 +570,7 @@
        (!mAccount ||
 	(mAccount && (*mFilterIt).applyOnAccount(mAccountId)))) ||
       ((mSet & KMFilterMgr::Explicit) && (*mFilterIt).applyOnExplicit())) {
-      
+
       // filter is applicable
     if ( FilterLog::instance()->isLogging() ) {
       QString logText( i18n( "<b>Evaluating filter rules:</b> " ) );
@@ -646,8 +646,11 @@
   if (!orgMsg || !orgMsg->parent()) {
     // Original message is gone, no point filtering it anymore
     mSrcFolder->removeMsg( mSrcFolder->find( msg ) );
+    kdDebug(5006) << "The original serial number is missing. "
+                  << "Cannot complete the filtering." << endl;
     mExecutingLock = false;
     processMessageTimer->start( 0, true );
+    return;
   } else {
     if (!folder) // no filter folder specified leave in current place
       folder = orgMsg->parent();
@@ -659,7 +662,7 @@
   mSrcFolder->addMsg( msg );
   mIgnore = false;
 
-  if (msg && kmkernel->folderIsTrash( folder ))
+  if (msg && folder && kmkernel->folderIsTrash( folder ))
     KMFilterAction::sendMDN( msg, KMime::MDN::Deleted );
 
   timeOutTime = QTime::currentTime();
@@ -722,8 +725,6 @@
 
 void ActionScheduler::copyMessageFinished( KMCommand *command )
 {
-  // FIXME remove the debug output
-  kdDebug(5006) << "##### ActionScheduler::copyMessageFinished( KMCommand *command )" << endl;
   if ( command->result() != KMCommand::OK )
     actionMessage( KMFilterAction::ErrorButGoOn );
   else 
--- branches/KDE/3.5/kdepim/kmail/kmfilteraction.cpp #486139:486140
@@ -1630,7 +1630,14 @@
       pclose(p);
       if ( !ba.isEmpty() ) {
         KPIM::ThreadWeaver::debug (1, "PipeJob::run: %s", QString(ba).latin1() );
+        KMFolder *filterFolder =  mMsg->parent();
         mMsg->fromByteArray( ba );
+        if ( filterFolder ) {
+          filterFolder->take( filterFolder->find( mMsg ) );
+          filterFolder->addMsg( mMsg );
+        } else {
+          kdDebug(5006) << "Warning: Cannot refresh the message from the external filter." << endl;
+        }
       }
 
       KPIM::ThreadWeaver::debug (1, "PipeJob::run: done.\n" );

Working fine here now! YAY! Thanks Andreas.

*** Bug 119646 has been marked as a duplicate of this bug. ***