Bug 140242 - [PATCH] kmailcvt freezes in the process of importing evolution 2.x mail
Summary: [PATCH] kmailcvt freezes in the process of importing evolution 2.x mail
Status: RESOLVED FIXED
Alias: None
Product: kmail
Classification: Applications
Component: kmailcvt (show other bugs)
Version: unspecified
Platform: Ubuntu Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-01-18 12:42 UTC by Raúl
Modified: 2009-03-19 00:30 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
minimal testcase (1.41 KB, application/octet-stream)
2007-09-04 18:11 UTC, nougad
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Raúl 2007-01-18 12:42:43 UTC
Version:           3 (using KDE KDE 3.5.5)
Installed from:    Ubuntu Packages

Altough the application doesn't crash, this process leaves the application in an unrecoverable state.

I have a .evolution folder of about 63MB. I wanted to convert them to kmail. I chose the evolution 2.x filter, I choose the default folder it took me to /home/user/.evolution/mail/local/ and pressed accept.

The conversion went on, but when it was at some stage of my local folder kmailcvt froze totally. I did import some other folders before this.

I got this backtrace with the help of gdb, HTH:
#0  0xb6f9415a in QStringData::setDirty () from /usr/lib/libqt-mt.so.3
#1  0xb7325eff in QString::setLength () from /usr/lib/libqt-mt.so.3
#2  0xb7326082 in QString::truncate () from /usr/lib/libqt-mt.so.3
#3  0xb7328318 in QString::fromUtf8 () from /usr/lib/libqt-mt.so.3
#4  0xb73527bd in QUtf8Codec::toUnicode () from /usr/lib/libqt-mt.so.3
#5  0xb7328680 in QString::fromLocal8Bit () from /usr/lib/libqt-mt.so.3
#6  0xb72fd202 in QFile::setErrorStringErrno () from /usr/lib/libqt-mt.so.3
#7  0xb72e7355 in QFile::writeBlock () from /usr/lib/libqt-mt.so.3
#8  0x08062ffb in FilterEvolution_v2::importMBox (this=0x8143158, info=0x819e638, mboxName=@0xbf974700, rootDir=@0xbf9746a8, targetDir=@0x806fd0c)
    at /build/buildd/kdepim-3.5.5/./kmailcvt/filter_evolution_v2.cxx:204
#9  0x0806435b in FilterEvolution_v2::import (this=0x8143158, info=0x819e638) at /build/buildd/kdepim-3.5.5/./kmailcvt/filter_evolution_v2.cxx:97
#10 0x080688da in KMailCVT::next (this=0x80fefc8) at /build/buildd/kdepim-3.5.5/./kmailcvt/kmailcvt.cpp:55
#11 0xb73cb3c0 in QWizard::qt_invoke () from /usr/lib/libqt-mt.so.3
#12 0xb787615b in KWizard::qt_invoke () from /usr/lib/libkdeui.so.4
#13 0x08067929 in KMailCVT::qt_invoke (this=0x80fefc8, _id=57, _o=0xbf97486c) at ./kmailcvt.moc:84
#14 0xb700b957 in QObject::activate_signal () from /usr/lib/libqt-mt.so.3
#15 0xb700c3fc in QObject::activate_signal () from /usr/lib/libqt-mt.so.3
#16 0xb73a0f5f in QButton::clicked () from /usr/lib/libqt-mt.so.3
#17 0xb70a9fe8 in QButton::mouseReleaseEvent () from /usr/lib/libqt-mt.so.3
#18 0xb7042729 in QWidget::event () from /usr/lib/libqt-mt.so.3
#19 0xb6fa2b88 in QApplication::internalNotify () from /usr/lib/libqt-mt.so.3
#20 0xb6fa4d46 in QApplication::notify () from /usr/lib/libqt-mt.so.3
#21 0xb76ccdb2 in KApplication::notify () from /usr/lib/libkdecore.so.4
#22 0xb6f353fd in QApplication::sendSpontaneousEvent () from /usr/lib/libqt-mt.so.3
#23 0xb6f34062 in QETWidget::translateMouseEvent () from /usr/lib/libqt-mt.so.3
#24 0xb6f3214c in QApplication::x11ProcessEvent () from /usr/lib/libqt-mt.so.3
#25 0xb6f49320 in QEventLoop::processEvents () from /usr/lib/libqt-mt.so.3
#26 0xb6fbd25e in QEventLoop::enterLoop () from /usr/lib/libqt-mt.so.3
#27 0xb6fbd06e in QEventLoop::exec () from /usr/lib/libqt-mt.so.3
#28 0xb6fa4731 in QApplication::exec () from /usr/lib/libqt-mt.so.3
#29 0x0806aa6e in main (argc=1, argv=0x820cf58) at /build/buildd/kdepim-3.5.5/./kmailcvt/main.cpp:54
#30 0xb7a438cc in __libc_start_main () from /lib/tls/i686/cmov/libc.so.6
#31 0x08050591 in _start ()
Comment 1 Philip Rodrigues 2007-01-19 21:41:00 UTC
Can you find the condition that causes the problem? Is there a particular message that causes kmailcvt to freeze, or does it occur for mailboxes larger than a certain size, or ...?
Comment 2 mauro 2007-04-04 15:05:46 UTC
Same problem for me, importing from Evolution 2.8.3 to Kmail 1.9.6 (KDE 3.5.6-0.3.fc6), kmailcvt Qt: 3.3.7, KDE: 3.5.6-0.3.fc6 Fedora-Core, KMailCVT: 3

It hangs just after the first folder has been imported and the windown must be killed.

Mailboxes where originally imported to evolution from ms outlook (win98).
The structure is some nested and contains some huge attachments
(du of .evolution/mail/local is about 402MB) 

I also tried starting at a branch... it imports a few more but with no header information and then hangs the same way.
Comment 3 nougad 2007-09-03 21:07:02 UTC
I think I have found the solution:

The Problem is the file *.ev-summary-meta in the ./evolution/mail/local/ Directory (or any Subdirectory). 

$ tree -L 1
.
|-- Inbox
|-- Inbox.cmeta
|-- Inbox.ev-summary
|-- Inbox.ev-summary-meta
|-- Inbox.ibex.index
|-- Inbox.ibex.index.data


In all Versions of the File filter_evolution_v2.cxx (tags and trunk Directory) are two If terms which filter the Files:

File: filter_evolution_v2.cxx:
...
void FilterEvolution_v2::import(FilterInfo *info)
{
...
  if (temp_mailfile.endsWith(".cmeta") || temp_mailfile.endsWith(".ev-summary") ||
      temp_mailfile.endsWith(".ibex.index") || temp_mailfile.endsWith(".ibex.index.data")) {}
  else {
      info->addLog( i18n("Start import file %1...").arg( temp_mailfile ) );
                importMBox(info, mailDir + temp_mailfile , temp_mailfile, QString::null);
  }
...
}
...

void FilterEvolution_v2::importDirContents(FilterInfo *info, const QString& dirName, const QString& KMailRootDir, const QString& KMailSubDir)
{
...
  if (temp_mailfile.endsWith(".cmeta") || temp_mailfile.endsWith(".ev-summary") ||
      temp_mailfile.endsWith(".ibex.index") || temp_mailfile.endsWith(".ibex.index.data")) {}
  else {
      info->addLog( i18n("Start import file %1...").arg( temp_mailfile ) );
      importMBox(info, (dirName + "/" + temp_mailfile) , KMailRootDir, KMailSubDir);
  }
...
}
...

The Problem is it excludes only the *.cmeta, *.ev-summary, *.ibex.index and *.ibex.index.data but not *.ev-summary-meta files.
The outcome of this is an infinite loop here:

File: filter_evolution_v2.cxx: (tag 3.5.7-version):
...
void FilterEvolution_v2::importMBox(FilterInfo *info, const QString& mboxName, const QString& rootDir, const QString& targetDir)
{
...
  while ( ! mbox.atEnd() &&  (l = mbox.readLine(input.data(),MAX_LINE)) && ((seperate = input.data()).left(5) != "From ")) {
      tmp.file()->writeBlock( input, l );
  }
...
}

The easiest way is to exclude the *.ev-summary-meta too:

if (temp_mailfile.endsWith(".cmeta") || temp_mailfile.endsWith(".ev-summary") ||
    temp_mailfile.endsWith(".ibex.index") || temp_mailfile.endsWith(".ibex.index.data")
    || temp_mailfile.endsWith(".ev-summary-meta")) {}

I hope this is helpful.
Comment 4 Thomas McGuire 2007-09-03 21:12:32 UTC
Can anybody here please attach a minimal testcase (sample mail which causes kmailcvt to hang)?
Comment 5 Danny Kukawka 2007-09-04 15:07:39 UTC
I take a look at it and test it in the next days. Thanks.
Comment 6 nougad 2007-09-04 18:11:31 UTC
Created attachment 21548 [details]
minimal testcase

Here is a minimal testcase.
I use for this test Evolution 2.10.1 from Ubunut-live-CD 7.04.
Important is the file Drafts.ev-summary-meta.

I only made a new mail and saved it as Draft. Then I clickt in Inbox. Evolution
saves then the file *.ev-summary-meta.
Comment 7 Nicolas Bigaouette 2007-09-14 19:58:34 UTC
I have a similar problem when importing from thunderbird. I begin importing my folders, but at one point it kind of freeze. I have many subfolders (Drafts, Factures, Inbox, Sent, Templates) and it always freeze before reaching the "Sent" folder. So in kmail the "Drafts", "Factures" and "Inbox" are imported. I do not know if all the messages were imported though.

I straced kmailcvt. It seems to enter a kind of loop somewhere :
read(9, "\0\0\0\6kmail\0\0\0\0\20anonymous-12234\0\0\0"..., 46) = 46
unlink("/tmp/kde-nicolas/kmailcvtdE2Qic.tmp") = 0
read(3, 0x6402e4, 4096)                 = -1 EAGAIN (Resource temporarily unavailable)
select(13, [3 4 5 7 9 11 12], [], [], {0, 0}) = 0 (Timeout)
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
stat("/tmp/kde-nicolas/", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
open("/tmp/kde-nicolas/kmailcvtNChdHb.tmp", O_RDWR|O_CREAT|O_EXCL, 0600) = 14
umask(0)                                = 022
umask(022)                              = 0
fchmod(14, 0600)                        = 0
getgid()                                = 1000
getuid()                                = 1000
fchown(14, 1000, 1000)                  = 0
fcntl(14, F_SETFD, FD_CLOEXEC)          = 0
fcntl(14, F_GETFL)                      = 0x8002 (flags O_RDWR|O_LARGEFILE)
fstat(14, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2af9e3892000
lseek(14, 0, SEEK_CUR)                  = 0
fstat(14, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
read(14, "", 4096)                      = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
[...many exactly the same...]
read(13, "tie 1.ZIP\"\r\n\r\nUEsDBBQAAAAIAOCSbC"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
write(14, "From - Fri Apr 09 14:01:30 2004\r"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
read(13, "+gbXvyrYKu7j2FdNqce/qbZPCA7Fc+gU"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
write(14, "uLJ0DqtwmAqfTQlEJFpEgwKJePxyolky"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
read(13, "H2Vn+vrpJtr17QfunGpUtLJ85/H81f+7"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
write(14, "+GYpvW6ABxxW8opotY6phDJUsdO9aGCW"..., 4096) = 4096
[...]
After that it is always this patern repeated:
read(13, "+gbXvyrYKu7j2FdNqce/qbZPCA7Fc+gU"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
write(14, "uLJ0DqtwmAqfTQlEJFpEgwKJePxyolky"..., 4096) = 4096
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
fstat(13, {st_mode=S_IFREG|0755, st_size=29105230, ...}) = 0
with different values in for the read and write, always looks like garbage...

I stoped the strace when the log was about 200MB...

Could it be because the folder in thunderbird was compressed? Or maybe encrypted?
Comment 8 Raúl 2007-09-14 20:50:51 UTC
Nicolas, this bug is about evolution. Your problem may perfectly be totally different. Please check: http://bugs.kde.org/show_bug.cgi?id=117335 and http://bugs.kde.org/show_bug.cgi?id=128699 and if you bugs doesn't match with any of them open a new bug report.

Thanks.
Comment 9 Danny Kukawka 2007-12-10 01:48:13 UTC
SVN commit 746733 by dkukawka:

- fixed variable names
- changed importDirContents() to reuse existing code to import
  mails from the root directory
- ignore *.ev-summary-meta files while import (bug #140242) 

CCBUG: 140242



 M  +24 -27    filter_evolution_v2.cxx  
 M  +1 -1      filter_evolution_v2.hxx  


WebSVN link: http://websvn.kde.org/?view=rev&revision=746733
Comment 10 Danny Kukawka 2007-12-10 01:49:18 UTC
*** Bug has been marked as fixed ***.